I'm still don't completely understand overflow flags but from what I've gathered if the two most significant bits are both positive and result in a negative and vice versa, the overflow flag would turn on. But what about in the case of mixed sign binary? My current problem is 70 - -65 which in binary would be
0100 0110
-1011 1111
1000 0111
and I'm assuming in this case the overflow flag would be turned on because 135 is outside of the range -128 to 127. Is this right and would there be a better way to find if the overflow flag is turned on/off?
So we know that a - b = a + (-b) and that with twos complement -b = ~b + 1 so a - b = a + ~b + 1. And that is what the processor is going to do in logic
fill it in
The top line is the carry out/in for each bit and the zero dangling off the left is the carry out of the 8 bit subtraction/addition. Note that some processors invert this into the carry flag calling it a borrow (if the original operation was a subtraction) others simply copy it over into the carry flag and you have to know that it is a not borrow.
We see two things here, the two ways to detect a signed overflow (the carry flag is the unsigned overflow). The easiest way to see a signed overflow is if the carry out and the carry in of the msbit do not match in this case the carry out is a 0 and the carry in is a 1 they do not match, this is a signed overflow, if these bits are considered by you the programmer (the logic does not consider them signed or unsigned, the beauty of twos complement) to be signed then this result does not fit in 8 bits so the result is not correct.
There is a way to detect it from the sign bits and result, but you have to do it like above with the inverted second operand using addition. If the most significant bits of the operands (with b inverted) match each other, but the result does not match then it is a signed overflow. So if we look at this in a truth table type form
a and b are the operands i is the carry in, c is the carry out and r is the result. The two cases where i and c are not equal are where you have a signed overflow. And if you look this rule applies if a = b and r != b then signed overflow.
Those are the two ways to know, you have to invert b though, ones complement not two. Often variables do not have a way to have an extra bit. You would want to use a 16 bit variable in a high level language to do this 8 bit math and you would need to do it twice once with 7 bits and once with 8 bits to see the carry in and carry out. Or you use the if a = b and r!=b for the msbit and not have to do as much work other than do the math with a subtract then use that result the a operand as is and the ones complement of the msbit of b. Well is it really less work? Would have to write it out to see.
And it is safer to say/think "signed overflow" rather than just saying/thinking "overflow", as this form of overflow has to do with addition and subtraction of signed numbers using twos compliment to represent negative numbers. While understanding that the carry out/carry flag is "unsigned overflow" for these operations. And not to be confused with a multiplication overflow if you find logic that does that (and uses the same flag) which would be like 0x40 * 0x40 = 0x1000 but with an 8 bit multiplier that does not output 16 bits 0x40 * 0x40 = 0x00 and it overflowed. Note that not all processors care. Some will support an nbit * nbit = 2*nbit which can handle all combinations, but you then need to have a signed multiply and an unsigned multiply, which is another topic. add and subtract do not have a signed and unsigned version due to the nature of twos complement.