x86 cmp Opcode, Pointers and Inline Literals

Question

I am using gdb to examine a program. In assembly, the code is doing:

cmp $0x5, %eax

However, when I examine the contents of %eax, I get: \020\343\377\377\377\177 when examined as a string.

How is \020\343\377\377\377\177 compared to $0x5 in assembly?

Answer 1

cmp, in this case, is comparing the value in eax to a constant 5. The value pointed to by eax, if you think it is in fact a pointer, isn't compared to the constant at all.

The comparison is done by subtraction - in your case, that means 5 is subtracted from the value in eax, and several flags (CF, OF, SF, ZF, AF, and PF, according to the documentation) are set appropriately. Normally the cmp instruction is followed by a conditional instruction of some kind (often a jump), to perform different actions depending on the results of the comparison.

If you tell us the value in eax, rather than interpreting eax as a pointer, I might be able to give you some more information. You can use p $eax or info registers to get the value of eax in gdb.