Assembly Homework Assignment

Question

Question:
Write a procedure that performs simple encryption by rotating each plaintext byte a varying number of positions in different directions. For example, in the following array that represents the encryption key, a negative value indicates a rotation to the left and a positive value indicates a rotation to the right. The integer in each position indicates the magnitude of the rotation: key BYTE -2, 4, 1, 0, -3, 5, 2, -4, -4, 6

Code:

Include Irvine32.inc

.data
msg BYTE "Hello", 0
key BYTE -2, 4, 1, 0, -3, 5, 2, -4, -4, 6

.code
main proc
mov ecx, LENGTHOF key   ;Loop Counter
mov edx, OFFSET msg     ;EDX Holds msg and will Display it
mov esi, OFFSET key     ;Point to first array element
mov ebx, 0              ;CMP number


top:
cmp [esi], ebx  ;if esi < ebx
jl ShiftLeft    ;jump to shift left

cmp [esi], ebx  ;if esi > ebx
jg ShiftRight   ;jump to shift right

cmp [esi], ebx
je NoShift

ShiftLeft:
mov cl, [esi]
SHL edx, cl
add esi, TYPE key
loop top

ShiftRight:
mov cl, [esi]
SHR edx, cl
add esi, TYPE key
loop top

NoShift:
add esi, TYPE key
loop top


call WriteString


invoke ExitProcess,0
main endp
end main

So I am having a few issues.
1.the cmp statements are going in reverse. So the first cmp should be cmping -2 and 0. -2 < 0, So it should be taking the jump and going to Shiftleft. However, it is doing the opposite and going to shiftright.
2. Am I incrementing to the next array index properly with the add esi, TYPE key line? 3. Am I understanding the question? I need to rotate my msg, "Hello", either left if the number in the array is negative, by that number, and to the right if the number is positive in the array.

Any help is great and thanks in advance.

Answer 1

I didn't run you code so may analysis may be wrong.

The cmp [esi], ebx do compare [esi] with ebx of course (it is just a [esi]-ebx). The mistake here is that you are comparing DWORDs instead of byte. As a rule of thumb, always specify memory operand size: cmp BYTE PTR [esi], bl this way the assembler can tell you if you are doing wrong.

You made other mistakes as pointed out by @WeatherVan.

I'd like to add that you don't need to do any jump due to the way two complement works and the symmetry of rotations. Remember: jump is expensive arithmetic is not.
If you take the low 3 bit (since you are rotating 8 bit data and 2^3=8) of the key bytes, you can always rotate to the right. A rotation of, say, 2 on the left is a rotation of 6 on the right and 6 is the two complement of 2 in 3 bits.

I also think that you need to repeat the key on the message, i.e. if the message is longer than the key you need to reload it from the start.

Here a sample program in NASM that you can use to better understand my advices

BITS 32

GLOBAL _main

SECTION .data

 msg    db  "Hello ",0
 key    db  -2, 4, 1, 0, -3, 5, 2, -4, -4, 6

SECTION .text

_main:
 mov esi, msg

.load_key:
 mov ebx, 10        ;LENGTHOF key
 mov edi, key

.loop:
 mov cl, BYTE [edi] ;Key
 lodsb          ;Char of message
 test al, al        ;End of string?
 jz .end

 and cl, 07h        ;We are working with 8 bit numbers, 3bits of operand are enough
 ror al, cl     ;Rotate
 mov BYTE [esi-01h], al

 inc edi        ;Next char
 dec ebx        ;Key left       
 jnz .loop
jmp .load_key



.end:
 ret

Answer 2

In your comparison

cmp [esi], ebx

this does not compare [esi] with ebx. It compares ebx with [esi] which is why you think the branch decision is incorrect. As a point of technique, you don't need to test all three conditions, since the last one must be true.

cmp [esi], ebx
je NoShift
jl ShiftRight   ;else fall thru to ShiftLeft

You are correctly moving the pointer to the next array position with

add esi, TYPE key

but you have forgotten to increment the message pointer edx.

I presume that you would encode the character by rotating the byte, not shifting it, and you mistakenly rotate the pointer instead of its target

mov cl, [esi]
rol BYTE PTR [edx], cl    

There is another problem - you are using cx for the loop control, but you are overwriting cl (the l.s. 8 bits of cx) with the bit shift counter. I'll leave you to figure out how to get around that.