Mod function returns 0 for Matlab

Question

I have a problem with the mod function output in Matlab. I am trying to perform some calculations for ECC double and add algorithm. I am reading data from a file and storing it in a variable and then performing some operations. All works smoothly except that I get 0 in temp1 when I use mod(X2,P). However if I put in values stored in X2(3.0323e+153) and P(1.1579e+77) on command window (mod( 3.0323e+153, 1.1579e+77)), I get the correct values. Can anyone please help me? Below is the part of script which is problematic.

P = hex2dec('FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F');
line = fread(fileID,[1,67],'*char');
while ~feof(fileID)
        PX = line(4:67);
        X = hex2dec(PX);
        X2 = X^2;
        temp1= mod(X2 , P)
    end
    line = fread(fileID,[1,69],'*char');
end
fclose(fileID);

Answer 1

I think the problem lies with how you're initializing P. From the documentation for hex2dec (emphasis mine):

d = hex2dec('hex_value') converts hex_value to its floating-point integer representation. The argument hex_value is a hexadecimal integer stored as text. If the value of hex_value is greater than the hexadecimal equivalent of the value returned by flintmax, then hex2dec might not return an exact conversion.

And the value of flintmax is:

>> flintmax
ans =
     9.007199254740992e+15

Quite a bit smaller than your value for P. In fact, if we use num2hex to look at the two ways you initialize P, you can see a clear difference:

>> P = hex2dec('FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F');
>> num2hex(P)
ans =
4ff0000000000000
>> num2hex(1.1579e+77)
ans =
4fefffda293c30de

As it turns out, the inexact conversion done by hex2dec results in a number that evenly divides into 3.0323e+153, thus giving you a remainder of 0:

>> mod(3.0323e+153, P)
ans =
     0
>> mod(3.0323e+153, 1.1579e+77)
ans =
     8.795697942083107e+76