Div(H) and Eval(H) gives different results

12.3k views Asked by At

I am using data from the same files in two different programs. I tried to get the same results in my new program using an Eval(H) instead of two operations of MULT and DIV(H)

Here are the variable types :

Var1,Var3,Var4,Var5 : 9P 2
Var2                : 9P 4

Original code :

C     Var1           MULT      Var2           Var3
C     Var3           DIV(H)    Var4           Var5

In my new program, I instead tried this :

C                    Eval(H)   Var5 = (Var1 * Var2) / Var4

But there's a .01 difference, i.e. : In the first method Var5 = 4.20 where in the second method Var5 = 4.19.

Any idea how I could correct the second method using only the Eval(H)?

2

There are 2 answers

2
Charles On BEST ANSWER

Without the actual values, it's a bit difficult to illustrate but you're likely running into issues with RPG's default precision rules and the fact you've moved from an explicitly defined intermediate results (ie. Var3) to an implicitly defined one.

With the implicitly defined intermediate results, the precision and scale of (Var 1 * Var2) would be 18P6 vs the 4P2 you defined Var3 at.

The "natural" precision and scale the entire statement is something like 63P44 (assuming I did my math right).

Try

eval(h) var5 = %dec((var1 * Var2):9:2) / var4;

To force the intermediate results back to the 9P2 you had with MULT & DIV

0
Buck Calabro On

When you use DIV, you define the lengths of any intermediate variables (like VAR3). When you use EVAL, the compiler has to store those intermediate values (such as VAR1 * VAR2) somewhere, and it internally defines the length. The RPG Reference manual describes the rules the compiler uses to calculate intermediate results precision. IBM i 7.2 > Programming > ILE languages > RPG > ILE RPG Reference > Operations, Expressions, and Functions > Expressions > Precision Rules for Numeric Operations.