Searched refs:PRECISION (Results 1 – 6 of 6) sorted by relevance
30 PRECISION = 5, enumerator387 scaledReal.full = GetReal(A) * uPow(10, PRECISION-1); /* DOUBLE CHECK THISSSS!!! */ in ConvertBackToInteger()517 int dec[PRECISION]; in uGetScaledDecimal()520 for (i = 0; i < PRECISION; i++) { in uGetScaledDecimal()524 scaledDecimal = scaledDecimal + dec[i]*uPow(10, PRECISION - 1 - i); in uGetScaledDecimal()
49 #define PRECISION 8 macro
865 header_write(ctx, buffer, offset, 1, PRECISION); in encode_frame_header()
292 |--NOTE THAT B4 THROUGH B8 ARE STORED IN DOUBLE PRECISION294 |--AND IS THEREFORE STORED AS SINGLE PRECISION.
5133 #--NOTE THAT B4 THROUGH B8 ARE STORED IN DOUBLE PRECISION5135 #--AND IS THEREFORE STORED AS SINGLE PRECISION.
5186 #--NOTE THAT A3 THROUGH A7 ARE STORED IN DOUBLE PRECISION
Completed in 57 milliseconds