Home
last modified time | relevance | path

Searched refs:precision (Results 1 – 25 of 50) sorted by relevance

12

/arch/x86/boot/
A Dprintf.c85 if (i > precision) in number()
86 precision = i; in number()
87 size -= precision; in number()
104 while (i < precision--) in number()
171 precision = -1; in vsprintf()
175 precision = skip_atoi(&fmt); in vsprintf()
179 precision = va_arg(args, int); in vsprintf()
181 if (precision < 0) in vsprintf()
182 precision = 0; in vsprintf()
207 len = strnlen(s, precision); in vsprintf()
[all …]
/arch/alpha/boot/
A Dstdio.c82 if (i > precision) in number()
83 precision = i; in number()
84 size -= precision; in number()
101 while (i < precision--) in number()
161 precision = -1; in vsprintf()
165 precision = skip_atoi(&fmt); in vsprintf()
169 precision = va_arg(args, int); in vsprintf()
171 if (precision < 0) in vsprintf()
172 precision = 0; in vsprintf()
204 len = strnlen(s, precision); in vsprintf()
[all …]
/arch/powerpc/boot/
A Dstdio.c118 if (i > precision) in number()
119 precision = i; in number()
120 size -= precision; in number()
137 while (i < precision--) in number()
197 precision = -1; in vsprintf()
201 precision = skip_atoi(&fmt); in vsprintf()
205 precision = va_arg(args, int); in vsprintf()
207 if (precision < 0) in vsprintf()
208 precision = 0; in vsprintf()
240 len = strnlen(s, precision); in vsprintf()
[all …]
/arch/arm/nwfpe/
A Dsoftfloat-specialize71 The pattern for a default generated single-precision NaN.
78 Returns 1 if the single-precision floating-point value `a' is a NaN;
91 Returns 1 if the single-precision floating-point value `a' is a signaling
104 Returns the result of converting the single-precision floating-point NaN
124 precision floating-point format.
163 The pattern for a default generated double-precision NaN.
170 Returns 1 if the double-precision floating-point value `a' is a NaN;
198 Returns the result of converting the double-precision floating-point NaN
218 precision floating-point format.
262 The pattern for a default generated extended double-precision NaN. The
[all …]
A Dfpa11.h30 int8 precision; member
A Dfpa11_cprt.c63 roundData.precision = SetRoundingPrecision(opcode); in PerformFLT()
107 roundData.precision = SetRoundingPrecision(opcode); in PerformFIX()
A Dfpa11_cpdo.c32 roundData.precision = SetRoundingPrecision(opcode); in EmulateCPDO()
/arch/m68k/fpsp040/
A Dutil.S98 | precision instruction, the rounding precision is then set to the force
99 | precision.
109 beql ovff_sgl |force precision is single
111 beql ovff_dbl |force precision is double
149 | The precision is in the fpcr.
170 | ;is the rounding precision
322 | get rounding precision (cmdreg3b{6:5})
332 | use precision from FPCR{7:6}
337 | use precision in FPCR{7:6}
390 | cmdreg3b return precision
[all …]
A Dx_store.S100 | dest_dbl --- write double precision value to user space
104 | a1 -> source in extended precision
110 |Changes extended precision to double precision.
133 subw #0x3fff,%d0 |subtract extended precision bias
136 addw #0x3ff,%d0 |add double precision bias
166 movel #0x8,%d0 |byte count for double precision number
A Dslog2.S19 | to double precision. The result is provably monotonic
20 | in double precision.
35 | traps, and precision control = double extended.
50 | traps, and precision control = double extended.
64 | traps, and precision control = double extended.
79 | traps, and precision control = double extended.
A Dsacos.S15 | result is subsequently rounded to double precision. The
16 | result is provably monotonic in double precision.
A Dstwotox.S16 | result is subsequently rounded to double precision. The
17 | result is provably monotonic in double precision.
199 fmovel %d1,%fpcr | ...set user's rounding mode/precision
303 fmovel %d1,%fpcr | ...set user's rounding mode/precision
A Dres_func.S112 | destination precision and mode.
476 | the round precision is found in the fpcr. If the rounding precision
766 btstl #6,%d0 |test for forced precision
787 | ;round precision/mode. This
831 | precision.
888 lsrl #6,%d1 |put precision in upper word
912 lsrl #6,%d1 |put precision in upper word
938 lsrl #6,%d1 |put precision in upper word
1006 | precision.
1247 | ;round precision/mode. This
[all …]
A Dsasin.S15 | result is subsequently rounded to double precision. The
16 | result is provably monotonic in double precision.
A Dsatanh.S16 | result is subsequently rounded to double precision. The
17 | result is provably monotonic in double precision.
A Dsgetem.S7 | precision number in fp0. sGETEXPD handles denormalized
12 | extended precision number and returned in fp0. The
79 andil #0xffffff00,%d0 |clear rounding precision and mode
A Dscosh.S15 | result is subsequently rounded to double precision. The
16 | result is provably monotonic in double precision.
A Dround.S21 | round --- round result according to precision/mode
24 | d1(high word) contains rounding precision:
50 | :rounding precision and sets
130 | selected rounding precision. It is called by the round subroutine
138 swap %d1 |have d1.w point to round precision
406 | d0: rounding precision
413 | d0 comes into this routine with the rounding precision. It
415 | rounding precision.
425 cmpib #0,%d0 |if 0 then extended precision
436 cmpil #1,%d0 |if 1 then single precision
A Dsetox.S22 | to double precision. The result is provably monotonic in double
23 | precision.
93 | constant := single-precision( 64/log 2 ).
95 | Using a single-precision constant avoids memory access.
96 | Another effect of using a single-precision "constant" is
104 | 3.1 R := X + N*L1, where L1 := single-precision(-log2/64).
105 | 3.2 R := R + N*L2, L2 := extended-precision(-log2/64 - L1).
131 | are single precision; A2 and A3 are double precision.
145 | 2^(J/64) to roughly 85 bits; T is in extended precision
146 | and t is in single precision. Note also that T is rounded
A Dssinh.S15 | result is subsequently rounded to double precision. The
16 | result is provably monotonic in double precision.
A Dbindec.S7 | Converts an input in extended precision format
11 | a0 points to the input extended precision value
33 | value is viewed as 2^^e * 1.f in extended precision.
143 | Constants in extended precision
147 | Constants in single precision
219 | as 2^^e * 1.f in extended precision. This value is stored
553 | mode and precision. The original FPCR is saved in L_SCR1.
613 | in extended precision, so the use of a previous power-of-ten
A Dstanh.S15 | result is subsequently rounded to double precision. The
16 | result is provably monotonic in double precision.
/arch/x86/math-emu/
A DREADME113 (a) the operands have a higher precision than the current setting of the
114 precision control (PC) flags.
242 precision of the argument x; e.g. an argument of pi/2-(1e-10) which is
265 later) for 'float' variables (24 bit precision numbers) when precision
267 bit precision numbers) when precision control is set to 53 bits (a
269 variables when precision control is set to 64 bits).
273 for pi which is accurate to more than 128 bits precision. As a
289 give much degraded precision. For example, the integer number
292 example) of this number to 64 bits precision it would be necessary to
293 have a value of pi which had about 150 bits precision. The FPU
[all …]
/arch/m68k/ifpsp060/
A Dfplsp.doc79 There are 3 entry-points for each instruction type: single precision,
80 double precision, and extended precision.
83 extended precision operand if program executes:
104 are passed in single precision format.
/arch/m68k/ifpsp060/src/
A Dfpsp.S1722 # underflow can happen for extended precision. extended precision opclass
6148 # d0 = round precision,mode #
6157 # in double precision. #
6699 # d0 = round precision,mode #
6708 # in double precision. #
7023 # precision. #
10183 # result precision is extended.
10271 # precision indicated. #
13212 # d0 = round precision/mode #
13426 # d0 = rnd precision/mode #
[all …]

Completed in 62 milliseconds

12