Home
last modified time | relevance | path

Searched refs:rounding (Results 1 – 25 of 51) sorted by relevance

123

/linux/drivers/staging/media/atomisp/pci/isp/kernels/ctc/ctc2/
H A Dia_css_ctc2.host.c37 int rounding = (dx + 1) >> 1; in ctc2_slope() local
49 rounding = -rounding; in ctc2_slope()
50 slope = (int)(dy_shift + rounding) / dx; in ctc2_slope()
/linux/Documentation/arch/arm/nwfpe/
H A Dtodo.rst41 The IEEE standard defines 4 rounding modes. Round to nearest is the
42 default, but rounding to + or - infinity or round to zero are also allowed.
43 Many architectures allow the rounding mode to be specified by modifying bits
45 the rounding mode one must specify it with each instruction.
49 bits describing the rounding mode. The emulator could be altered to
50 examine a flag, which if set forced it to ignore the rounding mode in
/linux/arch/m68k/fpsp040/
H A Dsint.S29 | the rounding mode.
40 | for the given sign and rounding mode.
78 bfextu FPCR_MODE(%a6){#2:#2},%d1 |use user's mode for rounding
115 movel #1,L_SCR1(%a6) |use rz mode for rounding
133 bfextu FPCR_MODE(%a6){#2:#2},%d1 |use user's mode for rounding
150 | rounding modes. L_SCR1 contains the rmode in the lower byte.
218 movel L_SCR1(%a6),%d1 |use selected rounding mode
H A Dutil.S12 | g_rndpr: returns the rounding precision.
96 | If the instruction is fsgldiv of fsglmul, the rounding precision must be
98 | precision instruction, the rounding precision is then set to the force
170 | ;is the rounding precision
314 | g_rndpr --- put rounding precision in d0{1:0}
322 | get rounding precision (cmdreg3b{6:5})
325 | get destination format - this is the also the rounding precision
350 | is the same as the rounding precision. Pass results from g_dfmtou.
358 movel CMDREG3B(%a6),%d0 |rounding precision in d0{10:9}
359 bfextu %d0{#9:#2},%d0 |move the rounding pre
[all...]
H A Ddecbin.S52 | greater than 10^27, a directed rounding scheme is used with
84 | 10**{1,2,4,8,16,32,64,128,256,512,1024,2048,4096}. No rounding
407 | Pwrten calculates the exponent factor in the selected rounding mode
432 bfextu %d3{#26:#2},%d2 |isolate rounding mode bits
438 moveb (%a1,%d2),%d0 |load new rounding bits from table
440 bfins %d0,%d3{#26:#2} |stuff new rounding bits in FPCR
H A Dround.S4 | handle rounding and normalization tasks
24 | d1(high word) contains rounding precision:
28 | d1(low word) contains rounding mode:
50 | :rounding precision and sets
60 | Use rounding mode as an index into a jump table for these modes.
67 | Jump table indexed by rounding mode in d1.w. All following assumes
130 | selected rounding precision. It is called by the round subroutine
406 | d0: rounding precision
413 | d0 comes into this routine with the rounding precision. It
415 | rounding precisio
[all...]
H A Dstwotox.S199 fmovel %d1,%fpcr | ...set user's rounding mode/precision
303 fmovel %d1,%fpcr | ...set user's rounding mode/precision
H A Dbindec.S57 | 10^^(abs(ISCALE)) using a rounding mode which is a
58 | function of the original rounding mode and the signs
64 | only one rounding error. The grs bits are collected in
78 | Perform FINT operation in the user's rounding mode.
81 | A12. Calculate YINT = FINT(Y) according to user's rounding
333 | the rounding mode as given in the following table (see
440 | only one rounding error. The grs bits are collected in
447 fmovel #rz_mode,%FPCR |set RZ rounding mode
550 | Perform FINT operation in the user's rounding mode. Keep
561 | A12. Calculate YINT = FINT(Y) according to user's rounding mod
[all...]
H A Dx_ovfl.S18 | on the current rounding mode:
H A Dx_unfl.S204 | d1{01:00} has rounding mode
205 | d1{17:16} has rounding precision
H A Dres_func.S180 | the round precision is found in the fpcr. If the rounding precision
194 | FPCR_MODE byte to get rounding on norms and zeros.
301 | to see if rounding should have occurred which would leave the lsb.
476 | the round precision is found in the fpcr. If the rounding precision
487 | FPCR_MODE byte to get rounding.
828 | will be correct for the user's rounding mode and precision. If
830 | and the result will be correct for the user's rounding mode and
1003 | will be correct for the user's rounding mode and precision. If
1005 | and the result will be correct for the user's rounding mode and
1257 | If the rounding precisio
[all...]
/linux/Documentation/arch/arm/vfp/
H A Drelease-notes.rst38 result underflows before rounding, but becomes a normalised number
39 after rounding, we do not signal an underflow exception.
/linux/fs/xfs/
H A Dxfs_bmap_util.c821 xfs_off_t rounding, start, end; in xfs_flush_unmap_range() local
829 rounding = max_t(xfs_off_t, xfs_inode_alloc_unitsize(ip), PAGE_SIZE); in xfs_flush_unmap_range()
830 start = rounddown_64(offset, rounding); in xfs_flush_unmap_range()
831 end = roundup_64(offset + len, rounding) - 1; in xfs_flush_unmap_range()
923 unsigned int rounding; in xfs_prepare_shift() local
944 rounding = xfs_inode_alloc_unitsize(ip); in xfs_prepare_shift()
945 offset = rounddown_64(offset, rounding); in xfs_prepare_shift()
947 offset -= rounding; in xfs_prepare_shift()
/linux/arch/arm/lib/
H A Ddelay-loop.S38 adds r1, r1, #0xffffffff @ rounding up ...
/linux/Documentation/hwmon/
H A Demc2103.rst27 represented, so some rounding is done. With a divider of 1, the lowest
H A Dsmsc47m1.rst71 represented, so some rounding is done. With a divider of 2, the lowest
H A Dlm80.rst56 represented, so some rounding is done. With a divider of 2, the lowest
H A Dlm78.rst57 represented, so some rounding is done. With a divider of 2, the lowest
H A Dlm87.rst51 represented, so some rounding is done. With a divider of 2, the lowest
/linux/drivers/video/fbdev/matrox/
H A Dmatroxfb_base.c453 int rounding; in matroxfb_test_and_set_rounding() local
459 case 4: rounding = 128; in matroxfb_test_and_set_rounding()
461 case 8: rounding = 64; /* doc says 64; 32 is OK for G400 */ in matroxfb_test_and_set_rounding()
463 case 16: rounding = 32; in matroxfb_test_and_set_rounding()
465 case 24: rounding = 64; /* doc says 64; 32 is OK for G400 */ in matroxfb_test_and_set_rounding()
467 default: rounding = 16; in matroxfb_test_and_set_rounding()
470 rounding = 32; in matroxfb_test_and_set_rounding()
474 rounding *= 2; in matroxfb_test_and_set_rounding()
476 over = xres % rounding; in matroxfb_test_and_set_rounding()
478 xres += rounding in matroxfb_test_and_set_rounding()
[all...]
/linux/arch/powerpc/lib/
H A Ddiv64.S32 addc r9,r4,r10 # rounding up (so the estimate cannot
/linux/lib/
H A Dstring_helpers.c57 static const unsigned int rounding[] = { 500, 50, 5 }; in string_get_size() local
117 remainder += rounding[j]; in string_get_size()
/linux/Documentation/userspace-api/media/v4l/
H A Dselection-api-configuration.rst19 control the rounding behaviour using
62 control rounding behaviour using
/linux/drivers/cpufreq/
H A DKconfig322 clock rate rounding through the clock framework can use this
324 harmless for CPUs that don't support rate rounding. The driver
326 itself if the CPU in question is not capable of rate rounding.
/linux/arch/powerpc/boot/
H A Ddiv64.S32 addc r9,r4,r10 # rounding up (so the estimate cannot

123