Skip to content

Commit bae1b94

Browse files
committed
Remove long integer output.
1 parent 3d1c7de commit bae1b94

File tree

1 file changed

+6
-6
lines changed

1 file changed

+6
-6
lines changed

Doc/tutorial/floatingpoint.rst

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -173,24 +173,24 @@ and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``),
173173
the best value for *N* is 56::
174174

175175
>>> 2**52
176-
4503599627370496L
176+
4503599627370496
177177
>>> 2**53
178-
9007199254740992L
178+
9007199254740992
179179
>>> 2**56/10
180-
7205759403792793L
180+
7205759403792794.0
181181

182182
That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The
183183
best possible value for *J* is then that quotient rounded::
184184

185185
>>> q, r = divmod(2**56, 10)
186186
>>> r
187-
6L
187+
6
188188

189189
Since the remainder is more than half of 10, the best approximation is obtained
190190
by rounding up::
191191

192192
>>> q+1
193-
7205759403792794L
193+
7205759403792794
194194

195195
Therefore the best possible approximation to 1/10 in 754 double precision is
196196
that over 2\*\*56, or ::
@@ -211,7 +211,7 @@ If we multiply that fraction by 10\*\*30, we can see the (truncated) value of
211211
its 30 most significant decimal digits::
212212

213213
>>> 7205759403792794 * 10**30 / 2**56
214-
100000000000000005551115123125L
214+
100000000000000005551115123125
215215

216216
meaning that the exact number stored in the computer is approximately equal to
217217
the decimal value 0.100000000000000005551115123125. Rounding that to 17

0 commit comments

Comments
 (0)