Faster Float Printing and Reading in SBCL

In short, on my laptop (x86-64):

• Printing a float with prin1, princ, ~A or ~S is about 5 times faster for numbers of moderate size, and 2-6 times faster on real-world data sets.
• format float directives (~F, ~E, ~G, ~$) are 4-6 times faster, and 5-13 times for small numbers.
• Reading floats with the reader is 1.1-1.8 times faster on real-world data, and about 6 times faster for numbers with large exponents.
• A new function, sb-ext:parse-float, parses a float from a string like parse-integer parses an integer. It skips the reader altogether and is 5-6 times faster than reading floats was before.

Some examples, in nanoseconds per call, before and after:

The gain grows with the size of the exponent. The old printer, Burger-Dybvig from 1996, computes with exact integers, and for a number like 1e-200 those integers have hundreds of digits. Over random bit patterns, which are mostly such extreme numbers, printing a double is 35 times faster. That is not what typical data looks like, though, so the realistic figure is the 2-6x on real data. The real-world numbers come from float-data: geographic coordinates, a star catalogue, weather data, neural-network weights and more.

Everything you normally print or read takes the new fast path. A few rare corners still use the old code, which gives exactly the same answers, it is just not faster. Printing with prin1, princ, ~A or ~S is always fast, for every single and double float on 64-bit platforms. format with a digit count is fast except in three corners: zero decimals for a number below 1, like (format nil "~,0F" 0.4); very high precision for exact powers of two, like (format nil "~,20F" 1.0); and very high precision for subnormals, the tiny floats below about 1e-308.

Reading is fast for numbers of up to 19 significant digits, which covers almost everything, since a double has only about 17 meaningful digits. Longer ones, like 0.10000000000000000000001, use the old code. Also, 32-bit platforms keep the old code everywhere.

The output is unchanged, with one deliberate exception: subnormal floats now print with their true shortest digits, 1.42954e-39 instead of 1.4295402e-39. Both read back as the same float.

The rule is that we have to match exactly what SBCL's original reader and printer produce. The tests compare against the old code itself, loaded from SBCL's own source, or against an exact reference computed with rational arithmetic, never against the new code's own expectations. Some of the checks:

• all 2,139,095,039 positive single-floats printed by zmij and by the old printer: identical, except for the shorter subnormals;
• all single-floats and one billion random doubles printed and read back: every one comes back bit for bit;
• about 20 million checks against Nigel Tao's parse-number test data, a collection of hard cases from several projects' test suites;
• millions of format calls with every combination of directive parameters, against the original functions.

The tests also found a bug in SBCL's reader: numbers with very many digits could be read as a wrong value, (read-from-string "<1000 nines>d0") returned 1.0d251 instead of signalling an error. The fix is now in upstream SBCL.

The patches pass SBCL's own test suite and its CI, including builds hosted by CLISP, CCL, CMUCL and ECL.

It started with Invistra. I used a fork of it to implement a clone of Emacs Lisp's format function in Common Lisp for a pet project, and I noticed that printing floats was slow (issue #8). At that time I was already aware of zmij and of Daniel Lemire's work. I am a frequent reader of his blog and of /r/cpp, so speeding up float printing with zmij stayed in the back of my head as a distant to-do. But I didn't know enough Common Lisp, nor SBCL's internals, to hack at it. A year of learning passed, this summer in particular I was hacking a lot on SBCL. I still don't consider myself an expert, by far not, but I thought I could try to pull this off.

I first tried to port and hack in zmij into Quaviver, which I also got somewhat familiar with when I was hacking on Invistra. It worked nicely, so after that, I ported zmij to SBCL and after zmij, fast_float.

I have given up and started to use LLMs, and I needed something to experiment with. The port of zmij into Quaviver was done with DeepSeek, and finished by hand mostly. DeepSeek's port worked, but the speed was far behind Quaviver's own Schubfach. I had to type-annotate it by hand, fix some constructs, refactor it and inline more. It was done with free, online, web-chat DeepSeek, before their 4.3 update, or whichever version they are at now. DeepSeek told me several times it was not possible to implement zmij in Lisp due to boxing. However, it is possible, and in the end it beats the built-in Schubfach, though only barely, I would even say it is perhaps noise. Quaviver's Schubfach is really good, because they already implement many of the improvements zmij made. What was clear already from that port, from Quaviver's benchmarks, is that SBCL would gain a lot: the difference between zmij in Quaviver and SBCL's built-in printer was dramatic, even when slower than Quaviver's built-in algorithms.

Then I took that finished port out of Quaviver and, with the help of Claude, got it into SBCL, and added the reading side and the format work on top.

Claude seems a bit better at coding overall, but even with Claude Opus 5 there were details I had to fix by hand. There were moments when it did things I didn't want, refactored in a bad way or typed code or text I didn't like. In other words, not everything is shiny; for example, this is how one of the tests looked:

(defun test (x)
  (dolist (w '(nil 1 6 10 15))
    (dolist (d '(nil 0 1 3 8))
      (dolist (e '(nil 1 2 3))
        (dolist (k '(1 0 2 -1 3))
          (dolist (ovf '(nil #\*))
            (dolist (pad '(#\Space #\_))
              (dolist (marker '(nil #\x))
                (dolist (atsign '(nil t))
                  (let ((args (list w d e k ovf pad marker atsign)))
                    ...))))))))))

I certainly wouldn't type that by hand. It did the job, but that is not something one would like to keep and come back to after a while. After a dialogue with AI and some examples shown to it, it now recursively walks a list of parameter values instead:

(defparameter *parameter-values*
  '((nil 1 6 10 15) (nil 0 1 3 8) (nil 1 2 3) (1 0 2 -1 3)
    (nil #\*) (#\Space #\_) (nil #\x) (nil t)))

(defun map-combinations (function value-lists)
  (labels ((walk (lists chosen)
             (if (endp lists)
                 (funcall function chosen)
                 (dolist (value (first lists))
                   (walk (rest lists) (append chosen (list value)))))))
    (walk value-lists '())))

Overall, though, I am satisfied with how it worked out. While I don't think the code is perfect, by any means, there are still atrocities in there, but it is better than I expected it to be. Considering the amount of time it takes to refactor an entire set of tests, I can live with the sub-optimal constructs in code I run occasionally. An entire test is generated in seconds, and so is a refactoring of the entire set of tests.

I have worked on this almost exclusively, in my free time, for about a week, over several iterations. I don't know how many hours exactly, but it wasn't a trivial thing, like press a button and AI just generates the code and texts. It took work. But without the automation from AI, it would have taken weeks for sure, probably months, considering the amount of tests and benchmarks. For a hobby project like this, I think it is OK. The patches themselves are not big, ~1400 SLOC for zmij and ~1000 SLOC for fast_float. That is one afternoon, if you know what you have to type and where. However, figuring out where and what to change takes time. Testing and debugging all of that takes much longer.

I type this because I see these discussions whether AI can replace human programmers, or not, or whether programming has lost the charm with AI or not. For me, AI is like a tractor: if you just turn the key, lower down the plow and let it drive, it will not just fail to do useful work, it will probably do a lot of damage too. However, if you know what you want to do, and steer it in the right direction, you can do a lot of work that would take much longer. At least my perception thus far. Personally, I don't think AI can replace humans, I don't think it even solves problems on its own satisfiyingly. But it can be an accelerator for solving problems. It seems to help with the administration around, exploring, typing, doing lots of menial stuff that take both time and focus. It is also very good at transforming things from one form to another, especially when there are patterns. Converting code from one language to another, or translating seems to be no problem for AI. I have tested with Claude, Gemini and DeepSeek. Each of them speeks all three languages I do. I don't know, it seems useful. Do I think it should be used the way it is, and hyped the way it is by big tech? Definitely not, and there are definitely ethical and socio-economical issues that are real blockers which society has to deal with, in my opinion.

The patches, tests and benchmarks, along with more details and descriptions, are available for review in this repository, one per topic, for anyone who wants to review them or try them out. Comments are welcome, as long as you keep the discussion decent and don't act like a douche.