Why does the Lisp reader return `(list 1 2 3)` instead of `(1 2 3)` when using reader macros?

53 views Asked by At

Recently, I tried to understand reader macros better. I have read an article about using reader macros to read in objects in JSON format.

With slightly adapted code from above article (it only reads (or is supposed to read) arrays [1,2,3] into lists (1 2 3))

(defun read-next-object (separator delimiter &optional (input-stream *standard-input*))
  (flet ((peek-next-char ()
           (peek-char t input-stream t nil t))
         (discard-next-char ()
           (read-char input-stream t nil t)))
    (if (and delimiter (char= (peek-next-char) delimiter))
        (progn
          (discard-next-char)
          nil)
        (let* ((object (read input-stream t nil t))
               (next-char (peek-next-char)))
          (cond
            ((char= next-char separator) (discard-next-char))
            ((and delimiter (char= next-char delimiter)) nil)
            (t (error "Unexpected next char: ~S" next-char)))
          object))))

(defun read-separator (stream char)
  (declare (ignore stream))
  (error "Separator ~S shouldn't be read alone" char))

(defun read-delimiter (stream char)
  (declare (ignore stream))
  (error "Delimiter ~S shouldn't be read alone" char))

(defun read-left-bracket (stream char)
  (declare (ignore char))
  (let ((*readtable* (copy-readtable)))
    (set-macro-character #\, 'read-separator)
    (loop
       for object = (read-next-object #\, #\] stream)
       while object
       collect object into objects
       finally (return `(list ,@objects)))))

the intent is to call read on strings and have it produce Lisp lists.

With following test code I get:

(with-input-from-string (stream "[1,2,3]")
  (let ((*readtable* (copy-readtable)))
    (set-macro-character #\[ 'read-left-bracket)
    (set-macro-character #\] 'read-delimiter)
    (read stream)))
;; => (LIST 1 2 3)

I expected to get (1 2 3) instead.

Now, when I change the current readtable "permanently", i.e. call set-macro-character in the top-level, and type [1,2,3] at the prompt, I get indeed (1 2 3).

Why, then, does

(with-input-from-string (stream "(1 2 3)")
  (read stream)))
;; => (1 2 3)

give the "expected" result? What am I missing? Is there some eval hidden, somewhere? (I'm aware of the quasi-quote above, but some in-between reasoning is missing...)

Thanks!

EDIT:

Using

(defun read-left-bracket (stream char)
  (declare (ignore char))
  (let ((*readtable* (copy-readtable)))
    (set-macro-character #\, 'read-separator)
    (loop
       for object = (read-next-object #\, #\] stream)
       while object
       collect object)))

I get what I expect. Entering '[1,2,3] at the REPL behaves like entering "real" lists.

Reading from strings also works as intended.

1

There are 1 answers

3
Rainer Joswig On BEST ANSWER

You have

(return `(list ,@objects))

in your code. Thus [...] is read as (list ...).

Next if you use the REPL and evaluate

> [...]

Then it is as you were evaluating

> (list 1 2 3)

which returns (1 2 3).