Summary

I was trying to use Lem as a programmatically controllable Common Lisp editor/IDE from an AI agent, similar to using Emacs with emacsclient --eval. After testing both the GUI/WebView build and the headless server, I found that the GUI version works for this use case, but the headless lem-server --mode websocket path appears incomplete or undocumented for reading editor/display output programmatically.

The main blocker is that lem-server accepts WebSocket JSON-RPC connections and responds to basic requests like login and ping/pong, but it does not push the bulk display updates that the GUI/WebView version sends. This makes it impossible to read screen text or REPL output from the headless server.

Environment

• macOS 26.3.1, Apple Silicon

• SBCL 2.6.4

• Quicklisp beta

• Lem nightly build: lem-macos.zip, dated 2026-05-12

• Lem source checkout at: /Users/kat/REPOS/lem

• Tested binaries/modes:

  • Lem.app / WebView GUI
  • lem-server --mode websocket --port <port>
  • locally built Lem source

Most important issues

1. lem-server --mode websocket does not push bulk display content

The headless server responds to WebSocket connection setup, login, and ping/pong, but it does not appear to emit the bulk notifications containing display operations such as put, clear-eol, clear-eob, or cursor movement.

This is the main blocker for programmatic integration because the client cannot read REPL output, buffer contents, prompts, or screen text.

Expected behavior:

lem-server --mode websocket --port NNNN

After a WebSocket login and redraw/input events, the server should emit display updates similar to the GUI/WebView version, including bulk notifications with put operations.

Actual behavior:

login works
ping/pong works
input appears accepted
bulk display content is never emitted

The GUI/WebView version does emit this content correctly on localhost:64265.

2. There is no documented supported way to evaluate Lisp forms headlessly and retrieve results

The desired use case is:

lem-client --eval '(+ 1 2 3)'

or equivalent JSON-RPC/WebSocket behavior:

{
  "method": "eval",
  "params": {
    "form": "(+ 1 2 3)"
  }
}

Expected result:

{
  "result": "6"
}

At the moment, the only working route I found is to drive the GUI/WebView frontend, open M-x start-lisp-repl, send input events, and scrape the resulting display updates from bulk WebSocket messages. That works, but it is fragile and requires the GUI process.

3. Address-in-use error when starting Swank on port 4006

I also hit a startup/runtime failure when Lem tried to create a Swank server on port 4006.

Observed error from the backtrace:

Socket error in "bind": EADDRINUSE (Address already in use)
...
SB-BSD-SOCKETS:SOCKET-ERROR :ERRNO 48 :SYSCALL "bind"
...
SWANK:CREATE-SERVER :PORT 4006 :DONT-CLOSE T

It would be helpful if Lem either:

• detected this and chose another port,
• showed a clearer user-facing message,
• documented how to configure the Swank port,
• or avoided hard-failing editor startup when the port is already occupied.

Other protocol issues encountered

These may be documentation issues rather than bugs, but they caused significant debugging time.

4. WebSocket client-to-server frames must be masked

Client frames that are not masked are silently dropped. This is correct per RFC 6455, but it would help if Lem’s WebSocket protocol documentation explicitly mentioned this.

5. Extended payload lengths must be handled

Login responses can exceed 125 bytes and require the WebSocket extended length marker 126 with a two-byte length field. Treating 126 as a literal payload length corrupts subsequent reads.

6. JSON-RPC keys appear to be case-sensitive and silently dropped when cased incorrectly

Quicklisp’s jonathan library serializes Common Lisp keywords like :jsonrpc as "JSONRPC", but Lem expects lowercase "jsonrpc". Messages with uppercase keys appear to be silently ignored.

It would be helpful to either:

• document that JSON keys must be lowercase exactly,
• return an error for invalid JSON-RPC messages,
• or accept JSON-RPC keys case-insensitively.

7. Sending \n in input-string does not evaluate the form

Sending a newline through input-string inserts a literal newline but does not trigger evaluation. Evaluation requires a separate key event for Return. This is understandable, but should be documented.

Working path discovered

The GUI/WebView version works:

1. Start Lem.app
2. Connect to ws://localhost:64265
3. Send JSON-RPC login
4. Send input events equivalent to M-x start-lisp-repl
5. Send an expression via input-string
6. Send a separate Return key event
7. Read bulk notifications and extract text from put operations

This successfully evaluated:

(+ 1 2 3 4 5)

and produced:

15

It also worked for editor forms such as:

(lem:buffer-name (lem:current-buffer))

Requested fixes / feature requests

1. Fix lem-server --mode websocket so it emits bulk display content like the GUI/WebView frontend.

2. Add a supported headless eval endpoint that accepts a Lisp form and returns the result.

3. Provide a CLI equivalent, for example:

   lem-client --eval '(+ 1 2 3)'

4. Document the WebSocket JSON-RPC protocol:

   • login
   • input
   • redraw
   • invoke
   • bulk display messages
   • expected frame masking
   • payload length handling
   • key event format

5. Document or expose the invoke method registry.

6. Improve error handling for Swank port conflicts, especially EADDRINUSE on port 4006.

7. Consider returning explicit JSON-RPC errors instead of silently dropping malformed or incorrectly cased messages.

Why this matters

Lem is very close to being usable as a scriptable Common Lisp IDE/editor backend for AI agents and other automation tools. The GUI/WebView frontend already makes this possible, but the headless server currently does not provide enough output to be used reliably for this purpose. A documented eval endpoint or working headless display stream would make Lem much easier to integrate with external tools.

Note: I spent about 4 hours debugging this because I really want Lem to work as a scriptable Common Lisp editor backend. I’m reporting the details not to complain, but because I think Lem is very close to supporting this workflow: the GUI/WebView path already works, while the headless server seems to be missing the output/display side needed for automation.