Skip to content

IO#write after a partial write blocks until a thread reading the same socket wakes up #9886

Open

Description

@route

With one thread blocked in IO#readpartial on a socket, IO#write on the same socket from another thread never resumes after a partial write: it stays blocked until the reading thread wakes up, i.e. until the peer sends something, even though the peer has drained the socket and it's writable again. CRuby finishes the write as soon as the peer reads.

Reproduction

require "socket"

server = TCPServer.new("127.0.0.1", 0)
client = TCPSocket.new("127.0.0.1", server.addr[1])
peer = server.accept

# A thread blocked reading the socket, as any client with a reader thread has.
Thread.new { client.readpartial(512) rescue nil }
sleep 0.3

# More than the socket buffers hold, so the write has to wait for the peer.
writer = Thread.new { client.write("x" * (16 * 1024 * 1024)) }
sleep 1

# The peer reads everything but never writes back.
Thread.new { loop { peer.readpartial(1 << 20) } rescue nil }

puts writer.join(5) ? "#{RUBY_ENGINE}: write finished" : "#{RUBY_ENGINE}: write still blocked 5s after the peer started reading"
peer.write("y")
puts writer.join(5) ? "#{RUBY_ENGINE}: write finished once the peer sent a byte" : "#{RUBY_ENGINE}: still blocked"
ruby: write finished
ruby: write finished once the peer sent a byte

jruby: write still blocked 5s after the peer started reading
jruby: write finished once the peer sent a byte

Same result on JRuby 10.1.2.0 (OpenJDK 21.0.2, macOS arm64, and Temurin 21 on GitHub's ubuntu-latest) and on JRuby 9.4.15.0. CRuby 4.0.1 for comparison.

Where it blocks

The writer waits for the monitor the reader holds while it's parked in select:

"writer" BLOCKED (on object monitor)
	at org.jruby.RubyThread.selectReadyOps(RubyThread.java:2400)
	- waiting to lock <0x000000061f780288> (a java.lang.Object)
	at org.jruby.util.io.OpenFile.readyOps(OpenFile.java:602)
	at org.jruby.util.io.OpenFile.ready(OpenFile.java:581)
	at org.jruby.util.io.OpenFile.waitWritable(OpenFile.java:519)
	at org.jruby.util.io.OpenFile.waitWritable(OpenFile.java:531)
	at org.jruby.util.io.OpenFile.binwriteInt(OpenFile.java:2505)
	at org.jruby.util.io.OpenFile.fwrite(OpenFile.java:2298)
	at org.jruby.RubyIO.write(RubyIO.java:1663)

"reader" RUNNABLE
	- locked <0x000000061f780cc8> (a sun.nio.ch.Util$2)
	- locked <0x000000061f780ae8> (a sun.nio.ch.KQueueSelectorImpl)
	at org.jruby.RubyThread.selectReadyOps(RubyThread.java:2415)
	- locked <0x000000061f780288> (a java.lang.Object)
	at org.jruby.util.io.OpenFile.readyOps(OpenFile.java:602)
	...
	at org.jruby.RubyIO.readpartial(RubyIO.java:3361)

selectReadyOps holds the channel's blocking lock for the whole select, so a thread waiting to write can't register its own interest until the reader's select returns. The same lock makes write_nonblock and IO.select for writing block in that situation (they need it to switch the channel to nonblocking mode).

Where we hit it

Ferrum, a Chrome DevTools Protocol client, keeps a reader thread on the websocket and writes from another thread. Any command bigger than the socket buffers hangs on JRuby until Chrome happens to send an event; Chrome is waiting for the whole message first, so a 64MB command times out while CRuby sends it in about a second. The write_nonblock variant made Browser.new hang in the handshake: rubycdp/ferrum#676.

Activity

  1. headius commented on Oct 27, 2026

    @headius
    Member

    Interesting. I suppose we could release the lock during the actual select call. Need to do some analysis to ensure that's safe.

  2. headius commented on Oct 27, 2026

    @headius
    Member

    So here's the situation.

    Our use of blockingLock this way is valid for situations like this where we temporarily set the stream to non-blocking and restore it to blocking afterwards. That's how I learned to do this situation, and a quick review by Codex seems to confirm it, with a particular example from OpenJDK here: https://github.com/openjdk/jdk8u/blob/master/jdk/src/share/classes/sun/nio/ch/SocketAdaptor.java#L182-L218

    Other examples out there that do not acquire the lock also do not reset the blocking mode. This differs from the behavior in JRuby.

    But several releases back, CRuby started setting all streams to non-blocking; JRuby has not fully realized that change yet.

    So this could be fixed by us following CRuby and immediately selectable channels to non-blocking, but there's broader implications for any logic that expects an unsatisfied bare "read" or "write" to block. Most call paths should be going through selector logic, but I can't guarantee they all do.

    #7466 is an unmerged PR that sets selectable channels to nonblocking along several call paths (maybe all the ones we care about). This change could be combined with a softer locking policy: only lock if the channel was blocking before and must be restored to that state).

    I think we should explore merging #7466 and implementing the soft lock policy for JRuby 10.1.4.0. We are hoping to release 10.1.3.0 this week and such a change would not have enough testing.

  3. added this to the JRuby 10.1.4.0 milestone on Oct 27, 2026
  4. headius commented on Oct 27, 2026

    @headius
    Member

    As expected, the changes in #7466 expose many places where we don't expect channels to be set nonblocking. This results in many IO calls suddenly throwing EAGAIN/EWOULDBLOCK because we aren't currently wrapping them in select.

    The good news is that with that patch and a softer locking policy from #9905, your example script runs correctly and matches CRuby:

    $ jruby blah.rb
    jruby: write finished
    jruby: write finished once the peer sent a byte
    $ cx ruby-4.0 ruby blah.rb
    ruby: write finished
    ruby: write finished once the peer sent a byte
    

    If you have some availability to assist with making #7466 green, that would help speed things along.

  5. sampokuokkanen commented on Nov 5, 2026

    @sampokuokkanen
    Contributor

    This shows up in Rails too. Action Cable's Redis adapter has a listener thread sitting in pubsub_client.next_event(60) (redis.rb#L118), and new subscriptions write subscribe to the same connection from another thread (redis.rb#L156). redis-client uses write_nonblock, so on JRuby master that subscribe takes ~59.5s instead of 1ms, and Rails' Redis adapter tests time out.

    Minimal version (CRuby: 0ms, JRuby master: ~4.7s):

    require "socket"
    require "io/wait"
    
    a, _b = Socket.pair(:UNIX, :STREAM)
    reader = Thread.new { a.wait_readable(5) }
    sleep 0.3
    t = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    a.write_nonblock("x", exception: false)
    puts "#{((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t) * 1000).round}ms"
    reader.join

    With sockets nonblocking from the start, the select shouldn't take blockingLock anymore, so I'd expect #9921 to fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions