Skip to content

Use Thread::Mutex for autoload and require locks like CRuby - #9877

Open
sampokuokkanen wants to merge 12 commits into
jruby:masterfrom
sampokuokkanen:scheduler-require-autoload
Open

sampokuokkanen wants to merge 12 commits into
jruby:masterfrom
sampokuokkanen:scheduler-require-autoload

Conversation

@sampokuokkanen

@sampokuokkanen sampokuokkanen commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Under a fiber scheduler, if one fiber was still loading a file and another fiber required it or touched one of its constants, the whole thread blocked. When the loading fiber was itself waiting on the scheduler (sleep, IO), nothing could resume it, and both hung for good.

Both locks are now a Thread::Mutex, as in MRI, so the scheduler wait comes from Mutex and LoadService needs no fiber code:

  • Autoload uses a lazily created Mutex instead of a Java monitor (MRI's autoload_data->mutex).
  • Require locks wrap a Mutex like MRI's thread shield (load_lock → rb_thread_shield_wait → do_mutex_lock), hidden from ObjectSpace like a thread shield.

Two smaller changes, both matching MRI: without a scheduler, a fiber that runs into another fiber's load on the same thread gets a ThreadError instead of hanging, and an autoload that failed with something like a NameError now reloads on retry. Without a scheduler the only extra cost is one Mutex per load in progress, and a quick Campfire boot showed no difference beyond run-to-run noise.

}

synchronized (this) {
return value;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this the only synchronized part left against the value, which is already volatile?

seems a bit redundant upon first check, assuming we're keeping the mutex approach as is...

@headius

headius commented Oct 28, 2026

Copy link
Copy Markdown
Member

Oy vey... scheduler-aware require and load was not on my bingo card. I'm in no hurry to merge this one.

@headius headius added this to the JRuby 10.1.4.0 milestone Oct 28, 2026
@sampokuokkanen sampokuokkanen changed the title Let fibers wait on another fiber's require and autoload through the fiber scheduler Use Thread::Mutex for autoload and require locks like CRuby Oct 28, 2026
@sampokuokkanen

Copy link
Copy Markdown
Contributor Author

It is quite a big change... 😅

CRuby fixed this in https://bugs.ruby-lang.org/issues/18663 by putting autoload on a Thread::Mutex.

I changed the title as this doesn't actually have anything about the fiber scheduler, the fiber scheduler support comes from the fiber-aware Mutex work from #9790.

@sampokuokkanen
sampokuokkanen force-pushed the scheduler-require-autoload branch from 619c7e2 to 0fce71e Compare November 6, 2026 02:12
Autoload#load held a Java monitor for the whole require. When the file
slept through the fiber scheduler, a sibling fiber referencing the same
constant blocked its thread on the monitor, and the loading fiber could
never be resumed, as in TestFiberScheduler#test_autoload.

Hold a Thread::Mutex instead, which waits through the scheduler like
MRI's autoload_mutex. The constant's value is set as soon as it is
defined, and the shared monitor used to keep other threads from seeing
it before the file finished loading, so reading the value now waits
for a load in progress elsewhere the same way.
A fiber requiring a file that another fiber was still loading blocked
its thread on the file's require lock. When the loading fiber was
waiting through the scheduler, e.g. sleeping in the file, it could
never be resumed and both hung. With continuation fibers the waiter
shares the loader's thread, so it instead looked like a circular
require and skipped the file.

MRI's require lock is a mutex, which fibers wait on through the
scheduler. Do the same: record the fiber that holds a require lock,
have fibers under a scheduler wait through its block hook, and unblock
them when the load finishes.
A fiber waiting through the scheduler for another require was only
woken when the loader finished. A plain thread waiting on the same
lock takes and drops it right after, and if the fiber woke while that
thread held it, the fiber blocked again with nobody left to wake it.
Wake waiters on every release, as Mutex does.

An autoload that failed with something other than a LoadError or
RuntimeError leaves ctx set, so when another thread retried it, the
file's own class body reached load again and Mutex raised a recursive
locking ThreadError. Like MRI, go straight to the load when this fiber
already holds the lock.

Create the autoload Mutex and the require lock's waiter queue only
when they are first needed, so nothing is allocated without a
scheduler, and pass the context to Autoload#getValue. Drop the
sibling-fiber branch, which only fires when fibers share a thread.
Autoload checked ownership with owned_p(context).isTrue(), a detour through a Ruby boolean. Add isOwned() next to isLocked() and use it.
getValue was synchronized when load held the same monitor, which made
readers wait for another thread's load. The loading Mutex does that
waiting now, so the synchronized block only wrapped a read of a
volatile field.
MRI's require lock is a thread shield, which is a Mutex underneath, so
the scheduler wait comes from do_mutex_lock and load.c knows nothing
about fibers. Do the same: a RequireLock holds a Thread::Mutex, and
waiting for another load is lock then unlock. That drops the separate
waiter queue and wakeup the ReentrantLock needed, and Mutex already
wakes the next waiter on every release and polls interrupts while
waiting.
Autoload only cleared ctx when the load failed with a LoadError or
RuntimeError. After a NameError, for example, ctx still named the
thread whose load failed, so that thread took itself for the loader:
retrying the constant gave NameError instead of loading the file
again, and while another thread retried the load it did not wait and
got NameError too. On master the monitor hid the second case.

Clear ctx whatever failed the load, and make whoever holds the lock
the loader, as MRI ties loading to the autoload mutex. That also
covers what the owned check in load was there for.
With ObjectSpace enabled, each_object(Thread::Mutex) listed the
mutexes behind require locks. MRI's thread shield mutex is hidden, so
create them through a Mutex.newInternal that skips ObjectSpace.
A fiber with no scheduler that requires or autoloads a file another
fiber of its thread is still loading now raises ThreadError, as on MRI,
instead of hanging. Spec both.

Group the require specs under one describe whose after hook finishes
any fiber left suspended, so a failed expectation cannot leave the
require lock held, and use one Fiber.yield fixture for both. Run the
autoload NameError retries from a shared before; the retry in another
thread was already covered by the spec that waits for it.
Autoload#load waits for a load in another thread or fiber first, the
CRuby link on loading() covers only the lazy mutex creation, the
autoload mutex stays visible on purpose, and tryLock on a fresh mutex
cannot fail.
The monitor used to keep setConstant out of a running load. Only the
thread whose ctx it is writes now, and ctx changes only under the
autoload Mutex, while readers of the fields never took the monitor.
The require spec covers this case, but autoload takes its own lock.
@sampokuokkanen
sampokuokkanen force-pushed the scheduler-require-autoload branch from 7531c99 to c364c55 Compare November 12, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants