Skip to content

Support for fiber scheduler API#7944

Merged
headius merged 34 commits intojruby:masterfrom
headius:threads_and_schedulers
Oct 31, 2023
Merged

Support for fiber scheduler API#7944
headius merged 34 commits intojruby:masterfrom
headius:threads_and_schedulers

Conversation

@headius
Copy link
Member

@headius headius commented Sep 29, 2023

Work in progress. Mostly modeling this after logic on CRuby HEAD.

Optimistically marking this for 9.4.5 but probably would be better to drop in the next major release.

* Only IO endpoints calling back into scheduler so far
* IO::Buffer totally stubbed out
* Organization of utility methods likely to change
For test_io_buffer.rb:test_inplace_operators

This also necessitated implementing initialize_copy for the dup'ed
buffer in the test, along with to_s, inspect, and hexdump for when
there's an error condition.
Moving to Java 13+ would allow us to do absolute puts without the
position changing.
Position is not part of IO::Buffer so we must constantly reset it.
Pushing toward the blessed path always receiving a ByteBuffer, so
we can avoid recreating it constantly for small reads.
This unfortunately does not work for most files in JRuby, since we
usually use a real native file descriptor on POSIX platforms. This
will need to be expanded to work with a native fileno but produce
a MappedByteBuffer.
This doesn't pass test_string_mapped because we do not lock
String objects like MRI does and cannot raise an error when the
String owned by the Buffer is modified independently.
@headius headius force-pushed the threads_and_schedulers branch from 6f6c79d to c726bcf Compare October 20, 2023 00:43
This cleans up how blocking works and fixes small parts of the
scheduler dispatch logic. Fibers start out nonblocking by default,
which means entering such a fiber puts the system in a schedulable
state (decrements root fiber's default blocking count of 1 to 0).
While schedulable, blocking calls will redispatch into the
scheduler to give it a chance to handle the blocking operation and
optionally transfer to another unblocked fiber.

The handling of blocking and unblocking will require some
additional work, since currently only a fiber yielding can unblock
its parent, but we are getting the pieces wired up.

All of CRuby's test/fiber/test_scheduler.rb passes with these
changes.
It still all passes 🎉
Working on failures in mri/fiber/test_io_buffer.rb
* This includes two PRs to improve behavior and testing:
  * ruby/ruby#8728
  * ruby/ruby#8729
@headius headius force-pushed the threads_and_schedulers branch from 4ae1fd7 to 68e1955 Compare October 23, 2023 14:13
@headius headius force-pushed the threads_and_schedulers branch from 68e1955 to 2779806 Compare October 25, 2023 06:22
@enebo enebo modified the milestones: JRuby 9.4.5.0, JRuby 9.4.6.0 Oct 31, 2023
@headius headius force-pushed the threads_and_schedulers branch from 54a8d4b to 6b96a41 Compare October 31, 2023 19:54
@headius headius modified the milestones: JRuby 9.4.6.0, JRuby 9.4.5.0 Oct 31, 2023
@headius
Copy link
Member Author

headius commented Oct 31, 2023

We will ship this, disabled, in 9.4.5.0. Set property jruby.experimental.fiber.scheduler to enable.

This commit adds the property "jruby.experimental.fiber.scheduler"
to control whether the Fiber::Scheduler subsystem will be used at
runtime. It is currently off by default, as this is an
experimental feature in all versions of Ruby that support it. Set
this property at the JVM level, or pass
-Xexperimental.fiber.scheduler to enable it.

None of the classes or constants associated with the fiber
scheduler will be defined if the property is not enabled.

In the future, this property may disappear when this feature is
finalized, or may remain under a different name to allow disabling
all scheduler checks.
@headius headius force-pushed the threads_and_schedulers branch from 6b96a41 to fef81e6 Compare October 31, 2023 19:58
@headius headius marked this pull request as ready for review October 31, 2023 19:58
@headius headius merged commit 3445110 into jruby:master Oct 31, 2023
@headius headius deleted the threads_and_schedulers branch October 31, 2023 21:46
headius added a commit to headius/jruby that referenced this pull request Feb 13, 2024
As part of moving toward the scheduler API, several methods from
the io-wait gem have been moved into IO proper.

Preliminary support for the scheduler API was merged in jruby#7944.
headius added a commit to headius/jruby that referenced this pull request Feb 13, 2024
As part of moving toward the scheduler API, several methods from
the io-wait gem have been moved into IO proper.

Preliminary support for the scheduler API was merged in jruby#7944.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants