Skip to content

File#flock drops a shared lock on Windows when a non-blocking upgrade to exclusive fails #10087

Closed as not planned

Description

@sferik

Environment Information

  • JRuby 10.0.9.0-SNAPSHOT (jruby-10.0 branch), and the same code is on master
  • Seen on macOS with -Xnative.enabled=false, which takes the same FileChannel path that Windows always uses

Expected Behavior

If a file holds a shared lock and flock(File::LOCK_EX | File::LOCK_NB) fails because another process also holds a shared lock, the file should keep its shared lock. Here, process A holds a shared lock, our file takes a shared lock and tries to upgrade it, then A lets go and a third process tries for an exclusive lock:

require "rbconfig"

path = "test.lock"
File.write(path, "")

holder = IO.popen([RbConfig.ruby, "-e", <<~RUBY], "r+")
  f = File.open(#{path.dump}, "r")
  f.flock(File::LOCK_SH)
  puts :held
  $stdout.flush
  $stdin.gets
  f.flock(File::LOCK_UN)
  puts :released
  $stdout.flush
  $stdin.gets
RUBY
holder.gets

file = File.open(path, "r+")
p file.flock(File::LOCK_SH)
p file.flock(File::LOCK_EX | File::LOCK_NB)

holder.puts
holder.gets

check = "File.open(#{path.dump}, 'r+') { |f| print f.flock(File::LOCK_EX | File::LOCK_NB) }"
puts IO.popen([RbConfig.ruby, "-e", check], &:read)
holder.puts

On CRuby 3.4.12, and on JRuby with native flock, this prints 0, false and false. The last false shows that file still holds its shared lock. CRuby's Windows flock emulation behaves the same way, because it calls LockFileEx without unlocking first, so a failed exclusive request leaves the shared lock in place:

https://github.com/ruby/ruby/blob/67814cf9b0f3a4bad6f7369b0c8e6fc69220de64/win32/win32.c#L354-L366

Actual Behavior

JRuby with -Xnative.enabled=false prints 0, false and 0. After the failed upgrade, file holds no lock at all, so the third process gets an exclusive lock.

PosixShim#tryLock releases the lock it already holds before it calls FileChannel#tryLock, and stores the result, which is null when the new lock can't be taken:

private int tryLock(ChannelFD fd, boolean exclusive) throws IOException {
FileLock fileLock = fd.currentLock.get();
if (fileLock != null) fileLock.release();
fileLock = fd.chFile.tryLock(0L, Long.MAX_VALUE, !exclusive);
fd.currentLock.set(fileLock);

I think the fix is to try the new lock before releasing the old one, but the JDK may not allow that. A second FileChannel lock on the same range from the same JVM throws OverlappingFileLockException. So it may have to release first and then take the shared lock back when the new one fails, which has a short window where another process could get in. PosixShim#lock releases first in the same way, but a blocking lock doesn't fail like this, so I think only the non-blocking case loses the lock.

I noticed this while fixing #10085 in #10086, and haven't run it on Windows itself.

Activity

  1. sferik commented on Nov 16, 2026

    @sferik
    ContributorAuthor

    I was wrong about this one. Keeping the shared lock after a failed upgrade is macOS behavior, not something flock(2) promises. On Linux, flock(2) releases the shared lock before it tries the exclusive one, and the man page says lock conversion isn't atomic. So CRuby 3.4 on Linux prints 0, false and 0 for the script above, the same as JRuby without native flock. JRuby's current Windows behavior is one of the normal ones, and it still lets an upgrade succeed when no other process holds a lock, so I'm closing this. The fix for #10085 in #10086 is separate and still needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions