Implementing distributed lock using files

Question

I have a network drive (Z:\) which is shared by multiple Windows computers. Is it possible to implement a cross-machine lock by simply creating/deleting files on this network drive?

For example, two computers, A and B, want to write to a shared resource with an ID of 123 at the same time.

One of the computers, say A, locks the resource first by creating an empty file Z:\locks\123. When B sees there is the lock file with the name of "123", B knows the resource 123 is being used by someone else, so it has to wait for Z:\locks\123 to be deleted by A before it can access the resource.

It's like critical section in multithreading, but I want to do it on multiple machines.

I'm trying to implement in Python. Here's what I came up with:

import os
import time


def lock_it(lock_id):

    lock_path = "Z:\\locks\\" + lock_id
    while os.path.exists(lock_path):
        time.sleep(5)  # wait for 5 seconds

    # create the lock file
    lock_file = open(lock_path, "w")
    lock_file.close()


 def unlock_it(lock_id):

     # delete the lock file
     lock_path = "Z:\\locks\\" + lock_id
     if os.path.exists(lock_path):
         os.remove(lock_path)

This won't work because there could be more than one processes exit the waiting status and create the lock file at the same time.

So again, the question is: Is it possible to implement a cross-machine locking mechanism on a shared storage?

David Wolever · Accepted Answer · 2012-06-06 04:17:41Z

8

... sort of.

First, you should create a lock directory instead of a lock file. Creating a directory (see os.mkdir) will fail if the directory already exists, so you can acquire the lock like this:

while True:
    try:
        os.mkdir(r"z:\my_lock")
        return
    except OSError as e:
        if e.errno != 21: # Double check that errno will be the same on Windows
            raise
        time.sleep(5)

Second (and this is where the "sort of" comes in) you'll want some way to notice when the person holding the lock has died. One simple way to do this might be to have them occasionally update a file inside the lock directory. Then if clients notice that file hasn't been updated in a while, they can remove the directory and try to acquire the lock themselves.

answered Jun 6, 2012 at 4:17

David Wolever

156k94 gold badges365 silver badges514 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

eliang Over a year ago

How about adding timeout? i.e., After a period of time, say 60 seconds, the lock is automatically released. Assuming that the locking time won't exceed this timeout. I can get the lock time with time.time() - os.path.getctime("Z:\my_lock") to check if a lock is overdue.

Andrew T Finnell · Accepted Answer · 2012-06-06 04:26:51Z

2

This will not work nearly as well you as might hope. You'll have other issues such as the network drive going away, in which case all your processes will either be stuck or think that no one is holding a lock.

I suggest you look into something like ZooKeeper. You will be able to create synchronous locks and recover in the event of network failures. The framework behind distributed locks is much more complex then creating a file on a network drive.

answered Jun 6, 2012 at 4:26

Andrew T Finnell

13.7k3 gold badges35 silver badges49 bronze badges

1 Comment

Jesvin Jose Over a year ago

I was thinking Memcache or queue software like Beanstalk.

Collectives™ on Stack Overflow

Implementing distributed lock using files

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related