Skip to content

fsmonitor: implement filesystem change listener for Linux#2147

Open
ptarjan wants to merge 13 commits intogit:masterfrom
ptarjan:claude/update-pr-1352-current-85Gk8
Open

fsmonitor: implement filesystem change listener for Linux#2147
ptarjan wants to merge 13 commits intogit:masterfrom
ptarjan:claude/update-pr-1352-current-85Gk8

Conversation

@ptarjan
Copy link
Copy Markdown
Contributor

@ptarjan ptarjan commented Dec 30, 2025

This series implements the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. It also fixes two memory leaks in the platform-independent daemon code and deduplicates the IPC and settings logic that is now shared between macOS and Linux.

The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor.

The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync.

New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore.

Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting.

This series builds on work from #1352 by Eric DeCosta and #1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback.

Changes since v12:

  • Dropped both the fsmonitor.c workaround and the read-cache.c skipHash fix per Dscho's review: split-index and index.skipHash are fundamentally incompatible
  • Added sane_unset GIT_TEST_SPLIT_INDEX to scalar clone tests in t9210 (patch 1/13), matching the existing workaround in test 16

Changes since v11:

  • Fix t9210 BUG assertion with GIT_TEST_SPLIT_INDEX=yes: guard fsmonitor_dirty bitmap access against split-index having fewer entries than the bitmap expects

Changes since v10:

  • Reverted pre_exec_cb callback back to simple close_fd_above_stderr flag per Junio's clarification (same as v8)

Changes since v9:

  • Fixed Windows build: close_fd_above_stderr() compiles as a no-op on Windows since there is no fork/exec

Changes since v8:

  • Replaced close_fd_above_stderr flag with generic pre_exec_cb callback in struct child_process per Junio's review, with close_fd_above_stderr() as a ready-made callback

Changes since v7:

  • Added patch 12: convert khash to strset in do_handle_client (Patrick's #leftoverbit suggestion)
  • Fixed "Forcing shutdown" trace message to start with lowercase
  • Fixed redundant statfs() call in find_mount() (caller already had the result)
  • Fixed CMakeLists.txt GIT-BUILD-OPTIONS: was hardcoded to "win32" for FSMONITOR_DAEMON_BACKEND and FSMONITOR_OS_SETTINGS, now uses the CMake variables
  • Fixed uninitialized strset on trivial response path (STRSET_INIT)
  • Removed V9FS_MAGIC from get_fs_typename() to match is_remote_fs() (9p is local VM mounts)
  • Split 30-second stop timeout into its own commit per review request
  • Fixed misleading indentation on shutdown assignment in handle_events()
  • Updated commit messages to describe all changes (test hardening, fsmonitor-ipc.c spawn changes)
  • Updated Makefile comment for FSMONITOR_OS_SETTINGS to mention fsm-ipc

Changes since v6:

  • Introduced FSMONITOR_OS_SETTINGS build variable (set to "unix" for macOS and Linux, "win32" for Windows) to eliminate if/else conditionals in Makefile, meson.build, and CMakeLists.txt per Junio's review
  • Moved fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND since path-utils files are platform-specific
  • Removed V9FS_MAGIC from remote filesystem detection (9p is used for local VM/container host mounts where fsmonitor works fine)
  • Removed redundant #include <libgen.h> (already provided by compat/posix.h)
  • Fixed cookie wait comment wording ("see" to "observe")
  • Rewrote commit messages for IPC and settings dedup patches

Changes since v5:

  • Split monolithic commit into 10-patch series per Patrick's review
  • Deduplicated fsm-ipc and fsm-settings into shared Unix implementations
  • Rewrote commit message with prose paragraphs, explain inotify vs fanotify, removed "Issues addressed" sections, added Based-on-patch-by trailers
  • Removed redundant includes already provided by compat/posix.h
  • Fixed error/trace message capitalization per coding guidelines
  • Fixed stale rename check interval from 1000 seconds to 1 second
  • Changed poll timeout from 1ms to 50ms to reduce idle CPU wake-ups
  • Replaced infinite pthread_cond_wait cookie loop with one-second pthread_cond_timedwait (prevents daemon hangs on overlay filesystems where events are never delivered)
  • Added pthread_cond_timedwait to Windows pthread compatibility layer
  • Separated test into its own commit with smoke test that skips when inotify events are not delivered (e.g., overlayfs with older kernels)
  • Fixed test hang on Fedora CI: stop_git() looped forever when ps was unavailable because bash in POSIX/sh mode returns exit 0 from kill with an empty process group argument. Fixed by falling back to /proc/$pid/stat for process group ID and guarding stop_git against empty pgid.
  • Redirect spawn_daemon() stdout/stderr to /dev/null and close inherited file descriptors to prevent the intermediate process from holding test pipe file descriptors
  • Call setsid() on daemon detach to prevent shells with job control from waiting on the daemon process group
  • Close inherited file descriptors 3-7 in the test watchdog subprocess
  • Added 30-second timeout to "fsmonitor--daemon stop" to prevent indefinite blocking
  • Added helpful error message when inotify watch limit (max_user_watches) is reached
  • Initialize fd_inotify to -1 and use fd >= 0 check for correct fd 0 handling
  • Use sysconf(_SC_OPEN_MAX) instead of hardcoded 1024 for fd close limit
  • Check setsid() return value

Changes since v4:

  • Added Meson build support

Changes since v3:

  • Fix crash on rapid nested directory creation (EEXIST from inotify_add_watch with IN_MASK_CREATE)
  • Extensive stress testing

Changes since v2:

  • Fix khash memory leak in do_handle_client

Changes since v1:

  • Fix hashmap memory leak in fsmonitor_run_daemon()

CC: Patrick Steinhardt ps@pks.im
cc: Paul Tarjan paul@paultarjan.com

@gitgitgadget-git
Copy link
Copy Markdown

Welcome to GitGitGadget

Hi @ptarjan, and welcome to GitGitGadget, the GitHub App to send patch series to the Git mailing list from GitHub Pull Requests.

Please make sure that either:

  • Your Pull Request has a good description, if it consists of multiple commits, as it will be used as cover letter.
  • Your Pull Request description is empty, if it consists of a single commit, as the commit message should be descriptive enough by itself.

You can CC potential reviewers by adding a footer to the PR description with the following syntax:

CC: Revi Ewer <revi.ewer@example.com>, Ill Takalook <ill.takalook@example.net>

NOTE: DO NOT copy/paste your CC list from a previous GGG PR's description,
because it will result in a malformed CC list on the mailing list. See
example.

Also, it is a good idea to review the commit messages one last time, as the Git project expects them in a quite specific form:

  • the lines should not exceed 76 columns,
  • the first line should be like a header and typically start with a prefix like "tests:" or "revisions:" to state which subsystem the change is about, and
  • the commit messages' body should be describing the "why?" of the change.
  • Finally, the commit messages should end in a Signed-off-by: line matching the commits' author.

It is in general a good idea to await the automated test ("Checks") in this Pull Request before contributing the patches, e.g. to avoid trivial issues such as unportable code.

Contributing the patches

Before you can contribute the patches, your GitHub username needs to be added to the list of permitted users. Any already-permitted user can do that, by adding a comment to your PR of the form /allow. A good way to find other contributors is to locate recent pull requests where someone has been /allowed:

Both the person who commented /allow and the PR author are able to /allow you.

An alternative is the channel #git-devel on the Libera Chat IRC network:

<newcontributor> I've just created my first PR, could someone please /allow me? https://github.com/gitgitgadget/git/pull/12345
<veteran> newcontributor: it is done
<newcontributor> thanks!

Once on the list of permitted usernames, you can contribute the patches to the Git mailing list by adding a PR comment /submit.

If you want to see what email(s) would be sent for a /submit request, add a PR comment /preview to have the email(s) sent to you. You must have a public GitHub email address for this. Note that any reviewers CC'd via the list in the PR description will not actually be sent emails.

After you submit, GitGitGadget will respond with another comment that contains the link to the cover letter mail in the Git mailing list archive. Please make sure to monitor the discussion in that thread and to address comments and suggestions (while the comments and suggestions will be mirrored into the PR by GitGitGadget, you will still want to reply via mail).

If you do not want to subscribe to the Git mailing list just to be able to respond to a mail, you can download the mbox from the Git mailing list archive (click the (raw) link), then import it into your mail program. If you use GMail, you can do this via:

curl -g --user "<EMailAddress>:<Password>" \
    --url "imaps://imap.gmail.com/INBOX" -T /path/to/raw.txt

To iterate on your change, i.e. send a revised patch or patch series, you will first want to (force-)push to the same branch. You probably also want to modify your Pull Request description (or title). It is a good idea to summarize the revision by adding something like this to the cover letter (read: by editing the first comment on the PR, i.e. the PR description):

Changes since v1:
- Fixed a typo in the commit message (found by ...)
- Added a code comment to ... as suggested by ...
...

To send a new iteration, just add another PR comment with the contents: /submit.

Need help?

New contributors who want advice are encouraged to join git-mentoring@googlegroups.com, where volunteers who regularly contribute to Git are willing to answer newbie questions, give advice, or otherwise provide mentoring to interested contributors. You must join in order to post or view messages, but anyone can join.

You may also be able to find help in real time in the developer IRC channel, #git-devel on Libera Chat. Remember that IRC does not support offline messaging, so if you send someone a private message and log out, they cannot respond to you. The scrollback of #git-devel is archived, though.

@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Dec 30, 2025

/submit

@gitgitgadget-git
Copy link
Copy Markdown

Error: User ptarjan is not yet permitted to use GitGitGadget

@ptarjan ptarjan force-pushed the claude/update-pr-1352-current-85Gk8 branch from 0fab68b to e079635 Compare December 30, 2025 07:01
@Ikke
Copy link
Copy Markdown
Contributor

Ikke commented Dec 30, 2025

/allow

@gitgitgadget-git
Copy link
Copy Markdown

User ptarjan is now allowed to use GitGitGadget.

@ptarjan ptarjan force-pushed the claude/update-pr-1352-current-85Gk8 branch 3 times, most recently from 5f45d96 to aa60645 Compare December 30, 2025 07:22
@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Dec 30, 2025

/submit

@gitgitgadget-git
Copy link
Copy Markdown

Submitted as pull.2147.git.git.1767082450088.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v1

To fetch this version to local tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v1:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v1

@gitgitgadget-git
Copy link
Copy Markdown

On the Git mailing list, Junio C Hamano wrote (reply to this):

Exciting.

It seems to die with leaks when "make SANITIZE=leak test" is run,
though.

Initialized empty Git repository in /home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit/.git/
fsmonitor-daemon is not watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit'
builtin:0.145521.20251230T113644.793433Z:0Q/Q
{"event":"data","sid":"20251230T113644.762195Z-H3cfff1b1-P0002386f","thread":"main","time":"2025-12-30T11:36:44.813131Z","file":"fsmonitor-ipc.c","line":99,"t_abs":0.052581,"t_rel":0.048830,"nesting":2,"category":"fsm_client","key":"query/response-length","value":"45"}
fsmonitor-daemon is watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit'
fsmonitor-daemon is not watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit'
fatal: fsmonitor--daemon is not running
not ok 2 - implicit daemon start
#
#               test_when_finished "stop_daemon_delete_repo test_implicit" &&
#
#               git init test_implicit &&
#               test_must_fail git -C test_implicit fsmonitor--daemon status &&
#
#               # query will implicitly start the daemon.
#               #
#               # for test-script simplicity, we send a V1 timestamp rather than
#               # a V2 token.  either way, the daemon response to any query contains
#               # a new V2 token.  (the daemon may complain that we sent a V1 request,
#               # but this test case is only concerned with whether the daemon was
#               # implicitly started.)
#
#               GIT_TRACE2_EVENT="$PWD/.git/trace" \
#                       test-tool -C test_implicit fsmonitor-client query --token 0 >actual &&
#               nul_to_q <actual >actual.filtered &&
#               grep "builtin:" actual.filtered &&
#
#               # confirm that a daemon was started in the background.
#               #
#               # since the mechanism for starting the background daemon is platform
#               # dependent, just confirm that the foreground command received a
#               # response from the daemon.
#
#               have_t2_data_event fsm_client query/response-length <.git/trace &&
#
#               git -C test_implicit fsmonitor--daemon status &&
#               git -C test_implicit fsmonitor--daemon stop &&
#               test_must_fail git -C test_implicit fsmonitor--daemon status
#
1..2

=================================================================
==git==145489==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 512 byte(s) in 1 object(s) allocated from:
    #0 0x56179e6ce042 in calloc (git+0x8c042) (BuildId: de5ce3c9d0b0c09380c910e6a9eb181e324abde6)
    #1 0x56179ea0aef4 in xcalloc wrapper.c:154:8
    #2 0x56179e8b17b7 in alloc_table hashmap.c:79:2
    #3 0x56179e8b174c in hashmap_init hashmap.c:168:2
    #4 0x56179e73e6fe in fsmonitor_run_daemon builtin/fsmonitor--daemon.c:1288:2
    #5 0x56179e73e141 in try_to_run_foreground_daemon builtin/fsmonitor--daemon.c:1448:11
    #6 0x56179e73dc44 in cmd_fsmonitor__daemon builtin/fsmonitor--daemon.c:1584:12
    #7 0x56179e6d2c8a in run_builtin git.c:506:11
    #8 0x56179e6d1910 in handle_builtin git.c:779:9
    #9 0x56179e6d2747 in run_argv git.c:862:4
    #10 0x56179e6d169b in cmd_main git.c:984:19
    #11 0x56179e7f7a7a in main common-main.c:9:11
    #12 0x7f091ea66ca7 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #13 0x7f091ea66d64 in __libc_start_main csu/../csu/libc-start.c:360:3
    #14 0x56179e69e280 in _start (git+0x5c280) (BuildId: de5ce3c9d0b0c09380c910e6a9eb181e324abde6)

@ptarjan ptarjan force-pushed the claude/update-pr-1352-current-85Gk8 branch from aa60645 to 7a7fe25 Compare December 30, 2025 12:04
@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Dec 30, 2025

/submit

@gitgitgadget-git
Copy link
Copy Markdown

Submitted as pull.2147.v2.git.git.1767096494372.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v2

To fetch this version to local tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v2:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v2

@ptarjan ptarjan force-pushed the claude/update-pr-1352-current-85Gk8 branch from 7a7fe25 to 207da68 Compare December 30, 2025 12:51
@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Dec 30, 2025

/submit

@gitgitgadget-git
Copy link
Copy Markdown

Submitted as pull.2147.v3.git.git.1767099302592.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v3

To fetch this version to local tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v3:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v3

@gitgitgadget-git
Copy link
Copy Markdown

On the Git mailing list, Junio C Hamano wrote (reply to this):

Thanks for a quick turnaround, but it would be more efficient if you
hunted all the leaks yourself, instead of getting a report for one
issue and updating the patch to fix that one issue.

Here is what I am getting these:

    $ make SANITIZE=address CC=clang && cd t && sh t7527-*.sh -i -v

Note that "-i" is to say "stop at the first one".


expecting success of 7527.12 'create some files':
        test_when_finished clean_up_repo_and_stop_daemon &&

        start_daemon --tf "$PWD/.git/trace" &&

        create_files &&

        test-tool fsmonitor-client query --token 0 &&

        grep "^event: dir1/new$" .git/trace &&
        grep "^event: dir2/new$" .git/trace &&
        grep "^event: new$"      .git/trace

fsmonitor-daemon is watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor'
builtin:0.1039108.20251230T123036.129805Z:0/event: dir1/new
event: dir1/new
event: dir2/new
event: dir2/new
event: new
event: new
HEAD is now at 1d1edcb initial
Removing dir1/new
Removing dir2/new
Removing new
not ok 12 - create some files
#
#               test_when_finished clean_up_repo_and_stop_daemon &&
#
#               start_daemon --tf "$PWD/.git/trace" &&
#
#               create_files &&
#
#               test-tool fsmonitor-client query --token 0 &&
#
#               grep "^event: dir1/new$" .git/trace &&
#               grep "^event: dir2/new$" .git/trace &&
#               grep "^event: new$"      .git/trace
#
1..12

=================================================================
==git==1039073==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 40 byte(s) in 1 object(s) allocated from:
    #0 0x55c18d8d4042 in calloc (git+0x8c042) (BuildId: 4097db008a82663ae0b3398128a7ab4e09bbdd21)
    #1 0x55c18dc10f14 in xcalloc wrapper.c:154:8
    #2 0x55c18d945f72 in kh_init_str builtin/fsmonitor--daemon.c:656:1
    #3 0x55c18d945828 in do_handle_client builtin/fsmonitor--daemon.c:871:10
    #4 0x55c18d945191 in handle_client builtin/fsmonitor--daemon.c:987:11
    #5 0x55c18dc283e2 in worker_thread__do_io compat/simple-ipc/ipc-unix-socket.c:532:9
    #6 0x55c18dc27a7f in worker_thread_proc compat/simple-ipc/ipc-unix-socket.c:606:9
    #7 0x55c18d8d64f4 in void* ThreadStartFunc<false>(void*) lsan_interceptors.cpp.o
    #8 0x7fe358257b7a in start_thread nptl/pthread_create.c:448:8
    #9 0x7fe3582d57b7 in __GI___clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Dec 30, 2025 via email

ptarjan pushed a commit to ptarjan/git that referenced this pull request Dec 30, 2025
Two patches with suggested improvements:

1. Documentation and minor fixes:
   - Document why V9FS is excluded from is_remote_fs() (supports inotify)
   - Replace magic number 1000 with STALE_RENAME_CHECK_INTERVAL_SEC constant
   - Document buffer size and timeout rationale

2. Linux-specific tests:
   - Test fsmonitor.socketdir configuration
   - Test directory rename tracking via inotify cookies
   - Test cross-directory moves
   - Test rapid nested directory creation
@ptarjan ptarjan force-pushed the claude/update-pr-1352-current-85Gk8 branch from 207da68 to c7fd346 Compare December 31, 2025 17:38
@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Dec 31, 2025

/submit

@gitgitgadget-git
Copy link
Copy Markdown

Submitted as pull.2147.v4.git.git.1767202894884.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v4

To fetch this version to local tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v4:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v4

@ptarjan ptarjan force-pushed the claude/update-pr-1352-current-85Gk8 branch from c7fd346 to 350c90b Compare December 31, 2025 19:20
@gitgitgadget-git
Copy link
Copy Markdown

On the Git mailing list, Patrick Steinhardt wrote (reply to this):

On Wed, Dec 31, 2025 at 05:41:34PM +0000, Paul Tarjan via GitGitGadget wrote:
> From: Paul Tarjan <github@paulisageek.com>
> 
> Implement fsmonitor for Linux using the inotify API, bringing it to
> feature parity with existing Windows and macOS implementations.
> 
> The Linux implementation uses inotify to monitor filesystem events.
> Unlike macOS's FSEvents which can watch a single root directory,
> inotify requires registering watches on every directory of interest.
> The implementation carefully handles directory renames and moves
> using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO
> event pairs.
> 
> Key implementation details:
> - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring
> - Maintains bidirectional hashmaps between watch descriptors and paths
>   for efficient event processing
> - Handles directory creation, deletion, and renames dynamically
> - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs()
> - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote
> - Creates batches lazily (only for actual file events, not cookies)
>   to avoid spurious sequence number increments
> 
> Build configuration:
> - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux
> - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset
> - Adds HAVE_LINUX_MAGIC_H for filesystem type detection

This would also need the below patch to support Meson. Would be great if
you include it, otherwise I can send it as a separate patch once this
topic lands. Thanks!

Patrick

-- >8 --

diff --git a/meson.build b/meson.build
index dd52efd1c8..0130d40702 100644
--- a/meson.build
+++ b/meson.build
@@ -1322,6 +1322,9 @@ endif
 fsmonitor_backend = ''
 if host_machine.system() == 'windows'
   fsmonitor_backend = 'win32'
+elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h')
+  fsmonitor_backend = 'linux'
+  libgit_c_args += '-DHAVE_LINUX_MAGIC_H'
 elif host_machine.system() == 'darwin'
   fsmonitor_backend = 'darwin'
   libgit_dependencies += dependency('CoreServices')

@gitgitgadget-git
Copy link
Copy Markdown

User Patrick Steinhardt <ps@pks.im> has been added to the cc: list.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via b53f761.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via f66cfca.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 96cccf4.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 0ee127b.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 6d38eb3.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via e86b5cd.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 92b7eef.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via abfd972.

@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Mar 31, 2026

/submit

@gitgitgadget-git
Copy link
Copy Markdown

Submitted as pull.2147.v12.git.git.1774937958.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v12

To fetch this version to local tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v12:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v12

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 1aa0857.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 8a81e7d.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 5e8a573.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 38a323e.

@gitgitgadget-git
Copy link
Copy Markdown

This patch series was integrated into seen via 0e0c37f.

ptarjan added 13 commits April 5, 2026 10:49
index.skipHash (Scalar default) and split-index are incompatible:
the shared index gets a null OID when skipHash skips computing the
hash, and the null OID causes the shared index to not be loaded on
re-read.  This triggers a BUG assertion in fsmonitor when the
fsmonitor_dirty bitmap references more entries than the (now empty)
index has.

Disable GIT_TEST_SPLIT_INDEX in the scalar clone tests that hit
this, matching the existing workaround in test 16.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
The `shown` kh_str_t was freed with kh_release_str() at a point in
the code only reachable in the non-trivial response path.  When the
client receives a trivial response, the code jumps to the `cleanup`
label, skipping the kh_release_str() call entirely and leaking the
hash table.

Fix this by initializing `shown` to NULL and moving the cleanup to the
`cleanup` label using kh_destroy_str(), which is safe to call on NULL.
This ensures the hash table is freed regardless of which code path is
taken.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
The `state.cookies` hashmap is initialized during daemon startup but
never freed during cleanup in the `done:` label of
fsmonitor_run_daemon().  The cookie entries also have names allocated
via strbuf_detach() that must be freed individually.

Iterate the hashmap to free each cookie name, then call
hashmap_clear_and_free() to release the entries and table.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
Add a pthread_cond_timedwait() implementation to the Windows pthread
compatibility layer using SleepConditionVariableCS() with a millisecond
timeout computed from the absolute deadline.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
The cookie wait in with_lock__wait_for_cookie() uses an infinite
pthread_cond_wait() loop.  The existing comment notes the desire
to switch to pthread_cond_timedwait(), but the routine was not
available in git thread-utils.

On certain container or overlay filesystems, inotify watches may
succeed but events are never delivered.  In this case the daemon
would hang indefinitely waiting for the cookie event, which in
turn causes the client to hang.

Replace the infinite wait with a one-second timeout using
pthread_cond_timedwait().  If the timeout fires, report an
error and let the client proceed with a trivial (full-scan)
response rather than blocking forever.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
The fsmonitor IPC path logic in fsm-ipc-darwin.c is not
Darwin-specific and will be reused by the upcoming Linux
implementation.  Rename it to fsm-ipc-unix.c to reflect that it
is shared by all Unix platforms.

Introduce FSMONITOR_OS_SETTINGS (set to "unix" for non-Windows, "win32"
for Windows) as a separate variable from FSMONITOR_DAEMON_BACKEND so
that the build files can distinguish between platform-specific files
(listen, health, path-utils) and shared Unix files (ipc, settings).

Move fsm-ipc to the FSMONITOR_OS_SETTINGS section in the Makefile, and
switch fsm-path-utils to use FSMONITOR_DAEMON_BACKEND since path-utils
is platform-specific (there will be separate darwin and linux versions).

Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com>
Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com>
Signed-off-by: Paul Tarjan <github@paulisageek.com>
The fsmonitor settings logic in fsm-settings-darwin.c is not
Darwin-specific and will be reused by the upcoming Linux
implementation.  Rename it to fsm-settings-unix.c to reflect that it
is shared by all Unix platforms.

Update the build files (meson.build and CMakeLists.txt) to use
FSMONITOR_OS_SETTINGS for fsm-settings, matching the approach already
used for fsm-ipc.

Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com>
Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com>
Signed-off-by: Paul Tarjan <github@paulisageek.com>
Implement the built-in fsmonitor daemon for Linux using the inotify
API, bringing it to feature parity with the existing Windows and macOS
implementations.

The implementation uses inotify rather than fanotify because fanotify
requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it
unsuitable for an unprivileged user-space daemon.  While inotify has
the limitation of requiring a separate watch on every directory (unlike
macOS's FSEvents, which can monitor an entire directory tree with a
single watch), it operates without elevated privileges and provides
the per-file event granularity needed for fsmonitor.

The listener uses inotify_init1(O_NONBLOCK) with a poll loop that
checks for events with a 50-millisecond timeout, keeping the inotify
queue well-drained to minimize the risk of overflows.  Bidirectional
hashmaps map between watch descriptors and directory paths for efficient
event resolution.  Directory renames are tracked using inotify's cookie
mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a
periodic check detects stale renames where the matching IN_MOVED_TO
never arrived, forcing a resync.

New directory creation triggers recursive watch registration to ensure
all subdirectories are monitored.  The IN_MASK_CREATE flag is used
where available to prevent modifying existing watches, with a fallback
for older kernels.  When IN_MASK_CREATE is available and
inotify_add_watch returns EEXIST, it means another thread or recursive
scan has already registered the watch, so it is safe to ignore.

Remote filesystem detection uses statfs() to identify network-mounted
filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers.
Mount point information is read from /proc/mounts and matched against
the statfs f_fsid to get accurate, human-readable filesystem type names
for logging.  When the .git directory is on a remote filesystem, the
IPC socket falls back to $HOME or a user-configured directory via the
fsmonitor.socketDir setting.

Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com>
Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com>
Signed-off-by: Paul Tarjan <github@paulisageek.com>
Add a close_fd_above_stderr flag to struct child_process.  When set,
the child closes file descriptors 3 and above between fork and exec
(skipping the child-notifier pipe), capped at sysconf(_SC_OPEN_MAX)
or 4096, whichever is smaller.  This prevents the child from
inheriting pipe endpoints or other descriptors from the parent
environment (e.g., the test harness).

Signed-off-by: Paul Tarjan <github@paulisageek.com>
When the fsmonitor daemon is spawned as a background process, it may
inherit file descriptors from its parent that it does not need.  In
particular, when the test harness or a CI system captures output through
pipes, the daemon can inherit duplicated pipe endpoints.  If the daemon
holds these open, the parent process never sees EOF and may appear to
hang.

Set close_fd_above_stderr on the child process at both daemon startup
paths: the explicit "fsmonitor--daemon start" command and the implicit
spawn triggered by fsmonitor-ipc when a client finds no running daemon.
Also suppress stdout and stderr on the implicit spawn path to prevent
the background daemon from writing to the client's terminal.

Additionally, call setsid() when the daemon starts with --detach to
create a new session and process group.  This prevents the daemon
from being part of the spawning shell's process group, which could
cause the shell's "wait" to block until the daemon exits.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
The "fsmonitor--daemon stop" command polls in a loop waiting for the
daemon to exit after sending a "quit" command over IPC.  If the daemon
fails to shut down (e.g. it is stuck or wedged), this loop spins
forever.

Add a 30-second timeout so the stop command returns an error instead
of blocking indefinitely.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
Add a smoke test that verifies the filesystem actually delivers
inotify events to the daemon.  On some configurations (e.g.,
overlayfs with older kernels), inotify watches succeed but events
are never delivered.  The daemon cookie wait will time out, but
every subsequent test would fail.  Skip the entire test file early
when this is detected.

Add a test that exercises rapid nested directory creation to verify
the daemon correctly handles the EEXIST race between recursive scan
and queued inotify events.  When IN_MASK_CREATE is available and a
directory watch is added during recursive registration, the kernel
may also deliver a queued IN_CREATE event for the same directory.
The second inotify_add_watch() returns EEXIST, which must be treated
as harmless.  An earlier version of the listener crashed in this
scenario.

Reduce --start-timeout from the default 60 seconds to 10 seconds so
that tests fail promptly when the daemon cannot start.

Harden the test helpers to work in environments without procps
(e.g., Fedora CI): fall back to reading /proc/$pid/stat for the
process group ID when ps is unavailable, guard stop_git() against
an empty pgid, and redirect stderr from kill to /dev/null to avoid
noise when processes have already exited.

Use set -m to enable job control in the submodule-pull test so that
the background git pull gets its own process group, preventing the
shell wait from blocking on the daemon.  setsid() in the previous
commit detaches the daemon itself, but the intermediate git pull
process still needs its own process group for the test shell to
manage it correctly.

Signed-off-by: Paul Tarjan <github@paulisageek.com>
Replace the khash-based string set used for deduplicating pathnames
in do_handle_client() with a strset, which provides a cleaner
interface for the same purpose.

Since the paths are interned strings from the batch data, use
strdup_strings=0 to avoid unnecessary copies.

Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Paul Tarjan <github@paulisageek.com>
@ptarjan
Copy link
Copy Markdown
Contributor Author

ptarjan commented Apr 6, 2026

/submit

@gitgitgadget-git
Copy link
Copy Markdown

Submitted as pull.2147.v13.git.git.1775498098.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v13

To fetch this version to local tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v13:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v13

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants