app, tests: Use the same PATH as on package based Fedora variants #5507

debarshiray · 2025-10-10T19:41:44Z

Some Linux distributions like Arch and Fedora are in different stages of unifying their bin and sbin directories [1,2]. This has been implemented by replacing the /sbin and /usr/sbin directories with relative symbolic links to their bin counterparts. Since, both those distributions have also adopted the /usr merge [3], this means:

/sbin → usr/bin or usr/sbin
/usr/sbin → bin

One notable difference between Arch and Fedora is their handling of /usr/local/sbin. It remains a separate directory on the former, and has been unified with /usr/local/bin in the package based variants of the latter but not those that use rpm-ostree.

Fedora variants that use rpm-ostree end up with different values for the PATH environment variable because systemd sets it based on the state of the bin and sbin directories [4]. Since, there are no Arch variants that use rpm-ostree and there are popular Fedora ones that do, and unifying /usr/local/sbin with its bin counterpart leads to a simpler PATH, it seems reasonable to replicate package based Fedora.

This will make new installations of rpm-ostree based operating systems use a unified /usr/local/sbin, and leave existing installations unchanged.

[1] https://archlinux.org/news/binaries-move-to-usrbin-requiring-update-intervention/
https://lists.archlinux.org/pipermail/arch-dev-public/2012-March/022625.html

[2] https://fedoraproject.org/wiki/Changes/Unify_bin_and_sbin

[3] https://systemd.io/THE_CASE_FOR_THE_USR_MERGE/

[4] systemd commit 0f36a4c897ff53eb
systemd/systemd@0f36a4c897ff53eb
systemd/systemd#32389
systemd/systemd#32337

https://bugzilla.redhat.com/show_bug.cgi?id=2400220

I don't know how to test this change properly. I copied the file to /etc/tmpfiles.d/rpm-ostree-0-integration.conf on a Fedora 42 Silverblue virtual machine, deleted /var/usrlocal/sbin, and rebooted. That seemed to give the expected results.

Some Linux distributions like Arch and Fedora are in different stages of unifying their 'bin' and 'sbin' directories [1,2]. This has been implemented by replacing the /sbin and /usr/sbin directories with relative symbolic links to their 'bin' counterparts. Since, both those distributions have also adopted the /usr merge [3], this means: * /sbin → usr/bin or usr/sbin * /usr/sbin → bin One notable difference between Arch and Fedora is their handling of /usr/local/sbin. It remains a separate directory on the former, and has been unified with /usr/local/bin in the package based variants of the latter but not those that use rpm-ostree. Fedora variants that use rpm-ostree end up with different values for the PATH environment variable because systemd sets it based on the state of the 'bin' and 'sbin' directories [4]. Since, there are no Arch variants that use rpm-ostree and there are popular Fedora ones that do, and unifying /usr/local/sbin with its 'bin' counterpart leads to a simpler PATH, it seems reasonable to replicate package based Fedora. This will make new installations of rpm-ostree based operating systems use a unified /usr/local/sbin, and leave existing installations unchanged. [1] https://archlinux.org/news/binaries-move-to-usrbin-requiring-update-intervention/ https://lists.archlinux.org/pipermail/arch-dev-public/2012-March/022625.html [2] https://fedoraproject.org/wiki/Changes/Unify_bin_and_sbin [3] https://systemd.io/THE_CASE_FOR_THE_USR_MERGE/ [4] systemd commit 0f36a4c897ff53eb systemd/systemd@0f36a4c897ff53eb systemd/systemd#32389 systemd/systemd#32337 https://bugzilla.redhat.com/show_bug.cgi?id=2400220

openshift-ci · 2025-10-10T19:41:55Z

Hi @debarshiray. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

gemini-code-assist

Code Review

This pull request correctly updates the tmpfiles.d configuration to create /usr/local/sbin as a symbolic link to bin, aligning rpm-ostree based systems with package-based Fedora variants. This change is well-reasoned and helps in achieving a more consistent PATH environment variable. My review includes a minor suggestion to add a comment to the configuration file to preserve the context of this change for future maintainability.

gemini-code-assist · 2025-10-10T19:43:11Z

src/app/rpm-ostree-0-integration.conf

 d /usr/local/include 0755 root root -
 d /usr/local/lib 0755 root root -
-d /usr/local/sbin 0755 root root -
+L /usr/local/sbin - - - - bin


For improved maintainability, it would be beneficial to add a comment explaining why /usr/local/sbin is being created as a symbolic link, especially since other entries in this file create directories. This captures the valuable context from your pull request description directly in the configuration file for future reference.

# Unify /usr/local/sbin with /usr/local/bin for a consistent PATH, # matching package-based Fedora variants. L /usr/local/sbin - - - - bin

jlebon · 2025-10-10T21:47:03Z

This seems sane though I think it will also affect el10-based composes using the latest rpm-ostree, where the sbin merge hasn't happened yet. Ideally we would scope this to just systems that had it merged, but we're not well setup for that here with the tmpfiles dropin being a shared thing.

jlebon · 2025-10-10T21:47:07Z

/ok-to-test

openshift-ci · 2025-10-10T22:32:06Z

@debarshiray: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/fcos-e2e	`67d2134`	link	true	`/test fcos-e2e`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

keszybz

Looks reasonable.

debarshiray · 2025-10-12T21:46:37Z

Thanks for the reviews!

This seems sane though I think it will also affect el10-based composes using the latest rpm-ostree, where the sbin merge hasn't happened yet.

I didn't realize that the upstream Git main can impact RHEL 10.

Ideally we would scope this to just systems that had it merged, but we're not well setup for that here with the tmpfiles dropin being a shared thing.

Should I add a build flag to configure.ac?

Or, we can wait until RHEL 10 is old enough to not be affected by this? This isn't that urgent. :)

debarshiray · 2025-10-12T21:59:30Z

tests/compose/libbasic-test.sh

 # Check symlinks injected into the rootfs.
 ostree --repo="${repo}" ls "${treeref}" /usr/local | grep '^l00777' > symlinks.txt
 assert_file_has_content_literal symlinks.txt '/usr/local -> ../var/usrlocal'
+assert_file_has_content_literal symlinks.txt '/usr/local/sbin -> bin'


After playing with ostree ls on Fedora Silverblue, it seems that OSTree doesn't know about what's going on inside /var/usrlocal. So, maybe I should leave this out? Is there some other way to test it? Does it need a test?

cgwalters · 2025-10-15T19:03:04Z

Or, we can wait until RHEL 10 is old enough to not be affected by this? This isn't that urgent. :)

we actually only just created a rhel9 branch (ref #5500 ) and yes currently targeting rhel10 with main still.

I would prefer to avoid creating a rhel10 branch unless it was really needed.

Should I add a build flag to configure.ac?

Yeah this sounds sane. That build flag could default to detecting from the build environment, which would avoid needing to set it for Fedora derivatives which always default to building with host == target.

jlebon · 2025-10-15T19:54:51Z

Another approach which might be less cumbersome than a compile flag is to:

split out that tmpfiles.d line into its own dropin (i.e. two dropins; one for each of merged, unmerged)

auto-detect in postprocess_final whether /usr/sbin in the target root is symlinked and use that to determine which dropin to inject. E.g. after this related logic:

rpm-ostree/src/libpriv/rpmostree-postprocess.cxx

Lines 468 to 491 in 9d79d6b

    
           if (!glnx_file_copy_at (pkglibdir_dfd, "rpm-ostree-0-integration.conf", NULL, rootfs_dfd, 
        
                                   tmpfiles_integration_path, 
        
                                   GLNX_FILE_COPY_NOXATTRS, /* Don't take selinux label */ 
        
                                   cancellable, error)) 
        
             return glnx_prefix_error (error, "Failed to copy rpm-ostree-0-integration.conf"); 
        
           if (treefile.get_opt_usrlocal () == rpmostreecxx::OptUsrLocal::StateOverlay) 
        
             { 
        
               if (!glnx_file_copy_at ( 
        
                       pkglibdir_dfd, "rpm-ostree-0-integration-opt-usrlocal-compat.conf", NULL, rootfs_dfd, 
        
                       "usr/lib/tmpfiles.d/rpm-ostree-0-integration-opt-usrlocal-compat.conf", 
        
                       GLNX_FILE_COPY_NOXATTRS, /* Don't take selinux label */ 
        
                       cancellable, error)) 
        
                 return FALSE; 
        
             } 
        
           else 
        
             { 
        
               if (!glnx_file_copy_at (pkglibdir_dfd, "rpm-ostree-0-integration-opt-usrlocal.conf", NULL, 
        
                                       rootfs_dfd, 
        
                                       "usr/lib/tmpfiles.d/rpm-ostree-0-integration-opt-usrlocal.conf", 
        
                                       GLNX_FILE_COPY_NOXATTRS, /* Don't take selinux label */ 
        
                                       cancellable, error)) 
        
                 return FALSE; 
        
             }

debarshiray · 2025-10-20T00:58:16Z

Thanks for the suggestions, @cgwalters & @jlebon ! I will look into it.

openshift-ci bot added the needs-ok-to-test label Oct 10, 2025

debarshiray force-pushed the wip/rishi/unify-usr-local-sbin-bin branch from 9db5c81 to 67d2134 Compare October 10, 2025 19:41

gemini-code-assist bot reviewed Oct 10, 2025

View reviewed changes

openshift-ci bot added ok-to-test and removed needs-ok-to-test labels Oct 10, 2025

keszybz approved these changes Oct 11, 2025

View reviewed changes

debarshiray commented Oct 12, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

app, tests: Use the same PATH as on package based Fedora variants #5507

app, tests: Use the same PATH as on package based Fedora variants #5507

debarshiray commented Oct 10, 2025

Uh oh!

openshift-ci bot commented Oct 10, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Oct 10, 2025

Uh oh!

jlebon commented Oct 10, 2025

Uh oh!

jlebon commented Oct 10, 2025

Uh oh!

openshift-ci bot commented Oct 10, 2025

Uh oh!

keszybz left a comment

Uh oh!

debarshiray commented Oct 12, 2025

Uh oh!

debarshiray Oct 12, 2025

Uh oh!

cgwalters commented Oct 15, 2025

Uh oh!

jlebon commented Oct 15, 2025

Uh oh!

debarshiray commented Oct 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

app, tests: Use the same PATH as on package based Fedora variants #5507

Are you sure you want to change the base?

app, tests: Use the same PATH as on package based Fedora variants #5507

Conversation

debarshiray commented Oct 10, 2025

Uh oh!

openshift-ci bot commented Oct 10, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

jlebon commented Oct 10, 2025

Uh oh!

jlebon commented Oct 10, 2025

Uh oh!

openshift-ci bot commented Oct 10, 2025

Uh oh!

keszybz left a comment

Choose a reason for hiding this comment

Uh oh!

debarshiray commented Oct 12, 2025

Uh oh!

debarshiray Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

cgwalters commented Oct 15, 2025

Uh oh!

jlebon commented Oct 15, 2025

Uh oh!

debarshiray commented Oct 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants