Skip to content

rgw/s3vectors: link librgw_lancedb dynamically - #73063

Open
mheler wants to merge 1 commit into
ceph:mainfrom
mheler:wip-rgw-lancedb-link
Open

mheler wants to merge 1 commit into
ceph:mainfrom
mheler:wip-rgw-lancedb-link

Conversation

@mheler

@mheler mheler commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Fixes: https://tracker.ceph.com/issues/82052

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands

You must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.

@mheler mheler changed the title rgw/s3vectors: move lancedb into its own static library rgw/s3vectors: split s3vector out of rgw_common into a static library Oct 25, 2026
@mheler mheler changed the title rgw/s3vectors: split s3vector out of rgw_common into a static library rgw/s3vectors: split s3vector out of rgw_common Oct 25, 2026
@mheler
mheler marked this pull request as ready for review October 25, 2026 08:21
@mheler
mheler requested a review from a team as a code owner October 25, 2026 08:21

@yuvalif yuvalif left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should take a different approach here.
this breaks the way the way that we do both SAL and the frontend handlers.

note that what we need to split is the librgw_lancedb not the s3vectors code.
if this is still not possible, i would take the approach that librgw_lancedb should be dynamically linked instead.

@mheler
mheler force-pushed the wip-rgw-lancedb-link branch from d338dd5 to 5e87576 Compare October 25, 2026 12:39
@mheler
mheler requested a review from a team as a code owner October 25, 2026 12:39
@mheler mheler changed the title rgw/s3vectors: split s3vector out of rgw_common rgw/s3vectors: link librgw_lancedb dynamically Oct 25, 2026
@mheler
mheler requested a review from yuvalif October 25, 2026 12:40
@mheler

mheler commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor Author

we should take a different approach here. this breaks the way the way that we do both SAL and the frontend handlers.

note that what we need to split is the librgw_lancedb not the s3vectors code. if this is still not possible, i would take the approach that librgw_lancedb should be dynamically linked instead.

Thanks. I've taken the approach that librgw_lancedb should be dynamically linked, and pushed that up.

@yuvalif
yuvalif requested a review from cbodley October 25, 2026 14:35
@yuvalif

yuvalif commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

looks like arm64 build is failing due to link issues:

 FAILED: src/rgw/rgw-lancedb-prefix/src/rgw-lancedb-stamp/rgw-lancedb-build lib/librgw_lancedb.so.1 /ceph/build/src/rgw/rgw-lancedb-prefix/src/rgw-lancedb-stamp/rgw-lancedb-build /ceph/build/lib/librgw_lancedb.so.1 

rgw_common put the Rust archive on every dependent binary's link line.
The jammy and noble deb builds run out of memory when these links run in
parallel.

Build and package librgw_lancedb as a shared library, then link
rgw_common against it. A link against it only reads its dynamic symbol
table. Move the aarch64 lld option to the cargo link, since the C++
links no longer carry the Rust code, and have cargo link with CMake's C
compiler, which passed the lld check, instead of its default cc. Keep
the test binary's own provider symbols private so calls from the test
environment resolve inside librgw_lancedb.so.

Assisted-by: Claude Opus 5.5
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
@mheler
mheler force-pushed the wip-rgw-lancedb-link branch from 5e87576 to 68d2083 Compare October 25, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants