Skip to content

Conversation

@nevack
Copy link
Member

@nevack nevack commented Nov 6, 2025

Very rough 1to1 implementation based on libuv instead of libevent.
It is functional, tested with macOS client.
The solely purpose of this PR is just to provide a bootstrap for @ckerr
#2462 (comment)

For now, only tr_peerIo is somewhat properly abstracted from async event library.
tr_session (especially BoundSocket) and tr_session_thread also need some real abstraction.

Left out of scope:

  • rpc-server - libuv has no analogue for evhttp, need to figure out something for http.
  • watchdir-kqueue and daemon - I just got bored by the time I got here.

Afterthoughts: Maybe, just maybe, asio is the path forward, it is a C++ library after all. Also there's uvw, a C++ wrapper around libuv, but I did not want to get into this before getting working plain libuv impl.

@nevack
Copy link
Member Author

nevack commented Nov 8, 2025

@ckerr Please ACK.

Are you interested in me helping to improve this?
Or do you have your own ideas and prefer to rebuild it from the ground up yourself?

@ckerr
Copy link
Member

ckerr commented Nov 8, 2025

@nevack I'm extremely interested in seeing work on this, yes! 😻

The big question IMO is the choice of library and I'd like to see how strongly you feel about using libuv here.

In libuv's favor:

  • It's very well-maintained.
  • It's got better out-of-the box async disk IO support.
  • I'm more familiar with it than asio because I work on libuv-adjacent code in my day job & am happy with it.
  • You're apparently familiar with it too, since you pushed up this PoC.

In standalone asio's favor:

  • It's also very well-maintained
  • Much better modern C++ integration
  • It's got an out-of-the-box replacement for rpc-server's use of evhttp.
  • It's a header-only library, so in theory we should only need to compile & link the parts we actually use
  • It's also got out-of-the-box support for an http / https client. Theoretically we could drop both libevent and libcurl.

Even though I'm less familiar with asio, it seems like it has much more upside for libtransmission's needs. WDYT? Would you be open to the idea of using that instead of libuv?

@nevack
Copy link
Member Author

nevack commented Nov 8, 2025

You're apparently familiar with it too, since you pushed up this PoC.

To be completely honest — not at all.
This was just an exercise for me; I just wanted to check if it’s doable and at what cost.
That leads to:

The big question IMO is the choice of library, and I'd like to see how strongly you feel about using libuv here.

I don’t really have a strong preference.
In my opinion, libuv being a C (and not a C++) library isn’t much of a concern, since writing a minimal wrapper for lifecycle and memory management isn’t difficult — as you’ve already done for libevent.

What I do see as a challenge with libuv is the additional research needed for HTTP server implementation. I don’t currently know of any lightweight and modern library that can be used in Transmission alongside libuv, and I don’t think a modern HTTP server is something that’s easily hand‑rolled 😄

WDYT? Would you be open to the idea of using that instead of libuv?

Yeah, I'm okay with that.

Here’s my proposal:

In the current implementation, you can see that many class hierarchies are duplicated.
You can even cleanly diff tr-lpd-libuv.cc vs. tr-lpd.cc or tr-udp-libuv.cc vs. tr-udp.cc, and notice that the only differences are in the underlying event-handling library.
This duplication is, of course, unnecessary and should be addressed.

What really needs to be done is to abstract these classes away from the specific event-handling library used so that the implementation can be swapped between libevent, libuv, asio, or something new in the future.
I don’t yet know what this abstraction will look like, but tr_session / tr_session_thread should not expose details like event_base or uv_loop. Instead, they should rely on abstracted primitives (Timers, Sockets, etc.).

Some parts are already reasonably well abstracted:

  • TimerMaker existed before this PR.
  • I introduced PeerIoEventHandler here and refactored tr_peerIo to use it.

Creating proper abstraction would even allow us to compare implementations and decide which one to pursue further.

Do you agree with this proposal?
If so, I can continue improving this PoC.
Perhaps the parts that abstract Transmission code from libevent could be merged separately to allow an independent review of the event-layer design and the respective libuv/asio/etc. implementations.

@ckerr
Copy link
Member

ckerr commented Nov 8, 2025

I agree on the idea of an abstraction layer. Violating that principle is what's made it so time-consuming to pry libevent out of libtransmission.

If I'm reading your last comment right, the next step you want me to do is review abstractions like PeerIoEventHandler so we can brainstorm on what the interfaces should look like. Is that correct? If so, I'm happy to self-assign for that task. Otherwise, lmk if there's something else you need from me.

@nevack
Copy link
Member Author

nevack commented Nov 8, 2025

If I'm reading your last comment right, the next step you want me to do is review abstractions like PeerIoEventHandler so we can brainstorm on what the interfaces should look like. Is that correct?

Yes, that would be a great starting point. Thank you!

I will push revised code now with fixed code style check.

@ckerr
Copy link
Member

ckerr commented Nov 8, 2025

self-assigning for the subtask of reviewing the abstractions introduced in this PoC as per #7770 (comment)

@ckerr ckerr self-assigned this Nov 8, 2025
fmt::fmt-header-only
vdkqueue
${SPARKLE_FRAMEWORK}
"-L/opt/homebrew/lib/ -luv" # TODO: proper libuv library intergration into the build infra.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: intergration -> integration

@Neustradamus
Copy link

We hope a change into the libevent project team members...

@nevack
Copy link
Member Author

nevack commented Nov 11, 2025

@ckerr Some progress update from my end.

Moved everything libevent releated to *-libevent.{h,cc} files, and moved newly introduced libuv impls to *-libuv.{h,cc}.

Also moved uv_loop creation from tr_session_thread ctor to inside session_thread_func.
AFAIK uv_loop should be inited and run on the same thread and we must run it inside session thread.
Your knowledge would be invaluable here.

So, as previously said, I continued working on two distinct parts here:

  1. Introduce abstraction layers - rough, but done. Not good enough to put it into separate PR for testing though. Waiting for feedback and suggestions. For now I clearly see I need to create common socket read/write event wrapper, as I already duplicated very similar code 3+ times. This could be some RAII class, that takes in socket and some callback, starts polling in ctor and stops in dtor.
  2. Trying to implement libuv based counterparts - macOS client starts and works (not fully tested yet), no rpc-server for known reasons, still did not touch daemon yet. I feel stupid for not starting with daemon and sticking to macOS for some reason. Anyway, your knowledge about libuv would be invaluable here. I don't have enough experience to verify that uv-based session thread is correct.

@nevack
Copy link
Member Author

nevack commented Nov 11, 2025

@ckerr

Introduced SocketEventHandler in libtransmission/socket-event-handler.h.
This class encapsulates the work related to polling socket events.
Everything migrated to use it.
As the result – no more duplication, and no dependency on specific async event library outside of SocketEventHandler implementations.

@nevack
Copy link
Member Author

nevack commented Nov 12, 2025

@ckerr PTAL.

I’m not sure where to move forward from here without your feedback.

Also CC @tearfur — maybe you have some time to take a look?
I think you’re quite familiar with the parts I’m touching here.

@tearfur
Copy link
Member

tearfur commented Nov 12, 2025

I'm definitely interested in this work.

I have zero experience with libuv and asio. So before I familiarise myself, please carry on without me.

What I can do though is help evaluate the abstraction layers. I'll try get my hands on them soon.

@nevack
Copy link
Member Author

nevack commented Nov 12, 2025

I have zero experience with libuv and asio. So before I familiarise myself, please carry on without me.

At least I'm not alone here 😄

What I can do though is help evaluate the abstraction layers.

That would definitely help, thank you!

I preserved commit history for now, so you can see step-by-step progress if needed.

@ckerr
Copy link
Member

ckerr commented Nov 12, 2025

Just a courtesy checkin: Even though I've been active in other PRs and quiet on this one, I haven't forgotten about this. It's just a "heavier" PR that I need to have a good block of time to stare at & think about.

@nevack
Copy link
Member Author

nevack commented Nov 12, 2025

Just a courtesy checkin: Even though I've been active in other PRs and quiet on this one, I haven't forgotten about this. It's just a "heavier" PR that I need to have a good block of time to stare at & think about.

By saying "At least I'm not alone here" I really meant "I'm not alone in having very little experience with libuv or asio."
Sorry if that came across as offensive — that wasn’t my intention at all.

@ckerr
Copy link
Member

ckerr commented Nov 12, 2025

I missed that. Thanks for the clarification but FWIW I didn't read it that way at all!

I was just wanting to check in because of the 2025 track record of letting some PRs gather dust 👍

@nevack
Copy link
Member Author

nevack commented Nov 14, 2025

Tests crash on Debian 11 with

libtransmission-test: ./src/unix/core.c:930: uv__io_stop: Assertion `loop->watchers[w->fd] == w' failed.

My investigation lead me to this:
libuv/libuv#2686
So this was fixed in 1.41.0
https://github.com/libuv/libuv/releases/tag/v1.41.0

https://packages.debian.org/bullseye/libuv1-dev
And Debian 11 has 1.40.0

That's a bummer 😞

@nevack
Copy link
Member Author

nevack commented Nov 15, 2025

There are still can be some object lifetime issues as can be seen in previous runs of sanitizer tests.

@nevack
Copy link
Member Author

nevack commented Nov 15, 2025

This is getting a bit out of hand with 38 commits, but I've got so far:

  • Almost everything is buildable

    • one exception is linux build for s390x (do we need to support discontinued IBM mainframe architecture?)
    • had to bump Debian from 11 to 12 to use newer system provided version of libuv
    • had to bump Windows minimum version from Vista to 10 and add explicit linking with userenv and dbghelp.
  • Tests and sanitized builds are passing.

    • Indeed, storing tr_peerIo, which is shared_ptr, as plain this in a lambda was a bad idea, so passing weak_ptr instead fixed sanitized problems.
    • Added guard inside rpc server to not init if no libevent base was created.

I think I won't spend more time on this before some initial review, as I just don't know what else can be done here.

For @ckerr and @tearfur

I suggest to ignore all build stuff, it's a mess for now, this PR is PoC and Draft for a reason.
Please, focus on new SocketEventHandler class and its integration in different parts of libtransmission.
I can move this into a separate PR, as abstraction layer can be reviewed and merged independently from libuv implementation.

For @ckerr

If we decide to go with libuv impl, maybe I have found a solution for http server.
There's a very popular header-only library for HTTP with SSL support
https://github.com/yhirose/cpp-httplib
This library uses blocking socket IO with its own thread pool, which can be overridden.
https://github.com/yhirose/cpp-httplib#override-the-default-thread-pool-with-yours

@LaserEyess
Copy link
Contributor

I want to help review/test this, and I think I may have some time for that this weekend, but I wanted to make one comment on something in this thread: please, please do not replace libcurl as the http client unless the replacement has support for connection reuse and multiplexing. Transmission might the only client that gets this right, and it truly matters with 10k+ announces when it comes to tracker traffic. That's not even to mention what the trackers see, you can measure it in GBs. To me this would be a devastating efficiency regression.

@jggimi
Copy link
Contributor

jggimi commented Nov 20, 2025

I tested this on OpenBSD. It builds with a dependency on the libuv in /usr/local/lib.

@tearfur
Copy link
Member

tearfur commented Dec 4, 2025

Small update: I've looked at this PR, and this overlapped with a tr_peer_socket refactor that I've wanted to do for a long time. I'll have to write up a draft of that refactor to sort out my ideas, so I will follow up with details later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

7 participants