Repository navigation
Make download_to_drive purely async #4993
Description
What kind of feature are you missing? Where do you notice a shortcoming of PTB?
The telegram.File.download_to_drive method is an async coroutine, but it performs synchronous file I/O (.write_bytes, shutil.copyfile).
These blocking calls can stall the asyncio event loop, especially with large files or heavy I/O load. This undermines the non-blocking promise of an async method and can hurt a bot's responsiveness.
Describe the solution you'd like
The method should be made fully asynchronous by replacing all synchronous file operations with non-blocking alternatives from the aiofiles library.
This change would make the entire file download process truly non-blocking.
I am happy to prepare and submit a Pull Request to implement this feature.
Describe alternatives you've considered
No response
Additional context
No response
Activity
Poolitzer commentedon Oct 9, 2025
@aviadlevy I would vote in favor of it, having more asyncio support is great. The library looks well documented and used, thus implementing won't be a big issue.
Bibo-Joshi commentedon Oct 9, 2025
Hi. I think we discussed aiofiles at some point before … anyways, as it would be a 3rd party dependency and we're rather reluctand to add those, I would say that it should at most be an optional dependency. Also, when introducing that, I'd strongly vote to apply it to as many file operations in our lib as possible for consistency
Bibo-Joshi commentedon Oct 9, 2025
For reference: Internal discussion can be found at https://t.me/c/1494805131/24140
aviadlevy commentedon Oct 9, 2025
@Poolitzer @Bibo-Joshi Thanks for the quick response.
Using
loop.run_in_executor()orasyncio.to_threadare another ways to achieve this and avoid a new dependency.I'm still willing to work on a PR for this (once a solution is agreed upon). My only concern is that since the changes should be applied consistently for all file ops, I might miss some areas as I'm not familiar with the entire codebase.
Given this, it might be more efficient for a core contributor to handle the implementation. However, the offer still stands if you can provide some guidance.
Bibo-Joshi commentedon Oct 9, 2025
TBF I'm not overly familiar with the nitty-gritty details of how file operations actually work on OSes …
From what I've gathered by a bit of research:
asyncio.to_threadas easy-dependency-free solution would hence make sense. It might even be faster than aiofiles / aiopath depending on the scenario, see Add performance benchmarks to the readme mosquito/aiofile#88asynciofor file operations heavily depends on the concrete use case (e.g. frequency of concurrent operations, file sizes)As PTB is a general purpose library, we usually try to aim for a "one size fits most" accompanied by "here are tools to use for special use cases" approach. That's why I'd like to avoid rushing to some action without some prior discussion 👼
Regarding consistency of application: Sorry for being a bit unspecific there. I'm okay with applying things step-by-step in different PRs. But at least an initial analysis on "where should this be applied" should be done IMO so that we could keep track.
github-actions commentedon Oct 16, 2025
This issue has been automatically closed due to inactivity. Feel free to comment in order to reopen or ask again in our Telegram support group at https://t.me/pythontelegrambotgroup.
Bibo-Joshi commentedon Oct 26, 2025
Hi @aviadlevy sorry for the late reply. We had a bit of internal discussion, also with one of our emeritus maintainers who is way more proficient in these OS-level details than the currently active team. The bottom line is basically that improving the concurrency of file operations is rather use-case specific (frequency & size of file operations, underlying OS, server capabilties …) and hard to handle for us as general purpose library. What PTB does offer is
download_to_memory. Using that gives you the option to handle the write operations in a tailored way that suits your use cases.As such I currently don't see that we'd be going in the originally proposed direction. However, I'm very much interseted to hear about specific examples & use-cases if
download_to_memory+ custom writing is still a bottleneck for you :)blixor7 commentedon Dec 15, 2025
I've run into this exact issue in production, large file downloads, stalling event loops during peak loads. In my projects, I bridged the gap by wrapping sync file writes in asyncio.to_thread() after using download_to_memory(), which gave us solid non-blocking behavior without deps. For example, in a recent DEX explorer bot handling 10k+ concurrent file ops on Ubuntu servers, this cut response times by ~40% under load compared to raw sync writes.
That said, offering asyncio.to_thread() or even loop.run_in_executor, directly in download_to_drive as an opt-in could make PTB more battle-tested for heavy users while keeping it lightweight. I've got benchmarks from my setups showing it outperforms aiofiles in thread-pool scenarios on Linux.