Repository navigation
Modules Integration Blocking Run Loop with ASGI Integration #2342
Description
How do you use Sentry?
Sentry Saas (sentry.io)
Version
1.28.1
Steps to Reproduce
Initialize the sentry SDK in a blank FastAPI application. Enable the following integrations:
FastApiIntegration(transaction_style="url"),
HttpxIntegration(),
LoggingIntegration(level=logging.INFO, event_level=logging.FATAL),
RedisIntegration(),
SqlalchemyIntegration(),
The first time you call report_exception the modules integration will load and cache all the modules you have for reporting purposes. Doing this can actually be fairly slow and causes run loop blocking which delays other requests. The call stack in question here is the following:
__call__ (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/fastapi/applications.py:282)
_sentry_patched_asgi_app (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/starlette.py:348)
_run_asgi3 (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:141)
_run_app (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:186)
_capture_exception (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/asgi.py:62)
capture_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/hub.py:350)
capture_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/client.py:522)
_prepare_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/client.py:284)
wrapper (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/scope.py:77)
apply_to_event (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/scope.py:647)
processor (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/modules.py:76)
_get_installed_modules (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/modules.py:57)
_generate_installed_modules (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/sentry_sdk/integrations/modules.py:32)
metadata (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/importlib_metadata/__init__.py:455)
read_text (/root/.pyenv/versions/3.10.11/lib/python3.10/site-packages/importlib_metadata/__init__.py:837)
read_text (/root/.pyenv/versions/3.10.11/lib/python3.10/pathlib.py:1134)
open (/root/.pyenv/versions/3.10.11/lib/python3.10/pathlib.py:1119)
open64 (/lib/x86_64-linux-gnu/libc-2.31.so:0)
The problem is that within the async def _run_app method of the ASGI middleware the capture_exception method which ends up directly calling _generate_installed_modules is synchronous so any IO it does directly will block the main run loop and prevent other coroutines from running.
To fix it should either run in a thread or not do any blocking IO. If the latter I think it would need to provide a way of preloading the modules at initialization time for example.
Expected Result
Run loop is not blocked when loading modules
Actual Result
Run loop is blocked when loading modules
Activity
antonpirker commentedon Sep 5, 2023
Hey @ejlangev
Thanks for reporting this! But it on our (big) list of things to fix. Will take some time though.
But PRs are always welcome, if you want to give fixing it a try!
antonpirker commentedon Jul 18, 2024
Hey @ejlangev !
It has been a while, but in November 2023 with version 1.37.0 we released a refactoring of
_generate_installed_modules.This change should speed up every call to
_generate_installed_modulesafter the first one considerably.Can you check if this solves your problem and makes your FastAPI application not blocking the run loop?
getsantry commentedon Aug 9, 2024
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you remove the label
Waiting for: Community, I will leave it alone ... forever!"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀