Skip to content

ConnectionHandler's per-destination connection concurrency (asyncio.Semaphore(5)) is hardcoded with no way to configure it #8322

Description

@alt-efi

Problem

ConnectionHandler.__init__ (mitmproxy/proxy/server.py) hardcodes a concurrency limit of 5 open connections per destination address:

self.max_conns = collections.defaultdict(lambda: asyncio.Semaphore(5))

open_connection() holds that semaphore slot for the connection's entire lifetime, not just the handshake:

async with self.max_conns[command.connection.address]:
    ...
    try:
        await self.handle_connection(command.connection)
    finally:
        ...

So a 6th concurrent connection to the same destination address blocks until one of the existing 5 fully closes — and there's no Options field, --set flag, or any other configuration surface for this value; it's a bare literal with zero visibility (confirmed against current main).

This is a real problem for anything that legitimately needs many concurrent connections to one host — e.g. a browser client that multiplexes many requests to one origin over HTTP/2, which mitmproxy re-originates as separate HTTP/1.1 connections upstream (http2=false on the outbound leg), or any tool proxying high-concurrency traffic toward a single target. Related, though it never landed on this specific mechanism: #4451 (closed 2021) discusses similarly-shaped "requests hang for a long time" symptoms without conclusively tying it to this cap.

Proposed fix

Make the cap a configurable Options field, defaulting to 5 to preserve current behavior:

# mitmproxy/options.py, near tcp_timeout
self.add_option(
    "connection_max_per_address",
    int,
    5,
    """
    Maximum number of concurrent connections mitmproxy will hold open to any single
    destination address. A destination that already has this many connections open
    will delay new ones until one of the existing connections closes.
    """,
)
# mitmproxy/proxy/server.py, ConnectionHandler.__init__
self.max_conns = collections.defaultdict(
    lambda: asyncio.Semaphore(context.options.connection_max_per_address)
)

Happy to open a PR with this exact change (plus a test) if the shape looks right — opening this issue first per CONTRIBUTING.md's guidance to talk before a larger change, though this one is small.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions