[Bug]: Gateway model proxy creates an HTTP client per request and never closes it #4489

Steps to reproduce

1. Create a gateway with a domain and run a service with model on it, for example vLLM serving Qwen/Qwen3-0.6B. Its model endpoint is https://gateway.<domain>.

2. On the gateway host, count the gateway app's open file descriptors:

   ls /proc/$(pgrep -f dstack._internal.proxy.gateway.main)/fd | wc -l

3. Send 10,000 streaming chat completion requests to the model endpoint, 100 at a time:

   import asyncio
   from openai import AsyncOpenAI
   
   client = AsyncOpenAI(base_url="https://gateway.<domain>", api_key="<dstack token>")
   sem = asyncio.Semaphore(100)
   
   async def request():
       async with sem:
           stream = await client.chat.completions.create(
               model="Qwen/Qwen3-0.6B",
               messages=[{"role": "user", "content": "Count to 20"}],
               stream=True,
           )
           async for _ in stream:
               pass
   
   async def main():
       await asyncio.gather(*(request() for _ in range(10_000)))
   
   asyncio.run(main())

4. Count the gateway app's open file descriptors again.

Actual behaviour

post_chat_completions gets its HTTP client from get_service_replica_client:

On a gateway the service has a domain, so get_service_replica_client creates a new httpx.AsyncClient on every call, and nothing closes it:

Each request opens a new connection to nginx, and the connection stays open until Python's garbage collector frees the client. A local script with the same pattern (httpx 0.28.1, a new client per streaming request to a local server, never closed) leaves about 280 client connections, 560 sockets counting both ends, open after 10,000 requests with 100 in flight, until garbage collection runs. With await client.aclose() after each request, none are left.

Under load, the model proxy holds hundreds of idle sockets, which adds to the file descriptor pressure behind #4338.

Expected behaviour

The model proxy reuses one HTTP client per service and closes it when the service is unregistered, so open sockets don't grow with the number of requests.

dstack version

master (8fb64b8)

Additional information

#4351 made the gateway recover after high load (#4338). The per-request client is still on master.