refactor(http-clients)!: Move redirect and cookie handling into a shared HttpClient base class - #2349

Description

HttpClient now has a single public method, send_request, which returns the response before its body is read. A client implements only _fetch, which sends one request. The base class follows redirects, sends and stores the session cookies on every hop, and drops credentials when a redirect leaves the origin, the same way for every client. crawl() and stream() are removed.

The timeout of send_request covers the redirect chain and reading the body, every client takes follow_redirects, and HttpResponse is an abstract base class that closes the response when used as an async context manager. The v2 upgrade guide describes what changes for existing code and custom clients.

Issues

• Closes: Reconsider HttpClient interface #874
• Part of Move redirect and cookie handling into a shared HttpClient base class #2307