Repository navigation
OkHttp client should not bail out when response body is >2gb - #410
codefromthecrypt merged 4 commits into
Conversation
|
I guess the question is why are you using feign for requests where the response body is > 2GB? |
|
I'm downloading files. This allows me to reuse my connection/retrial/errorDecoder code for the file download, which generally works really well! |
|
the javadoc on this field says the following: /**
* length in bytes, if known. Null if not. <br> <br><br><b>Note</b><br> This is an integer as
* most implementations cannot do bodies greater than 2GB. Moreover, the scope of this interface
* doesn't include large bodies.
*/Feign was made for control APIs, and most of the time they don't attempt to send large content without chunking or breaking it up into pieces. There's some assumptions that both the request and response can fit into a byte array, so just changing here might not be the end of this story. I don't really want to complicate the javadoc saying it might not be set because the body is larger than 2GB, as that makes the feature scope of feign muddy. It might be ok to make a special repo for big content, with disclaimers that feign wasn't written to be a large content api. For example, a BigBody interface and a trusted implementation like OkHttp could like there. It would make sense to have resume if doing something so huge as restarting from byte 4.9GB would be awful. |
I think that is determined by the registered decoder, we use a custom
Currently the I'm not especially worried about the body integer length anymore because I can parse the length header externally. I'll push a change to this branch to pull out the long->Integer conversion to a shared static function in |
6ab1b89 to
23594e6
Compare
|
I don't like toBodyLength.. it doesn't satisfy rule of three to be a util. let's reuse the lines we would have added to prove things work. Personally, I'd rather throw an exception for all of the clients when they hit >2GB, like we did in OkHttp, but I think the below could be less draconian and help you progress without burdening others. swap the lines in the toBodyLength commit for below, keeping the actual logic local (in some cases as they were).
sound reasonable? For super brownie-points on a rainy day... extract a pluggable version of the http client tests :) |
Is this possible!? I've tried to do it in the past, but unfortunately the mock server was smart enough to return the actual length of the provided body.
Makes sense, my rationale for the Utility was to define shared behavior in a shared place, that way if it gets modified in the future, we don't end up with different behavior between client wrapper implementations. Thanks for the feedback, I'll put together some tests! |
Instead, we return null length and the response can be processed similarly to a chunked body.
Default, apache, and okhttp client tests extend the Abstract test, so that we can avoid duplicated code while fully testing each implementation.
23594e6 to
39c2293
Compare
|
Updated to share Client tests across implementations, no longer shares body length utility, but documents expected functionality in the interface. I'm not sure how to generate a large response body, MockWebServer only allows a Buffer and String body type, both of which appear to require the full contents in memory. |
|
in case we have timezone trouble.. here's a thought. I was thinking just
add the "Content-Length" header in the mock response, as opposed to making
a special body. If that doesn't work, well.. things become more interesting.
|
| server.enqueue(new MockResponse().setStatus("HTTP/1.1 " + 200)); | ||
|
|
||
| TestInterface api = | ||
| Feign.builder().target(TestInterface.class, "http://localhost:" + server.getPort()); |
|
nicely done. only a couple cleanups. this is notable enough to add to CHANGES I think |
OkHttp client should not bail out when response body is >2gb Instead, we return null length and the response can be processed similarly to a chunked body. Also, shares common client tests in AbstractClientTest
OkHttp client should not bail out when response body is >2gb Instead, we return null length and the response can be processed similarly to a chunked body. Also, shares common client tests in AbstractClientTest
Instead, we return null length and the response can be processed
similarly to a chunked body.
I'd like to eventually expose the body length as a long, but I'm not sure how we'd prefer to do that.
We could change the return type from Integer -> Long, but that would be api breaking.
Or alternatively add a second method to Body,
Long lengthLong();(orLong getLength();) and deprecate the integer value.