This topic was originally brought up in PR #297.
StreamReader class allows to implement protocols parsers in a very convenient way already. Instead of managing buffers and callbacks, one can layout even complex protocols as a series of await read(..), await readline(..) (and soon await readuntil(..)) calls.
I think that adding an ability to adjust the buffer limit per operation is a great idea. We just need to come up with a good API design for it.
One example of where custom read limits would be useful is a MIME parser. It makes sense to set the default StreamReader's limit to a few megabytes, and then use readline(limit=8*1024) for reading headers, and readuntil(separator=boundary) for reading attachments.
@gvanrossum also suggested to add a StreamReader.set_buffer_limit method. Which actually sounds like a better option. For MIME parser, for example, we could initialize StreamReader with its default limit, read headers, and then increase the limit to make it possible to use readuntil for parsing attachments.
This topic was originally brought up in PR #297.
StreamReaderclass allows to implement protocols parsers in a very convenient way already. Instead of managing buffers and callbacks, one can layout even complex protocols as a series ofawait read(..),await readline(..)(and soonawait readuntil(..)) calls.I think that adding an ability to adjust the buffer limit per operation is a great idea. We just need to come up with a good API design for it.
One example of where custom read limits would be useful is a MIME parser. It makes sense to set the default
StreamReader's limit to a few megabytes, and then usereadline(limit=8*1024)for reading headers, andreaduntil(separator=boundary)for reading attachments.@gvanrossum also suggested to add a
StreamReader.set_buffer_limitmethod. Which actually sounds like a better option. For MIME parser, for example, we could initializeStreamReaderwith its defaultlimit, read headers, and then increase the limit to make it possible to usereaduntilfor parsing attachments.