Tiny maybe-bug in obsolete line folding handling #59

I was looking at urllib3/urllib3#1318, and realized I wasn't quite sure what the right rules were for handling whitespace in headers using the "obsolete line-folding rule". Specifically, if we have a header like Name: value1${WHITESPACE1}\r\n${WHITESPACE2}value2 then what is the header's logical value?

Currently h11 treats it as: value1${WHITESPACE1} value2

That PR makes urllib3 treat it as: value1 value2

RFC 7230 says:

     header-field   = field-name ":" OWS field-value OWS

     field-name     = token
     field-value    = *( field-content / obs-fold )
     field-content  = field-vchar [ 1*( SP / HTAB ) field-vchar ]
     field-vchar    = VCHAR / obs-text

     obs-fold       = CRLF 1*( SP / HTAB )

(And then in the text it specifies that obs-fold tokens should be replaced by one or more SP.)

As written, this actually says that ${WHITESPACE1} being non-empty is an error, and only Name: value1\r\n${WHITESPACE2}value2 is legal (and it becomes Name: value1 value2). So h11 doesn't quite follow that.

The reason h11 doesn't quite follow it though is that the spec's ABNF is buggy – its field-content rule is just wrong: https://www.rfc-editor.org/errata/eid4189

The suggestion in that errata is:

     field-content  = field-vchar [ 1*( SP / HTAB / field-vchar ) field-vchar ]
     obs-fold       = OWS CRLF 1*( SP / HTAB )

This matches the urllib3 interpretation, but not the current h11 interpretation.

Of course, the comment on that errata is "this is not right", but OTOH "The [fix] is the best approach for now, and a better fix will be developed", someday, maybe.

So I guess ideally h11 should switch to strip trailing whitespace from header lines that are followed by obs-fold lines.

This is the opposite of a high-priority bug.