You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
UTF-8 Parsing of HTTP Headers Fails in Async Server Connection (std::wstring support status) #311
I'm developing a HTTP server with cpp-netlib (0.10.1) and for the means to support Digest authentication i need to have std::wstring support (since usernames and passwords could contain non ASCII characters and those then would be present in the Digest headers. I saw that the existing server type tags all have default_string in them, but i managed to create a new one that has default_wstring in it, so far so good - but I'm getting compilation errors for the server_options type, since it uses the string type from the server tag, but has ASCII strings as initializers - well I thought i could avoid that one with just adding L" prefix to the default values since I'll be using only std::wstring version. But then the underlying boost ASIO part starts to fall since it's suing std::string to pass endpoint names etc., which of course makes sense.
So since i don't really understand the std::wstring support status, I'm not sure if this is an actual bug, but if it is, is trunk version already supporting std::wstring or is there any way I can tackle this? Does the options struct really need the string type from the Tag? Maybe i can fix that and live without it?
So at the moment, I'm down at the point where Boost ASIO fails with:
async_server.hpp(115) : error C2664: 'boost::asio::ip::basic_resolver_query::basic_resolver_query(const std::string &,boost::asio::ip::resolver_query_base::flags)' : cannot convert parameter 1 from 'std::basic_string<_Elem,_Traits,_Ax>' to 'const std::string &'
1> with
1> [
1> InternetProtocol=boost::asio::ip::tcp
1> ]
1> and
1> [
1> _Elem=wchar_t,
1> _Traits=std::char_traits<wchar_t>,
1> _Ax=std::allocator<wchar_t>
1> ]
1> Reason: cannot convert from 'std::basic_string<_Elem,_Traits,_Ax>' to 'const std::string'
1> with
1> [
1> _Elem=wchar_t,
1> _Traits=std::char_traits<wchar_t>,
1> _Ax=std::allocator<wchar_t>
1> ]
And I'm not sure what approach to use. Many thanks.
So the quick answer to this is, that you're probably better served by converting stuff that your handler deals with in std::wstring to Base64'd std::string and let the clients handle it appropriately. You can use the tip of 0.10-devel to get to the Base64 work that's already gotten in there -- not sure if it works with std::wstring though.
Just curiously, why is your Digest message returning std::wstring's? I presume based on your compiler errors message that you're using a Microsoft library -- in which case I might ask you to use a version of the digest message generator that exposes a const char* or equivalent, and Base64 encode that. Would that work better for you?
Thanks for the quick response, this leaves me with the feeling that cpp-netlib was the right choice among the others.
Maybe I didn't express myself clearly, I am not generating anything, I'm implementing server side standard HTTP Digest authentication (http://en.wikipedia.org/wiki/Digest_access_authentication - this, so that we are both talking about the same thing) - the thing is that when using the default async server i get an assert in the i/o level when I for example enter a password with unicode characters in Chrome and that is passed back to the server in the Digest authentication headers in the username attribute, I didn't actually check what happens after the assert since I felt I was doing it wrong (maybe when the request comes in the handler the data in std::string is not lost, but I doubt it). So I tried switching to a server with default_wstring tag, but I failed to compile it - seems that the default_wstring tag is really broken in 0.10, at least there is no way out. I made a quick look into the trunk, but there were other issues:
On Windows parts of the C++11 are not implemented so some default functions and constexprs do not compile on VS2013, but well that I managed to solve quickly wrapping your copy constructors with the same macro boost uses to disable default C++11 copy ctors (don't remember the name of the macro by heart, sorry) and by not building tests.
Then, with trunk I stumbled upon something that I saw for the first time and was not able to explain with my C++ knowledge - http://stackoverflow.com/questions/19053922/c-forward-declaring-a-type-def, i thought maybe I am not getting something right so I asked for advice , but seems like it is just work in progress
So, I guess you've understood the root cause of my issues and that Base64 is not an option since the utf8 stuff comes in from browser at i/o level - so what are my options then? Try to fix 0.10.1 to work with std::wstring , use some other branch?
Typically what browsers do when encoding unicode is by percent-encoding things, so that it's 7-bit ASCII clean on the wire (as this is what HTTP/1.0 and HTTP/1.1 expect). Even with unicode characters, they can get percent-encoded appropriately before being sent through as a header. You don't need std::wstring support anywhere in the stack for this to happen properly. The other option that browsers can use is to use ASCII-encoded UTF-8, which works just fine in std::string.
It may be easier if you can provide sample code which can communicate better what issues you're running into.
I'll also mail you a vs2013rc project if it's any use. So what happens is, if you do a 127.0.0.1/ from Chrome in the first request the first if branch goes through and then the browser prompts for password, use some unicode value (e.g. I use Latvian words Jānis / Bānis), what happens then is boost asserts in spirit (actually in work where I did this with boost 1.49.0 i think it was a different place but i might be mistaken):
in /boost/spirit/home/support/char_encoding/standard.hpp
and if i pass the asserts the the request handler does not receive the authentication header at all, so still looks like i'm in a bad situation. Also, on the side, if default options are used (the address() and port() calls are not made) then an exception is thrown when cpp-netlib tries to to make an ip v4 address at
First-chance exception at 0x76674B32 in DigestTest.exe: Microsoft C++ exception: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<std::bad_cast > at memory location 0x0109E82C.
DigestTest.exe!boost::throw_exceptionstd::bad_cast(const std::bad_cast & e) Line 70 C++
DigestTest.exe!boost::asio::ip::address::to_v4() Line 109
but that is just fyi. Hope my code and explanations help you to give me guidance.
Boost 1.49.0 is really old. If you can use a later version of Boost you would have better success with cpp-netlib 0.10.1.
Output all the headers you do receive -- I think your std::find_if(...) is not finding the headers correctly. If you see that the Authorization header is not really there, but you're positive that Chrome is sending the Authorization header (either by inspecting the TCP traffic, say with a tool like tcpdump/wireshark), then this means it's a bug in the header parsing of cpp-netlib.
As for the default options, it's better to set them as the defaults don't mean anything (actually they're not valid).
i know, simply our project uses vs2008 beacause of home made ATL stuff that changes in vs2010, so I'm a bit stuck in 1.49 and bot sure if vs2008 will be able to build later versions, but that is another story
Welll the find_if seems fine, when i use a non utf8 password i get it in the header vector and find_if finds it, if i use a utf8 password then as i mentioned i get an assert in boost/spirit/home/support/char_encoding/standard.hpp, function isalnum(int ch), so seems like boost is not expecting non ASCII characters at this point.
I sae thath trunk already has all the values fot the options templre as std::string, does 0.10 have a branch where this is done too?
Just for an update - I decided to check wether setting locale before starting the server could help in any with the isalnum and other functions but Windows does not supprot utf-8 locale :(
I looked into 0.10 and 0.11 development branches and it seems that there the options class still uses the string type from the server, what about 1.0 - is that work looking into or is that in very early stages?
I think this may be a bug in the HTTP server's parser implementation. What I would think proper browsers would do here is use ASCII encoding of UTF-8, and that somehow the parser is not recognizing the headers. Can you print out the contents of the HTTP request being sent by Chrome (using the developer console)? That will better help with the debugging.
I understand that the crash happens in Boost.Spirit, but the HTTP server's parser implementation uses these. If you can give me a stack trace on where the error occurs, I can better look at where it might actually fail. If you're going to use a version other than 0.10.1, use the tip of 0.10-devel.
The options class will always use string types for the server. I don't see a need to change this to support std::wstring. Again, there's absolutely no need to deal with std::wstring in HTTP even on Windows.
Yeah, i mean eventually i came to the same conclusions, that is why i did not get' why the options struct uses the string_type from tag instead of explicitly using std::string. Nevertheless after of couple of hours going through your and Boost's code I've come down to this:
Basically i really don't need to have the string_type set as std::wstring, since yeah, this is only related to one specific request header
I defined my own server tags using your examples (basically the same set as for async_server but with a different name so i don't get all the default request structures), used them for server_base template, did my own specialization with these tags for not_quite_pod_request_base and there explicitly set header_type to request_header_wide. After these manipulations all that did not work from cpp-netlib code was parse_headers since it did not have a prototype with std::wstring container, well i provided one using Boost.Locale functions to translate the std::string to std::wstring (assuming browsers would use utf-8 which is also implied by the digest authentication RFC) and then got too tired and wend home, will test this tomorrow but I think that would be the best solution for this, since it basically reuses all the stock functionality and should be able to retrieve my utf-8 encoded header. I mean, the digest header situation could actually be the only situation where you get utf-8 stuff in the http headers, so it's realy rare (even though not sure if http rfc states what you can and what you can't do in your own custom headers). Any pointers if my approach is the most efficient one in this situation?
So 1.0 will have everything in std::string which will be fine. But 1.0 isn't here yet.
Again, you don't need std::wstring to get the UTF-8 characters -- UTF-8 uses just 7 bits anyway and is backwards-compatible with ASCII (http://en.wikipedia.org/wiki/UTF-8) so using std::wstring will not get you anything. You'd be wasting your time going through the std::wstring route. Don't read too much into the tag dispatch mechanism, that wasn't meant for std::wstring switching only but for supporting other string types that aren't std::string.
The bug is in the HTTP parser implementation, and unless you can provide me with an HTTP request that causes the server to fail parsing, I can't help pinpoint where in the HTTP server parser the code is failing.
If you have a stack trace on where the parser is failing -- which functions, calling what, where, with line numbers in the cpp-netlib implementation -- based on bad input, you can fix it yourself by doing the following:
See whether you can make minimal changes to the request parser so that the failure goes away.
I can help you do this, but I will need the actual input, or an example input that will cause problems. It might even be that it's because the HTTP request contains a multi-line header and that causes the issue. All this would be really easy if you can get me an example dump of an HTTP request causing your server to fail with that assertion.
Yeah, i know i don't need std::wstring explicitly, simply the reasoning behind is that Boost.Spirit parser which uses all the std::isalphanum etc., will not fail on wchar_t but that's an asumption right now.
As for the test case, after i get some results tommorow, i can try to make a similar test case like in the cpp-netlib solution, but the example code above (my 3 post, i also mailed you a solution) demonstretes the test case, just run the server, connect to it with Chrome, enter a username and password with utf8 symbols in the browser window and you will get the assert. I will rerun the example and get the stack trace in a sec.
msvcr120d.dll!_chvalidator(int c, int mask) Line 56 C++
msvcr120d.dll!isalnum(int c) Line 262 C++
DigestTest.exe!boost::spirit::char_encoding::standard::isalnum(int ch) Line 44 C++
DigestTest.exe!boost::spirit::char_class::classify<boost::spirit::char_encoding::standard>::is<char>(boost::spirit::tag::alnum __formal, char ch) Line 301 C++
I couldn't post the whole trace because the Boost.Spirit stack is just too long, but basically this shows what i said, not your parser bet the Boost.Spirit one is failing, is there any way to force Boost.Spirit to use utf-8?
As you already mentioned, I am aware of the differences between uft-16 and utf-8 and i know wchar_t is meant for utf-16 not storing utf-8, I understand that my initial willingness to use std::wstring all through lead to the opposite but since I was not anyhow familiar with how things work in cpp-netlib, that was a clue looking at the code out of the box. Simply, with my knowledge of Boost.Spirit i don't know how to overcome this easier than doing a proper utf-8 to utf-16 translation and then parsing that with a Boost.Spirit parser that i meant for utf-16. If there is a way to tell the parser to just expect utf-8 sequences in the std::string from parse_headers argument i'd be much happier to use that without defining the server tags etc. i'm doing now:)
DigestTest.exe!boost::network::http::parse_headers(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & input, std::vector<boost::network::http::request_header_narrow,std::allocator<boost::network::http::request_header_narrow> > & container) Line 50 C++
That's the one that I think needs fixing. In essence, this implementation (in 0.10-devel) should account for the fact that there may be UTF-8 characters in the header:
Np, i'll try to provide a test case tomorrow. I've been looking around the Boost.Spirit examples and - with the given grammar definition, is there any way to tell it to use the unicode namespace from Boost.Spirit?
Ok, this is what i came down to, and again, I'm just asking for comments since your Boost.Spirit knowledge seems to be better than mine, but at least this approach works:
without the u32 iterator the standart validators still assert so that is a must as far as i see and it makes sense, but I was not sure about the wrapper struct, could there be a better way? Using the containers variable directly does not work since when assigning from the 32bit integer type to char it just looses value, but is there a way to tell fusion to call some explicit operation when doing the cast? if so that would remove the foreach loop and would end up quite clean.
Well not really, that namespace only has the char definitions, not the parse function. And i tried using just the unicode alnum terminal with the std::string passed to parse, but i understand that the parser actually uses the string character type as a template argument for char_parser so i again ended up with an ASCII parser, so i think that the iterators are mandatory, searching in stackoverflow.com also gave the same answer. My doubt is about how to make the output more efficient, to avoid the for-each thing.
@rubu -- do you have a pull request that you can send in on top of 0.10-devel so I can include it into 0.11.0 by the end of the week? A test would be nice to have too.
Hi, no i haven't made on, will do this today or tomorrow. Btw, since i've
been extensively using cppnetlib for the past month, another thing came up -
what about the query string - your parser in the query_map() function
uses the following rule for the key:
key = spirit::qi::char_("a-zA-Z_") >>
*spirit::qi::char_("a-zA-Z_0-9/%");
value = +spirit::qi::char_("a-zA-Z_0-9/%");
while http://en.wikipedia.org/wiki/Query_string states that the characters
'.' and '-' should also be supported. I'm simply sending a url encoded link
in a query string and right now it is not being parsed by the query map
function so i am using other means.
@rubuhttps://github.com/rubu -- do you have a pull request that you
can send in on top of 0.10-devel so I can include it into 0.11.0 by the end
of the week? A test would be nice to have too.
—
Reply to this email directly or view it on GitHubhttps://github.com//issues/311#issuecomment-29152827
.
Any update on this @rubu -- I've just tagged 0.11.0-rc0 and will be releasing 0.11.0 early next week. Do you know when you're able to update this PR with tests?
I began looking into that and i wanted to ask - do you have any test case
that launches a server and sends a request and verifies it ? since in the
existing tests you usually do a request against boost.org or some well
known domain. Or should I build one from the ground?
Any update on this @rubuhttps://github.com/rubu -- I've just tagged
0.11.0-rc0 and will be releasing 0.11.0 early next week. Do you know when
you're able to update this PR with tests?
—
Reply to this email directly or view it on GitHubhttps://github.com//issues/311#issuecomment-29490839
.
If you add a test case there (or a few of them) and make sure that the parser is still doing what it's supposed to do (for example, not fail with UTF-8 headers) then that should be sufficient. Another thing you can do is if you can isolate the function (the parse_headers function) with just sample HTTP requests as strings and add that test file to the project, that would work too.
Activity
deanberris commentedon Oct 4, 2013
So the quick answer to this is, that you're probably better served by converting stuff that your handler deals with in std::wstring to Base64'd std::string and let the clients handle it appropriately. You can use the tip of 0.10-devel to get to the Base64 work that's already gotten in there -- not sure if it works with std::wstring though.
Just curiously, why is your Digest message returning std::wstring's? I presume based on your compiler errors message that you're using a Microsoft library -- in which case I might ask you to use a version of the digest message generator that exposes a
const char*or equivalent, and Base64 encode that. Would that work better for you?rubu commentedon Oct 4, 2013
Thanks for the quick response, this leaves me with the feeling that cpp-netlib was the right choice among the others.
Maybe I didn't express myself clearly, I am not generating anything, I'm implementing server side standard HTTP Digest authentication (http://en.wikipedia.org/wiki/Digest_access_authentication - this, so that we are both talking about the same thing) - the thing is that when using the default async server i get an assert in the i/o level when I for example enter a password with unicode characters in Chrome and that is passed back to the server in the Digest authentication headers in the username attribute, I didn't actually check what happens after the assert since I felt I was doing it wrong (maybe when the request comes in the handler the data in std::string is not lost, but I doubt it). So I tried switching to a server with default_wstring tag, but I failed to compile it - seems that the default_wstring tag is really broken in 0.10, at least there is no way out. I made a quick look into the trunk, but there were other issues:
So, I guess you've understood the root cause of my issues and that Base64 is not an option since the utf8 stuff comes in from browser at i/o level - so what are my options then? Try to fix 0.10.1 to work with std::wstring , use some other branch?
deanberris commentedon Oct 4, 2013
Can you provide where the assertion is failing?
Typically what browsers do when encoding unicode is by percent-encoding things, so that it's 7-bit ASCII clean on the wire (as this is what HTTP/1.0 and HTTP/1.1 expect). Even with unicode characters, they can get percent-encoded appropriately before being sent through as a header. You don't need std::wstring support anywhere in the stack for this to happen properly. The other option that browsers can use is to use ASCII-encoded UTF-8, which works just fine in std::string.
It may be easier if you can provide sample code which can communicate better what issues you're running into.
rubu commentedon Oct 4, 2013
Ok, thanks, I have that code at work, but I'll try and build 0.10.1 at home to provide an example.
rubu commentedon Oct 4, 2013
Ok, here is an example:
I'll also mail you a vs2013rc project if it's any use. So what happens is, if you do a 127.0.0.1/ from Chrome in the first request the first if branch goes through and then the browser prompts for password, use some unicode value (e.g. I use Latvian words Jānis / Bānis), what happens then is boost asserts in spirit (actually in work where I did this with boost 1.49.0 i think it was a different place but i might be mistaken):
in /boost/spirit/home/support/char_encoding/standard.hpp
and if i pass the asserts the the request handler does not receive the authentication header at all, so still looks like i'm in a bad situation. Also, on the side, if default options are used (the address() and port() calls are not made) then an exception is thrown when cpp-netlib tries to to make an ip v4 address at
First-chance exception at 0x76674B32 in DigestTest.exe: Microsoft C++ exception: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<std::bad_cast > at memory location 0x0109E82C.
but that is just fyi. Hope my code and explanations help you to give me guidance.
deanberris commentedon Oct 4, 2013
Few things:
With this example what is the assertion failure?
rubu commentedon Oct 5, 2013
Hi,
i know, simply our project uses vs2008 beacause of home made ATL stuff that changes in vs2010, so I'm a bit stuck in 1.49 and bot sure if vs2008 will be able to build later versions, but that is another story
Welll the find_if seems fine, when i use a non utf8 password i get it in the header vector and find_if finds it, if i use a utf8 password then as i mentioned i get an assert in boost/spirit/home/support/char_encoding/standard.hpp, function isalnum(int ch), so seems like boost is not expecting non ASCII characters at this point.
I sae thath trunk already has all the values fot the options templre as std::string, does 0.10 have a branch where this is done too?
Best Regards,
Rudolfs Bundulis
rubu commentedon Oct 7, 2013
Just for an update - I decided to check wether setting locale before starting the server could help in any with the isalnum and other functions but Windows does not supprot utf-8 locale :(
I looked into 0.10 and 0.11 development branches and it seems that there the options class still uses the string type from the server, what about 1.0 - is that work looking into or is that in very early stages?
deanberris commentedon Oct 7, 2013
I think this may be a bug in the HTTP server's parser implementation. What I would think proper browsers would do here is use ASCII encoding of UTF-8, and that somehow the parser is not recognizing the headers. Can you print out the contents of the HTTP request being sent by Chrome (using the developer console)? That will better help with the debugging.
I understand that the crash happens in Boost.Spirit, but the HTTP server's parser implementation uses these. If you can give me a stack trace on where the error occurs, I can better look at where it might actually fail. If you're going to use a version other than 0.10.1, use the tip of 0.10-devel.
The options class will always use string types for the server. I don't see a need to change this to support std::wstring. Again, there's absolutely no need to deal with std::wstring in HTTP even on Windows.
rubu commentedon Oct 7, 2013
Yeah, i mean eventually i came to the same conclusions, that is why i did not get' why the options struct uses the string_type from tag instead of explicitly using std::string. Nevertheless after of couple of hours going through your and Boost's code I've come down to this:
deanberris commentedon Oct 7, 2013
So 1.0 will have everything in std::string which will be fine. But 1.0 isn't here yet.
Again, you don't need std::wstring to get the UTF-8 characters -- UTF-8 uses just 7 bits anyway and is backwards-compatible with ASCII (http://en.wikipedia.org/wiki/UTF-8) so using std::wstring will not get you anything. You'd be wasting your time going through the std::wstring route. Don't read too much into the tag dispatch mechanism, that wasn't meant for std::wstring switching only but for supporting other string types that aren't std::string.
The bug is in the HTTP parser implementation, and unless you can provide me with an HTTP request that causes the server to fail parsing, I can't help pinpoint where in the HTTP server parser the code is failing.
If you have a stack trace on where the parser is failing -- which functions, calling what, where, with line numbers in the cpp-netlib implementation -- based on bad input, you can fix it yourself by doing the following:
I can help you do this, but I will need the actual input, or an example input that will cause problems. It might even be that it's because the HTTP request contains a multi-line header and that causes the issue. All this would be really easy if you can get me an example dump of an HTTP request causing your server to fail with that assertion.
rubu commentedon Oct 7, 2013
Yeah, i know i don't need std::wstring explicitly, simply the reasoning behind is that Boost.Spirit parser which uses all the std::isalphanum etc., will not fail on wchar_t but that's an asumption right now.
As for the test case, after i get some results tommorow, i can try to make a similar test case like in the cpp-netlib solution, but the example code above (my 3 post, i also mailed you a solution) demonstretes the test case, just run the server, connect to it with Chrome, enter a username and password with utf8 symbols in the browser window and you will get the assert. I will rerun the example and get the stack trace in a sec.
rubu commentedon Oct 7, 2013
... tons of spirit stuff...
I couldn't post the whole trace because the Boost.Spirit stack is just too long, but basically this shows what i said, not your parser bet the Boost.Spirit one is failing, is there any way to force Boost.Spirit to use utf-8?
As you already mentioned, I am aware of the differences between uft-16 and utf-8 and i know wchar_t is meant for utf-16 not storing utf-8, I understand that my initial willingness to use std::wstring all through lead to the opposite but since I was not anyhow familiar with how things work in cpp-netlib, that was a clue looking at the code out of the box. Simply, with my knowledge of Boost.Spirit i don't know how to overcome this easier than doing a proper utf-8 to utf-16 translation and then parsing that with a Boost.Spirit parser that i meant for utf-16. If there is a way to tell the parser to just expect utf-8 sequences in the std::string from parse_headers argument i'd be much happier to use that without defining the server tags etc. i'm doing now:)
deanberris commentedon Oct 7, 2013
That's the one that I think needs fixing. In essence, this implementation (in 0.10-devel) should account for the fact that there may be UTF-8 characters in the header:
https://github.com/cpp-netlib/cpp-netlib/blob/0.10-devel/boost/network/protocol/http/server/impl/parsers.ipp#L38
If you can provide the actual HTTP request that we can use for testing/reproduction, we can fix this in 0.10-devel appropriately.
Thanks @rubu !
rubu commentedon Oct 7, 2013
Np, i'll try to provide a test case tomorrow. I've been looking around the Boost.Spirit examples and - with the given grammar definition, is there any way to tell it to use the unicode namespace from Boost.Spirit?
deanberris commentedon Oct 7, 2013
You may be able to do something with Boost 1.54.0 and doing something like this at the start of the function:
http://www.boost.org/doc/libs/1_54_0/libs/spirit/doc/html/spirit/qi/reference/basics.html#spirit.qi.reference.basics.character_encoding_namespace
I'm not sure whether this will work with an earlier version of Boost.
rubu commentedon Oct 7, 2013
Ok, thanks, I'll try that.
rubu commentedon Oct 8, 2013
Ok, this is what i came down to, and again, I'm just asking for comments since your Boost.Spirit knowledge seems to be better than mine, but at least this approach works:
and i defined the request_header_narrow_utf8_wrapper like
and added it to fusion with
without the u32 iterator the standart validators still assert so that is a must as far as i see and it makes sense, but I was not sure about the wrapper struct, could there be a better way? Using the containers variable directly does not work since when assigning from the 32bit integer type to char it just looses value, but is there a way to tell fusion to call some explicit operation when doing the cast? if so that would remove the foreach loop and would end up quite clean.
deanberris commentedon Oct 8, 2013
I have a simpler solution. Instead of:
Just say:
Then you'd be fine without having to do any other magic?
rubu commentedon Oct 8, 2013
Well not really, that namespace only has the char definitions, not the parse function. And i tried using just the unicode alnum terminal with the std::string passed to parse, but i understand that the parser actually uses the string character type as a template argument for char_parser so i again ended up with an ASCII parser, so i think that the iterators are mandatory, searching in stackoverflow.com also gave the same answer. My doubt is about how to make the output more efficient, to avoid the for-each thing.
deanberris commentedon Nov 24, 2013
@rubu -- do you have a pull request that you can send in on top of 0.10-devel so I can include it into 0.11.0 by the end of the week? A test would be nice to have too.
rubu commentedon Nov 25, 2013
Hi, no i haven't made on, will do this today or tomorrow. Btw, since i've
been extensively using cppnetlib for the past month, another thing came up -
what about the query string - your parser in the query_map() function
uses the following rule for the key:
*spirit::qi::char_("a-zA-Z_0-9/%");
value = +spirit::qi::char_("a-zA-Z_0-9/%");
while http://en.wikipedia.org/wiki/Query_string states that the characters
'.' and '-' should also be supported. I'm simply sending a url encoded link
in a query string and right now it is not being parsed by the query map
function so i am using other means.
2013/11/24 Dean Michael Berris notifications@github.com
glynos commentedon Nov 25, 2013
Can you create an issue for this in the uri projec, in order that this fix is in 1.0?
https://github.com/cpp-netlib/uri/issues?state=open
rubu commentedon Nov 25, 2013
Ok, when i start making the pull request i'll do this as well, today looks
quite busy but i'll get it done today evening or tomorrow
2013/11/25 Glyn Matthews notifications@github.com
deanberris commentedon Nov 26, 2013
Thanks @rubu -- looking forward to the PR for 0.10-devel.
rubu commentedon Nov 26, 2013
Hi,
just did a PR for the fix, will work on the test during today.
Rudolfs
2013/11/26 Dean Michael Berris notifications@github.com
deanberris commentedon Nov 29, 2013
Any update on this @rubu -- I've just tagged 0.11.0-rc0 and will be releasing 0.11.0 early next week. Do you know when you're able to update this PR with tests?
rubu commentedon Nov 29, 2013
I began looking into that and i wanted to ask - do you have any test case
that launches a server and sends a request and verifies it ? since in the
existing tests you usually do a request against boost.org or some well
known domain. Or should I build one from the ground?
Best regards,
Rudolfs Bundulis
2013/11/29 Dean Michael Berris notifications@github.com
deanberris commentedon Nov 29, 2013
@rubu There's a test already just for the parsers last I checked:
https://github.com/cpp-netlib/cpp-netlib/blob/0.11-devel/libs/network/test/http/request_incremental_parser_test.cpp
If you add a test case there (or a few of them) and make sure that the parser is still doing what it's supposed to do (for example, not fail with UTF-8 headers) then that should be sufficient. Another thing you can do is if you can isolate the function (the parse_headers function) with just sample HTTP requests as strings and add that test file to the project, that would work too.