You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Develop an example using the HTTP client as a web crawler #108
This sounds like a good idea. However I'm afraid of the effort that would need to go into something like this. Parsing HTML is scary stuff and finding the URL's and interpreting rel="nofollow" in a DOM along with the myriad other link elements in a document is a tall order.
That said, I'd be willing to accept contributions to this effect.
My motivation for opening this issue is to start generating more complex and interesting examples for the HTTP client. At this stage I am not worried about the complexity of the HTML parsing.
Perhaps I'm rising a zombie, but nonetheless, I created this example https://github.com/tex/cpp-netlib-example
Would you consider it as a good example of using cpp-netlib as a simple web crawler?
Activity
deanberris commentedon Apr 25, 2012
This sounds like a good idea. However I'm afraid of the effort that would need to go into something like this. Parsing HTML is scary stuff and finding the URL's and interpreting rel="nofollow" in a DOM along with the myriad other link elements in a document is a tall order.
That said, I'd be willing to accept contributions to this effect.
glynos commentedon Apr 26, 2012
My motivation for opening this issue is to start generating more complex and interesting examples for the HTTP client. At this stage I am not worried about the complexity of the HTML parsing.
tex commentedon Jan 3, 2018
Perhaps I'm rising a zombie, but nonetheless, I created this example
https://github.com/tex/cpp-netlib-example
Would you consider it as a good example of using cpp-netlib as a simple web crawler?