Skip to content

LiveScriptAdvanced/web-corpus

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

30 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

web-scraper.py

Usage

There are three main flags you need feed the script:

python3 web-scraper.py --seed --verbose --outfile

--seed is a requirement, it is the seed URL from which the program scrapes

--verbose will print out to the terminal which URLs were accepted or rejected

--outfile is the file to which the scraped text is saved

About

Tool for scraping text from websites

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 100.0%