I want to download a .CSV file from this page https://data.anbima.com.br/certificado-de-recebiveis?view=precos, using requests. Get(). When I use the Inspect, there is no link directly to the file.
The page probably uses an API call to Download. I've studied the request using the Network panel on Chrome, but I got stuck in how to pass the right parameters in Python.
I'm trying to use the requests.get( link, params= )
Try using the Pandas library, it automatically downloads csv's from urls now Pandas Read CSV Documentation.
import pandas as pd
csv_url = 'csv_url'
df=pd.read_csv(csv_url)
Alternatively you can us lxml and requests libraries to scrape for the csv link. Grab the xpath of the link element from your webpage and save the link url string as a var.
Note lxml returns as a list.
from lxml import html
from lxml import etree
import requests
url = 'website'
#Open webpage
session_requests = requests.session()
result = session_requests.get(url)
tree = html.fromstring(result.text)
#xpath to csv URL (inspect element and right click to get xpath)
url_list = tree.xpath('xpath') #to get the link add /@href to the end of your xpath
#xpath returns link as a list, this just pulls link text out of the list
csv_url = url_list[0]
