[Bug] firecrawl.scrape not respecting location parameter #2307
Description
Describe the Bug
When running firecrawl.scrape() and providing the location parameter it will very often disregard that and crawl using US servers. In my example I have tried setting location to GB
To Reproduce
Steps to reproduce the issue:
pnpm install @mendable/firecrawl-js@4.4.1- Create a script that runs
firecrawl.scrapewithlocationset - Determine if the script actually ran with a proxy, for me it was easy since the host redirected to a subdomain for different locations.
Expected Behavior
I expect the crawler to always be redirected to the GB URL, not the US url.
Environment (please complete the following information):
- OS: macOS
- Deployment Type: Cloud
- Firecrawl Version: 4.4.1
- Node.js Version: 22.16.0
Logs
As you can see, for the first request url matches sourceURL. But in the second crawl request it does not.
@scopes/crawler:crawl:hawke: 🚀 Starting crawl of 3 URLs
@scopes/crawler:crawl:hawke: 🔍 Extracting scope data from: https://uk.hawkeoptics.com/frontier-30-1-6x24-l4a-dot.html
@scopes/crawler:crawl:hawke: scopeMetaData {
@scopes/crawler:crawl:hawke: url: 'https://uk.hawkeoptics.com/frontier-30-1-6x24-l4a-dot.html',
@scopes/crawler:crawl:hawke: scrapeId: '17917724-7b69-41e7-9398-b7d618f63e7d',
@scopes/crawler:crawl:hawke: sourceURL: 'https://uk.hawkeoptics.com/frontier-30-1-6x24-l4a-dot.html',
@scopes/crawler:crawl:hawke: statusCode: 200,
@scopes/crawler:crawl:hawke: contentType: 'text/html; charset=iso-8859-1',
@scopes/crawler:crawl:hawke: proxyUsed: 'basic',
@scopes/crawler:crawl:hawke: creditsUsed: 5
@scopes/crawler:crawl:hawke: }
@scopes/crawler:crawl:hawke: 🔍 Extracting scope data from: https://uk.hawkeoptics.com/frontier-30-1-6x24-tactical-dot.html
@scopes/crawler:crawl:hawke: scopeMetaData {
@scopes/crawler:crawl:hawke: url: 'https://us.hawkeoptics.com/frontier-30-1-6x24-tactical-dot.html',
@scopes/crawler:crawl:hawke: scrapeId: '4396d4e8-2e50-4ff9-8553-1d6b00ef6e8e',
@scopes/crawler:crawl:hawke: sourceURL: 'https://uk.hawkeoptics.com/frontier-30-1-6x24-tactical-dot.html',
@scopes/crawler:crawl:hawke: statusCode: 200,
@scopes/crawler:crawl:hawke: contentType: 'text/html; charset=iso-8859-1',
@scopes/crawler:crawl:hawke: proxyUsed: 'basic',
@scopes/crawler:crawl:hawke: creditsUsed: 5
@scopes/crawler:crawl:hawke: }