3,839 questions
Tooling
0
votes
14
replies
225
views
simple/fast way to transfer files from Linux to Windows
I have two computers. One with Arch Linux installation with any modern software. Another with 32bit Windows 7.
My goal is to transfer around 100GB of files from Linux to Windows. I have USB 3.0 to ...
1
vote
1
answer
78
views
How to use wget with --accept-regex to mirror GitHub issues and PRs
I'm trying to use Wget to download an HTML mirror of a GitHub repo (for example, this repo: https://github.com/seanh/oatcake).
In case it matters, I'm on macOS and using wget from Homebrew.
I don't ...
0
votes
0
answers
48
views
Eliminating unnecessary files from a website
Someone gave me a copy of a website serving all of my late father's photography in a rather fancy Javascript-enhanced presentation. He created it with some scanning tool and never put it online, and ...
0
votes
0
answers
75
views
Get Firefox DevTools data from Linux command line
Is it possible to get data from Firefox DevTools command line in Linux?
I would like to get the cookies of the POST command (seen on the attached image). I have tried traditional ways using curl and ...
-1
votes
1
answer
41
views
Shell parameter not being received [duplicate]
I am trying to create a shell script that updates my public ip to the DynU service.
this command works fine in the terminal
wget -O - v4.ident.me 2>/dev/null && echo
The issue comes when I ...
-1
votes
1
answer
73
views
Download a file procedure in python better than Selenium
I'm trying to download a file result from this site:
https://www.coordinador.cl/operacion/documentos/registro-de-instrucciones-de-operacion-rio-sscc-energia/
And for that i use Selenium for select ...
0
votes
2
answers
182
views
Remove wget argument from Yocto do_fetch
I'm using Yocto Mickledore, and my build fails on the do_fetch of multiple python libraries.
The log files show that the use of --passive-ftp as a wget argument is to blame.
DEBUG: Fetcher accessed ...
0
votes
0
answers
31
views
How to correct my wGet scripts download images from the Lunar Orbiter and Lunar Sample open directories
I've tried countless combinations of options in wGet scripts with no success at getting what I actually want. I tried countless options taken from answered questions here as well.
I need help using ...
0
votes
1
answer
80
views
wget mirror returning 404 error despite website being accessible in browser
I'm trying to mirror a website using wget but getting a 404 error, even though the site is accessible through a browser.
Command used:
wget --mirror --convert-links --adjust-extension --page-...
0
votes
0
answers
254
views
SSL Communication failing in tomcat 10 in FIPS mode
tomcat version 10.1.19
Connector Config
port = 8443
protocol="org.apache.coyote.http11.Http11NioProtocol"
defaultSSLHostConfigName="defaultConfig"
Symptom: curl waits for about 20 ...
-5
votes
1
answer
99
views
How to prevent specific static files (e.g., hashed JS files) from being exposed in HTTP requests using Wget with ASP.NET Core?
i'm using ASP.NET Core as the backend for my Angular application. When I make requests to the server (e.g., using Wget), I notice that hashed JavaScript files like external-module.d1aa0dsdp55.js are ...
0
votes
0
answers
59
views
FTP server - caged
I try to access the FTP server to get the CAGED files. But as an error, it shows that the files are corrupted. When I try to access the message “Corrupt input data”. So I'm unsure whether, in fact, ...
0
votes
1
answer
50
views
Wget downloads folder structure but no files
I'm trying to download all .nc files from this website:
https://data.seaice.uni-bremen.de/MultiYearIce/ascat-amsr2/final/Antarctic/netcdf/2021/
Using:
wget -m https://data.seaice.uni-bremen.de/...
0
votes
2
answers
121
views
How to recursively download path or filenames, without downloading the actual files?
I want to download the filenames or paths of over 1 mil files on a NASA data repository, without actually downloading the files. What is the best way to do this? Does wget supports this functionality?
...
-2
votes
1
answer
179
views
Is it possible to install cgywin and wget for windows xp? [closed]
I need to run windows xp system and use cygwin and wget tools. Is it possible? I googled and some links are not working anymore.
Updated: I have made it work to instrall cygwin on xp systems by ...
0
votes
0
answers
76
views
Accessing folder through os.scandir() after changes made to folder
I am trying to iterate through a folder with html files to filter them according to whether they contain a keyword or not in their string form. I download them to a folder through wget and ...
0
votes
0
answers
286
views
"/bin/bash: wget: command not found" error in Jupyter notebook environment
I mainly use Google Colab, but Google Colab had a payment issue, so yesterday I tried to run the 100% same code in Jupyter Notebook.
My code is :
!pip install --upgrade mxnet
!pip install --upgrade ...
0
votes
0
answers
43
views
Get data from Datalogger via wget/curl
I need to download the data from a datalogger, day by day, which only allows ftp/wget/curl. I need to create a script which download the files from the folder (or the folder itself) day by day every ...
0
votes
1
answer
241
views
How to download html with curl or wget from a page using a JavaScript login sequence?
I am trying to scrape the "status page" of my PV system as follows:
In a regular browser, I simply open this page: https://enlighten.enphaseenergy.com/web/5000000/today/graph/hours
This ...
-1
votes
1
answer
384
views
Creating directories with wget
I need to download files from several pages, using wget -r -l 1 -nd -H --accept-regex 'https://blogspot.com/s[0-9]{4}/[0-9]{3}.pdf' -i list.txt; in the TXT file I have a list of all the pages from ...
0
votes
1
answer
100
views
Download file from TeamCity with SSO using bash
I need to download an artefact from TC using a bash command (it should work in Ubuntu 22.04.3 deb based server). TeamCity authentication mode is SSO only via MS account and 2FA authentication. Is it ...
-1
votes
1
answer
58
views
Wget give different result with python request
I have been trying to extract transaction records from this website: https://www.house730.com/en-us/deal/?type=rent.
Looking into stack overflow, I have stumbled into a solution that uses urllib....
2
votes
1
answer
222
views
Pipe wget into sh - issue with line breaks
I'm trying to download a bash script using wget and pipe it over to sh but I'm running into problems. It works fine in separate steps (wget to download, then execute), but not when piping to sh. The ...
1
vote
2
answers
98
views
Python Selenium Script Using Drop-down menus
I am trying a fun project related to future Canadian Health and safety Industry career aspirations that will also help me build Python skills. There is a website that provides data on Canadian ...
0
votes
1
answer
90
views
how to fix syntax error on using wget on Colab
wget 'https://dsmn.ml/files/d2-net/d2_tf.pth' -O models/d2_tf.pth
File "<ipython-input-23-6839219d5fcc>", line 1
wget "https://dsmn.ml/files/d2-net/d2_tf.pth" -O ...
-1
votes
2
answers
1k
views
Trying to pip instal wget with cmd [closed]
I am trying to install wget with pip through the following commands:
C:\Users\jacob>pip uninstall wget
Found existing installation: wget 3.2
Uninstalling wget-3.2:
Would remove:
c:\users\...
0
votes
2
answers
163
views
wget python .tmp error doesn't work on specific web URL(web crawling )
Hello I have a weird problem in Python using wget, will be so grateful if someone could give me a help.
what I want to do :
download the file('.pdf','.djvu') from specific website(ex. wiki) with wget, ...
0
votes
2
answers
400
views
download all pdf files from website doesn't support wildcard
I want to download all pdf files in the website of "https://journals.ametsoc.org/view/journals/mwre/131/5/mwre.131.issue-5.xml".
I tried many thing with wget as:
wget --wait 10 --random-wait ...
-2
votes
1
answer
61
views
Can't download from GitHub
Trying to get the following file from GitHub and the file does not download'''!wget -P {HOME}/weights -q https://github.com/WongKinYiu/yolov9/releases/tag/v0.1.1/yolov9-c.pt'''
I know they are there.
'...
0
votes
2
answers
728
views
docker build config proxy wget: can't connect to remote host
I'm not really good at network,
I can't see what changed with the command wget inside when I ran docker build --build-arg https_proxy="http://proxy-fg:8080/" and wget outside from the vm? ...
0
votes
0
answers
1k
views
requests.get(): certificate verify failed: unable to get local issuer certificate, despite public certificate being used to verify [duplicate]
I am attempting to access some data from a website through python, but after my first attempt:
requests.get('https://website.com')
I was met with the following error:
HTTPSConnectionPool(host='www....
1
vote
1
answer
313
views
Using WGET to Get Links on the Page
I'm using:
wget --spider --force-html -r -l5 http://example.com 2>&1 | grep '^--' | awk '{print $3}' > urls.txt
It works great; however, it doesn't seem to copy the 'href=' links on each ...
1
vote
1
answer
167
views
How to correctly read an endless stream from InternetRadio using Indy TIdHTTP?
I am writing a UPNP-Controller.
To supply the Artist and Title, it's necessary that I can get the metadata, probably supplied as XML somewhere in the Stream.
For that, I want to read the (endless) ...
1
vote
0
answers
265
views
`curl` applying lowercase to headers
There is a website that I need to scrape just a single page and I discovered it blocks access from curl but allows from wget. Making exactly the same request (same headers, cookies, same origin IP...) ...
0
votes
0
answers
62
views
How to stop a wget mirror?
I accidentally mirrored a whole website using wget. Whenever I try to delete the downloaded file, it keeps reappearing & trying to download again. How do I stop this?
I tried looking through the ...
0
votes
1
answer
1k
views
Unable to install wget on mac (mojave 10.14.6) result: Error: An exception occurred within a child process
I tried installing wget running this command line
brew install wget
The result:
Error: An exception occurred within a child process:
CompilerSelectionError: libunistring cannot be built with any ...
0
votes
0
answers
193
views
Trying to import wget libraries but it's not happening why? It says requirement satisfied but not able to import it. Giving module not found error
get_ipython().system('pip install wget')
import wget
Requirement already satisfied: wget in
c:\users\divya\appdata\local\programs\python\python312\lib\site-packages (3.2)
------------...
2
votes
1
answer
92
views
list pages in a website of links to reports
I'm trying to list of all the pages of a certain level on a website. The website is kind of mostly text links of reported events, which each link to a more detailed report.
So the main link to the ...
-1
votes
1
answer
131
views
Error 403: Server Failed to Authenticate the request
I am trying to download a file from from Azure Artifact package using their Rest API.
wget --header="Authorization: Basic $PAT" "$URL" -O $filename
I am getting error message
...
0
votes
1
answer
1k
views
curl or wget command to verify a host is serving HTTPS on port 443?
I'm not married to either tool, I'm just hoping to find the fastest way to verify that all hosts in a list I pipe to either curl or wget are listening for HTTPS traffic on 443 rather than some other ...
2
votes
1
answer
93
views
wget converts fingerprints from ? into %3F
I want to mirror my cms-driven website to create a html-only static version. Local viewing is enabled with the -k option (--convert-links).
However, wget converts question marks ? to percent-encoded ...
-1
votes
2
answers
46
views
wget cannot change filenames when downloading
I'm having a problem downloading a xampp file using wget and choosing to change the filename, but it doesn't work, can anyone help me?
this is command:
enter image description here
I tried a few ...
0
votes
1
answer
497
views
Get the latest download link using Curl
I am trying to use curl to download the latest linux link from here: https://redream.io/download
The most recent download link is: https://redream.io/download/redream.x86_64-linux-v1.5.0-1106-g01ef607....
0
votes
0
answers
730
views
Proxmox Wget not working stuck on HTTP request sent, awaiting response... / after connection success
if i try to wget a file on proxmox like downloading a template or inside the host shell, i keep getting HTTP request sent, awaiting response... , for example trying to download the template text for ...
4
votes
0
answers
393
views
Cant call local AWS lambda container with curl/httr2
I have an AWS Lambda container that I want to be able to ping from the terminal or tools such as R's httr2. I tested it with Postman and it works and gives the appropriate response/output, I can see ...
2
votes
2
answers
213
views
Enabling cookies in libwget causes Segmentation Fault
I am trying to learn using libwget on Debian 12. I've installed the package "wget2-dev" which recursively installed "libwget0" package. I wrote a very basic and simple application ...
1
vote
1
answer
5k
views
Download file in an authenticated state with wget in bash
I'm trying to download the Civit AI model Realities Edge XL Turbo in a shell, but since the author has required the users to be logged in to download it, I am having trouble to use wget inside my ...
0
votes
1
answer
5k
views
How to download file from yandex drive to google drive directy using google colab?
Let's suppose that I have any archive or file on yandex drive like that: https://disk.yandex.ru/d/KGA6qXDT87pTVA
I want to download it directly to my google drive, how could I do it?
First idea that ...
-1
votes
1
answer
621
views
MacOS luarocks: "Failed searching Manifest". "wget" returns "Library not loaded /usr/local/opt/libunistring/lib/libunistring.2.dylib"
This is kind of a convoluted issue that people seem to stumble upon in different ways, so here's the context I found myself in, and the different solutions I know worked for others, that should be ...
0
votes
1
answer
139
views
It is possible to replace curl in Julia Download with wget?
By Default Julia uses curl to download files on Linux:
/usr/bin/env julia
julia> using Downloads
julia> Downloads.download("https://pkg.julialang.org/registries"; verbose=true)
It is ...