Looping FTP in a bash script

Question

I need to constantly send files automatically over FTP. Originally I just had a script that fed into stdin for FTP, but realized that for each time I do this, I need to close then reopen the connection between server and client. Reconnection takes more time than actually sending the file. I've tried avoiding this by making a separate script to open the connection and running a script to send the file in a loop.

Here's the issue: after running the connection script, ftp automatically disconnects. Here's the connection script.

#!/bin/bash

HOST='192.168.1.2'
USER='user'
PASSWD='passwd'

echo "open $HOST
user $USER $PASSWD
ascii" > /tmp/ftp.$$

ftp -ivn < /tmp/ftp.$$ >> ftplog.txt
rm /tmp/ftp.$$

and this is the script to send the file.

#!/bin/bash
echo "put localfile.txt remotfile.txt" > /tmp/ftp.$$
ftp -ivn < /tmp/ftp.$$ >> ftplog.txt
rm /tmp/ftp.$$

The connection script opens the connection fine but closes it again once it runs. Any way to avoid this?

I should clarify that I am not uploading a list of files, but a single file that is updated by another script and is sent after update. This one file is sent over and over in as close to real time as possible.

Hi, it seems unnecessary complexity is throwed inside here. Why having 3 script execute one after another with input/output redirection, while you could just parametrize a script with a built-in function to FTP? — Francesco Gualazzi
– Francesco Gualazzi, Commented Jul 10, 2015 at 13:33
Can you go into a little more depth? I'm open to other solutions, just trying to avoid constantly closing and reopening my connection each time a file is sent. — James H
– James H, Commented Jul 10, 2015 at 13:39
You can use FTP's mput to upload many files in one connection, but in order to automate this all files should be in a single folder — Francesco Gualazzi
– Francesco Gualazzi, Commented Jul 10, 2015 at 13:43

Francesco Gualazzi · Accepted Answer · 2015-07-10 13:52:16Z

1

you can upoload all files in a folder with the following script

#!/bin/bash

HOST='your host'
USER="your user"
PASSWD="pass"
REMOTEPATH='/upload'

function usage(){
    echo "Usage $0 <folder>"
}

function ftp(){
  ftp -n $HOST <<END_SCRIPT
  quote USER $USER
  quote PASS $PASSWD
  cd $REMOTEPATH
  lcd $1
  mput *
  quit
  END_SCRIPT
}

if [[ $# -ne 1 ]]
    then
        usage
    else
        ftp $1 
        exit 0
fi

edited Jul 10, 2015 at 13:52

answered Jul 10, 2015 at 13:47

Francesco Gualazzi

9392 gold badges10 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

James H Over a year ago

This would be a great option if I were looking to upload many different files. However, I'm looking to upload a single file each time it is updated by a different program - Essentially synchronizing a file on the remote machine with a file on the local machine. It's not a group of files being uploaded simultaneously, but a single file being sent at many different points in time.

Francesco Gualazzi Over a year ago

Then why not use rsync?

James H Over a year ago

rsync is definitely still an option. I think rsync may not be as fast as ftp, even with ftp opening and closing connections, simply because ssh is going to slow things up. I will time them both and find out.

David W. Over a year ago

Instead of using mput in the ftp function, use get or put. The standard way to control ftp is to create a netrc script either through an actual file, or in some versions of ftp, through a heredoc as demonstrated. Better would be to use scp which is more secure since the password is not passed as plaintext over the network. You need to setup ssh key pairs, so scp doesn't ask for a password.

James H Over a year ago

I should note that the files being sent are extremely small, so compression/transmission times are negligible compared to the time required to establish a connection.

drmaa · Accepted Answer · 2020-07-09 11:10:02Z

0

My solution to upload all files added today to remote ftp server.

#!/bin/bash

HOST='hostname'
USER='username'
PASSWD='password'

# Local directory where the files are stored.
cd "/local/directory/from where to upload files/"

# To get all the files added today only.
TODAYSFILES=`find -maxdepth 1 -type f -mtime -1`


# remote server directory to upload backup
REMOTEDIR="/directory on remote ftp computer/"

for FILENAME in ${TODAYSFILES[@]}; do
  ftp -n -v $HOST << EOT
  ascii
  user $USER $PASSWD
  prompt
  cd $REMOTEDIR
  put $FILENAME
  bye
  EOT
done

answered Jul 9, 2020 at 11:10

drmaa

3,74411 gold badges55 silver badges78 bronze badges

Collectives™ on Stack Overflow

Looping FTP in a bash script

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related