Home

Debian/Ubuntu Tips & Tricks

Debuntu.org: .deb packages, Unix/Linux Tutorials and Articles.

sponsors


User login

Poll

Syndicate

Syndicate content


Tips


wget, wput or how to easily download-upload over internet

Most unix administrator might know wget, a HTTP, HTTPS and FTP client developped by the GNU project. Using that tool, you can easily download HTTP page but also packages... Basically anything you can access through your web browser or ftp client.

Uploading something through Internet is, too me, quite annoying in text mode because you need to log in, change path and put the file.

wput is here to resolve this.

wput is great, but before telling you anything read the following line:

WARNING: Mind that when uploading on a ftp server using wput, your password will be seing in clear text. I will explain more about this later on.

Well, let's go through it. A basic use of wget is quite staightforward, for instance, if you want to get the index page of debuntu.org simply type:

tester@laptop:~$wget http://www.debuntu.org
--16:42:32-- http://www.debuntu.org/
=> `index.html'
Resolving debuntu.org... XXX.XXX.XXX.XXX
Connecting to debuntu.org|XX.XXX.XXX.XXX|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
[ <=> ] 26,868 --.--K/s
16:42:33 (220.38 KB/s) - `index.html' saved [26868]

This saved the homepage of debuntu.org in index.html, now if you want get saved the file with another name type:

tester@laptop:~$ wget http://www.debuntu.org -O debuntu-index.html
--16:45:11-- http://www.debuntu.org/
=> `debuntu-index.html'
Resolving debuntu.org... XXX.XXX.XXX.XXX
Connecting to debuntu.org|XXX.XXX.XXX.XXX|:80... connected.
HTTP request sent, awaiting response.... 200 OK
Length: unspecified [text/html]
[ <=> ] 26,868 139.75K/s
16:45:11 (139.27 KB/s) - `debuntu-index.html' saved [26868]

The -O switch allows you to define another name for saving the datas you download. In this case: debuntu-index.html.

There is a lot more options available but this will require pages and pages to go through it. People interested in getting it might simply check out the manpage.

Now, let'see how wput works. A basic use looks like this:

wput FILE URL

So, if I have an account on example.com with user chantra and password foobar and want to upload myfile.txt to /dir1/mydir onto that host, I have to type:

tester@laptop:~$wput myfile.txt ftp://chantra:foobar@example.com/dir1/mydir/

And there I go. But this is where the security issues are:

  1. Somebody typing:

    $ps -aux

    at the same time will see something like:

    tester 15295 11.0 0.1 1640 664 pts/0 D+ 16:58 0:00 wput myfile.txt ftp://chantra:foobar@example.com/dir1/mydir/

  2. this is recording in your ~/.bash_history so somebody gaining access to your account will be able to exploit this to access your remote account on example.com

Therefore I would say that wput is really convenient but to use on your local machine only, not on a server where thousand user are connected (such places like university campus are to avoid ;) ).

security issue way out

1) a way out for the "ps aux" issue is the "-i" option (extract from wput manpage):

−i file
Reads URLs and filenames from file. If there are URLs on the command-line too, these will be retrieved first, unless sorting is enabled. See also the URL-Input-Handling section. If file is −, the URLs will be read from stdin.[...]
Do not do things like find | wput ftp://host/ −i −! Wput would upload all files from the current directory (since the first output of find will be ’.’) and [...]

so you can do:

echo 'ftp://chantra:foobar@example.com/dir1/mydir/' | wput -i - myfile.txt;

and on the ps aux you'll see:

tester 12249  0.0  0.0  1756  676 pts/0    S+   00:28   0:00 wput -i - myfile.txt

2)a way out for the "~/.bash_history" issue:

cat << EOF > wput_cmd.sh;
echo 'ftp://chantra:foobar@example.com/dir1/mydir/' | wput -i - myfile.txt;
EOF
. wput_cmd.sh;
rm wput_cmd.sh;

in the history you'll see:

cat << EOF > wput_cmd.sh;
. wput_cmd.sh;
rm wput_cmd.sh;


paranoic-level security ;)

...if you want paranoic-level security ;)

Prepare the file before the 'cat...':

touch wput_cmd.sh;
chmod 600 wput_cmd.sh;