Use wget with proxy

Rajeswari sent a mail with this query:

Can some one help us out in unzipping the files which are there on the server by using a program and put those extracted files to a particular folder on the server path.

Example: Go to the link http://www.thaifxrates.net/

Click on CSV Button, you will see a zip file.

This was my response:

Hi Rajeswari,

You can use wget utility to get the file from the server

You can use unzip utility which is present in all unix servers like this as an operating system call in your program

$ /usr/sfw/bin/wget http://www.thaifxrates.net/ER_CSV_TH.zip
--11:07:42-- http://www.thaifxrates.net/ER_CSV_TH.zip
=> `ER_CSV_TH.zip'
Resolving http-proxy.justanexample.com... 192.168.40.200
Connecting to http-proxy.justanexample.com[192.168.40.200]:80... connected.
Proxy request sent, awaiting response... 407 Proxy Authentication Required
11:07:42 ERROR 407: Proxy Authentication Required.

It errored out because I had not put in the proxy username and password. So I added that:

$ export http_proxy="http://proxyuser:password@http-proxy.justanexample.com:80"
$ echo $http_proxy
http://proxyuser:password@http-proxy.justanexample.com:80

$ /usr/sfw/bin/wget http://www.thaifxrates.net/ER_CSV_TH.zip
--11:08:57-- http://www.thaifxrates.net/ER_CSV_TH.zip
=> `ER_CSV_TH.zip'
Resolving http-proxy.justanexample.com... 192.168.40.200
Connecting to http-proxy.justanexample.com[192.168.40.200]:80... connected.
Proxy request sent, awaiting response... 200 OK
Length: 2,256 [application/zip]

100%[=====================================>] 2,256 --.--K/s

11:08:58 (21.51 MB/s) - `ER_CSV_TH.zip' saved [2256/2256]

$ unzip ER_CSV_TH.zip
Archive: ER_CSV_TH.zip
inflating: ER_CSV1_2009-07-14.csv
inflating: ER_CSV2_2009-07-14.csv
inflating: ER_CSV3_2009-07-14.csv
$

In case your download breaks due to whatever reason, you can resume the download with wget option -c (continue)

So you can execute

/usr/sfw/bin/wget -c http://www.thaifxrates.net/ER_CSV_TH.zip to resume your download