Various

This chapter contains all the stuff that could not fit anywhere else.

Proxies

Proxies are special-purpose HTTP servers designed to transfer data from remote servers to local clients. One typical use of proxies is lightening network load for users behind a slow connection. This is achieved by channeling all HTTP and FTP requests through the proxy which caches the transferred data. When a cached resource is requested again, proxy will return the data from cache. Another use for proxies is for companies that separate (for security reasons) their internal networks from the rest of Internet. In order to obtain information from the Web, their users connect and retrieve remote data using an authorized proxy.

Wget supports proxies for both HTTP and FTP retrievals. The standard way to specify proxy location, which Wget recognizes, is using the following environment variables:

http_proxy

This variable should contain the URL of the proxy for HTTP connections.

ftp_proxy

This variable should contain the URL of the proxy for HTTP connections. It is quite common that HTTP_PROXY and FTP_PROXY are set to the same URL.

no_proxy

This variable should contain a comma-separated list of domain extensions proxy should not be used for. For instance, if the value of no_proxy is `.mit.edu', proxy will not be used to retrieve documents from MIT.

In addition to the environment variables, proxy location and settings may be specified from within Wget itself.

`-Y on/off'

`--proxy=on/off'

`proxy = on/off'

This option may be used to turn the proxy support on or off. Proxy support is on by default, provided that the appropriate environment variables are set.

`http_proxy = URL'

`ftp_proxy = URL'

`no_proxy = string'

These startup file variables allow you to override the proxy settings specified by the environment.

Some proxy servers require authorization to enable you to use them. The authorization consists of username and password, which must be sent by Wget. As with HTTP authorization, several authentication schemes exist. For proxy authorization only the Basic authentication scheme is currently implemented.

You may specify your username and password either through the proxy URL or through the command-line options. Assuming that the company's proxy is located at `proxy.srce.hr' at port 8001, a proxy URL location containing authorization data might look like this:

http://hniksic:[email protected]:8001/

Alternatively, you may use the `proxy-user' and `proxy-password' options, and the equivalent `.wgetrc' settings proxy_user and proxy_passwd to set the proxy username and password.

Distribution

Like all GNU utilities, the latest version of Wget can be found at the master GNU archive site prep.ai.mit.edu, and its mirrors. For example, Wget 1.5.3 can be found at ftp://prep.ai.mit.edu/pub/gnu/wget-1.5.3.tar.gz

Mailing List

Wget has its own mailing list at [email protected], thanks to Karsten Thygesen. The mailing list is for discussion of Wget features and web, reporting Wget bugs (those that you think may be of interest to the public) and mailing announcements. You are welcome to subscribe. The more people on the list, the better!

To subscribe, send mail to [email protected]. the magic word `subscribe' in the subject line. Unsubscribe by mailing to [email protected].

The mailing list is archived at http://fly.cc.fer.hr/archive/wget.

Reporting Bugs

You are welcome to send bug reports about GNU Wget to [email protected]. The bugs that you think are of the interest to the public (i.e. more people should be informed about them) can be Cc-ed to the mailing list at [email protected].

Before actually submitting a bug report, please try to follow a few simple guidelines.

1. Please try to ascertain that the behaviour you see really is a bug. If Wget crashes, it's a bug. If Wget does not behave as documented, it's a bug. If things work strange, but you are not sure about the way they are supposed to work, it might well be a bug.
2. Try to repeat the bug in as simple circumstances as possible. E.g. if Wget crashes on `wget -rLl0 -t5 -Y0 http://yoyodyne.com -o /tmp/log', you should try to see if it will crash with a simpler set of options. Also, while I will probably be interested to know the contents of your `.wgetrc' file, just dumping it into the debug message is probably a bad idea. Instead, you should first try to see if the bug repeats with `.wgetrc' moved out of the way. Only if it turns out that `.wgetrc' settings affect the bug, should you mail me the relevant parts of the file.
3. Please start Wget with `-d' option and send the log (or the relevant parts of it). If Wget was compiled without debug support, recompile it. It is much easier to trace bugs with debug support on.
4. If Wget has crashed, try to run it in a debugger, e.g. gdb `which wget` core and type where to get the backtrace.
5. Find where the bug is, fix it and send me the patches. :-)

Portability

Since Wget uses GNU Autoconf for building and configuring, and avoids using "special" ultra--mega--cool features of any particular Unix, it should compile (and work) on all common Unix flavors.

Various Wget versions have been compiled and tested under many kinds of Unix systems, including Solaris, Linux, SunOS, OSF (aka Digital Unix), Ultrix, *BSD, IRIX, and others; refer to the file `MACHINES' in the distribution directory for a comprehensive list. If you compile it on an architecture not listed there, please let me know so I can update it.

Wget should also compile on the other Unix systems, not listed in `MACHINES'. If it doesn't, please let me know.

Thanks to kind contributors, this version of Wget compiles and works on Microsoft Windows 95 and Windows NT platforms. It has been compiled successfully using MS Visual C++ 4.0, Watcom, and Borland C compilers, with Winsock as networking software. Naturally, it is crippled of some features available on Unix, but it should work as a substitute for people stuck with Windows. Note that the Windows port is neither tested nor maintained by me--all questions and problems should be reported to Wget mailing list at [email protected] where the maintainers will look at them.

Signals

Since the purpose of Wget is background work, it catches the hangup signal (SIGHUP) and ignores it. If the output was on standard output, it will be redirected to a file named `wget-log'. Otherwise, SIGHUP is ignored. This is convenient when you wish to redirect the output of Wget after having started it.

$ wget http://www.ifi.uio.no/~larsi/gnus.tar.gz &
$ kill -HUP %%     # Redirect the output to wget-log

Other than that, Wget will not try to interfere with signals in any way. C-c, kill -TERM and kill -KILL should kill it alike.

Go to the first, previous, next, last section, table of contents.