Sirobot is a Perl-based web fetch tool that can traverse into the tree and download files in parallel.
Sirobot takes URLs as arguments and is able to download them as well as all given images and links in those html files recursively, too.
The main advantage over other tools like GNU wget is the ability to fetch several files concurrently which effectively speeds up your download.
Here are the ingredients you need to run Sirobot:
· a working Perl 5 interpreter.
You can find out if this is true by typing perl -v. Some text like "This is perl, version 5.005_03 built for i386-linux" along with a copyright notice should appear.
· the perl libraries LWP, URI, IO.
First, find out what libraries are missing:
perl -e 'use LWP;print("LWP is installedn");'
perl -e 'use URI;print("URI is installedn");'
perl -e 'use IO;print("IO is installedn");'
If they are all installed and working, a short message will appear for each package. Otherwise, Perl will display an error message, stating it couldn't locate < missingpackage >.pm in @INC.
You can always get missing libraries from CPAN (see below):
LWP (package libwww-perl, NOT LWPng and NOT LWP-attic!):
URI (package URI):
IO (package IO):
· optionally: the Curses library
The Curses library is used by Sirobot to show progress bars and other informations in a more appealing way.
First, find out whether the library is installed or not:
perl -e 'use Curses;print("Curses is installedn");'
The same rules are valid for this package as for LWP, IO and URI.
· optionally: a SSL library
To use https (secure HTTP) you need a SSL library that extends the capabilities of LWP. Due to USA's export restrictions for cryptographic software, LWP is shipped without SSL support and must be obtained otherwise.
· Sirobot itself