HarvestMan

1.4.6 GPL (GNU General Public License)    
3.2/5 13

  1,144 downloads

HarvestMan is a full-featured, multi-threaded Web crawler and offline browser.

description

download

specs

HarvestMan is a multithreaded off-line browser.It has many features for customizing offline browsing through URL filters, word filters, domain filters, URL priorities, depth-fetching, fetch levels, file limits, time limits, robot exclusion protocols, and many more.

It is useful to download an entire Web site or certain files from a Web site to the hard disk for offline browsing later.

It supports HTTP/HTTPS and FTP protocols and can work across proxies.

What's New in This Release:

Fixed bugs in the setup.py and install scripts so that they work with Python 2.4.
read more   
Last updated on September 9th, 2005

#Web crawler #offline browser #multi-threaded web crawler #HarvestMan #Web #crawler #offline

0 User reviews so far.

SUBMIT