noncrawl 0.1

A web crawler creating link soups

  Add it to your Download Basket!

 Add it to your Watch List!

0/5

Rate it!
send us
an update
LICENSE TYPE:
GPL (GNU General Public License) 
USER RATING:
UNRATED
  0.0/5
DEVELOPED BY:
Niels Serup
HOMEPAGE:
metanohi.org
CATEGORY:
ROOT \ Internet \ HTTP (WWW)
noncrawl is a web crawler that saves only links. It crawls the web but does not attempt to do everything. Instead, its only purpose is to recursively check sites for links to other sites, which are then also checked for links to other sites, etc.

So, if site Y links to site X, that piece of information is saved, and if site X has not been checked yet, it will be crawled just like site Y was.

Last updated on July 29th, 2010

requirements

#web crawler #link soups #web #crawler #link #soups

Add your review!

SUBMIT