Grub Next Generation Python client is a distributed web crawling system (clients/servers) which helps to build and maintain free (as in freedom) index of the Web. This is the console/graphical client written in Python.
Search is part of the fundamental infrastructure of the Internet. And, it is currently broken.
Why is it broken? It is broken for the same reason that proprietary software is always broken: lack of freedom, lack of community, lack of accountability, lack of transparency. Here, we will start to change all that.
Grub started back in 2000 with a simple concept of distributing part of the search process pipeline: crawling. In a way, we were a bit ahead of our time, but our intention then was what it is now. We want to help fix search.
Now, with the help of Wikia, community members, contributers, and Open Source developers our time has come again. Come be part of something greater. Come help us change the World.
Product's homepage
Here are some key features of "Grub Next Generation Python client":
· Python >= 2.4 Compatibility
· Multiple crawlers support
· Configurable
Requirements:
· Python
What's New in This Release: [ read full changelog ]
· Fix fetchWorkUnit return values. Closes Bug http://grub.org/?q=en/node/264
· Support for new workunits. More HTTP headers per request.
· Replacing socket.gaierror with socket.error to catch all socket module exceptions. Closes bug: http://grub.org/?q=en/node/283
· Added Uploading thread. Closing bug http://grub.org/?q=en/node/302
· Added upload speed meter by Tbohdan. Closes bug http://grub.org/?q=en/node/319
· Fix --help
· and code cleanup, small code fixes
Find us on Google+
