web robots

I have written a versatile web-crawler in Python. Ruya http://ruya.sourceforge.net/ It features all aspects of a well-behaved, decent web-crawler for the Internet. Check out the… Read More »Python web-crawler – Ruya