Copyright (C) 2000-2012 |
Manpages LWP::RobotUASection: User Contributed Perl Documentation (3)Updated: 2001-04-27 Index Return to Main Contents NAMELWP::RobotUA - A class for Web RobotsSYNOPSISrequire LWP::RobotUA; $ua = new LWP::RobotUA 'my-robot/0.1', 'me@foo.com'; $ua->delay(10); # be very nice, go slowly ... # just use it just like a normal LWP::UserAgent $res = $ua->request($req); DESCRIPTIONThis class implements a user agent that is suitable for robot applications. Robots should be nice to the servers they visit. They should consult the /robots.txt file to ensure that they are welcomed and they should not make requests too frequently.But, before you consider writing a robot take a look at <URL:http://info.webcrawler.com/mak/projects/robots/robots.html>. When you use a LWP::RobotUA as your user agent, then you do not really have to think about these things yourself. Just send requests as you do when you are using a normal LWP::UserAgent and this special agent will make sure you are nice. METHODSThe LWP::RobotUA is a sub-class of LWP::UserAgent and implements the same methods. In addition the following methods are provided:
SEE ALSOLWP::UserAgent, WWW::RobotRulesCOPYRIGHTCopyright 1996-2000 Gisle Aas.This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
IndexThis document was created by man2html, using the manual pages. Time: 19:57:37 GMT, April 23, 2024 |