Many robots also pass a special user-agent to the web server when fetching content.[15] A web administrator could also configure the server to automatically return failure (or pass alternative content) when it detects a connection using one of the robot
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.