It is important to have all the latest IP addresses for the search engine spiders as part of your cloaking software. Your software does include automatic updating of the IP addresses, but no system is perfect. Occasionally one will slip by our notice and you'll see a new IP address before we do.
If you come across a search engine robot that is not recognized by your software, it is in your best interests to let us know about it. You can submit it to IPLists.com and they will research it and get back to you. They will also let us know about it.
If IPLists.com determines that your entry is a search engine robot residing on a previously unknown IP address, the IP address will be added to the master lists, and you'll receive it in an automatic update. Alternatively, you could insert the IP address into your list manually.
Here are some tips for detecting unknown spiders:
Look for hits to your cloaked pages with no referring URL. This can be a sign of a robotic visitor. Usually, the web browser of a human visitor will provide a referring URL, but a search engine robot will almost never do so.
Look for unfamiliar User Agents. After reveiwing your logs for awhile, you will get to know the common User Agents. If you see one with which you aren't familiar, send a notice to IPLists.com. You can also look up User Agents at the following resources:
Note that there are many robots and spiders that do not belong to search engines. You may see some of these visiting your cloaked pages. We do not add the IP addresses of these to the search engine IP lists. Only those IP addresses which can be verified as actually belonging to legitimate search engines are added. Of course, you are always free to add any IP addresses you like to your own lists.
© 2002 Volatile Graphix, Inc. KloakIt™ is a trademark of Volatile Graphix, Inc.