搜索引擎机器人名称大全

  有了解网站的人一般都知道有个robots文本文件,这个文件的作用是可以限制搜索引擎的机器人访问网站目录页面的权限。当然也可以直接写在meta标签里。

  什么是机器人,机器人就是常说的搜索引擎的蜘蛛。每个搜索引擎都会其特定的蜘蛛,而且有的搜索引擎还不只有一种蜘蛛,像Google百度这种大的搜索引擎都有好几种不同用途的蜘蛛。。

  既然有那么多的机器人,那要怎么来区分他们呢?每个搜索引擎都会以他们自己的名字来作为其机器人的名称代号。当他们到我们网站来爬行的时候,就会在网站日志里留下记录。

  今天整理了很多搜索引擎的机器人名称,等号前面是搜索机器人的代号、访问名,等号后面是所代表搜索引擎、网站。

AbachoBOT=Abacho.com
abcdatos_botlink=Abcdatos.com
http://www.abcdatos.com/botlink/=Abcdatos.com
AESOP_com_SpiderMan=Aesop.com
ah-ha.com crawler (crawler@ah-ha.com)=ah-ha.com
ia_archiver=Archive.org
Scooter=Altavista.com
Mercator=Altavista.com
Scooter2_Mercator_3-1.0=Altavista.com
roach.smo.av.com-1.0=Altavista.com
Tv_Merc_resh_26_1_D-1.0=Altavista.com
AltaVista-Intranet=Altavista.co.uk
jan.gelin@av.com=Altavista.co.uk
FAST-WebCrawler=alltheweb.com
crawler@fast.no=alltheweb.com
Acoon Robot=acoon.de
antibot=antisearch.net
Atomz=atomz.com
Buscaplus Robi=buscaplus.com
CanSeek/=canseek.ca
support@canseek.ca=canseek.ca
ChristCRAWLER=christcrawler.com
Crawler=crawler.de
admin@crawler.de=crawler.de
DaAdLe.com ROBOT/=daadle.com
RaBot=daum.net
Agent-admin/=daum.net
phortse@hanmail.net=daum.net
contact/jylee@kies.co.kr=kies.co.kr
DeepIndex=deepindex.com
DittoSpyder=ditto.com
Jack=domanova.co.uk
Speedy Spider=entireweb.com
ArchitextSpider=excite.com
ArchitectSpider=excite.com
Arachnoidea=euroseek.net
arachnoidea@euroseek.net=euroseek.net
EZResult=ezresults.com
Fast PartnerSite Crawler=fastsearch.net
FAST Data Search Crawler=fastsearch.net
KIT-Fireball=fireball.de
FyberSearch=fybersearch.com
GalaxyBot=galaxy.com
geckobot=geckobot.com
GenCrawler=gendoor.com
GeonaBot=geona.com
Googlebot=Google.com
googlebot@googlebot.com=Google.com
google=Google.com
moget/2.0=goo.ne.jp
moget@goo.ne.jp=goo.ne.jp
Aranha=girafa.com
Slurp.so/1.0=Yahoo
slurp@inktomi.com=Yahoo
Slurp/2.0j=Yahoo
www.inktomisearch.com=Yahoo
Slurp/2.0-KiteHourly=Yahoo
Slurp/2.0-OwlWeekly=Yahoo
spider@aeneid.com=Yahoo
Slurp/3.0-AU=Yahoo
Toutatis 2.5-2=hoppa.com
Hubater=hubat.com
IlTrovatore-Setaccio=iltrovatore.it
IncyWincy=incywincy.com
UltraSeek=infoseek.com
InfoSeek Sidewinder=infoseek.com
Mole2/1.0=intags.de
webmaster@intags.de=intags.de
MP3Bot=mp3bot.de
C-PBWF-ip3000.com-crawler=ip3000.com
ip3000.com-crawler=ip3000.com
kuloko-bot/0.2=kuloko.com
LNSpiderguy=lexis-nexis.com
NetResearchServer=look.com
MantraAgent=looksmart.com
NetResearchServer=loopimprovements.com
Lycos_Spider_(T-Rex)=lycos.com
JoocerBot=joocer.com
HenryTheMiragoRobot=mirago.co.uk
mozDex/=mozdex.com
MSNBOT/0.1=MSN
Gulliver=northernlight.com
ObjectsSearch/0.01=objectssearch.com
PicoSearch/=picosearch.com
PJspider=portaljuice.com
DIIbot=powerinter.net
nttdirectory_robot=navi.ocn.ne.jp
super-robot@super.navi.ocn.ne.jp=navi.ocn.ne.jp
griffon=super.navi.ocn.ne.jp
griffon@super.navi.ocn.ne.jp=super.navi.ocn.ne.jp
Spider/maxbot.com=maxbot.com
admin@maxbot.com=maxbot.com
gazz/1.0=Unknown Spider
gazz@nttrd.com=Unknown Spider
NationalDirectory-SuperSpider=nationaldirectory.com
dloader(NaverRobot)/=naver.com
dumrobo(NaverRobot)/=naver.com
Openfind piranha=openfind.com
Shark=openfind.com
robot-response@openfind.com.tw=openfind.com.tw
Openbot/=openfind.com.tw
psbot=picsearch.org
CrawlerBoy=pinpoint.com
ip3000.com=petersnews.com
AlkalineBOT=AlkalineBOT
Fluffy the spider=searchhippo.com
info@searchhippo.com=searchhippo.com
Scrubby/=scrubtheweb.com
asterias=singingfish.com
speedfind ramBot xtreme=speedfind.de
Kototoi/0.1=s.u-tokyo.ac.jp
Searchspider/=searchspider.com
SightQuestBot/=sightquest.com
Spider_Monkey/=spidermonkey.ca
Surfnomore Spider v1.1=surfnomore.com
Robot@SuperSnooper.Com=supersnooper.com
teoma_agent1=teoma.com
teoma_admin@hawkholdings.com=teoma.com
Teradex_Mapper=mapper.teradex.com
mapper@teradex.com=mapper.teradex.com
ESISmartSpider=travel-finder.com
Spider TraficDublu=traficdublu.ro
Tutorial Crawler=tutorgig.com
UK Searcher Spider=uksearcher.co.uk
Vivante Link Checker=vivante.com
appie=walhello.com
Nazilla=websmostlinked.com
www.WebWombat.com.au=webwombat.com.au
marvin/infoseek=webseek.de
marvin-team@webseek.de=webseek.de
MuscatFerret=webtop.com
WhizBang! Lab=whizbanglabs.com
ZyBorg=wisenut.com
WIRE WebRefiner=wire.co.uk
WSCbot=worldsearchcenter.com
Yandex=yandex.com
Yellopet-Spider=yellowpet.com
Iron33=verno.ueda.info.waseda.ac.jp/
ALink=Link Checkers
AMeta=Link Checker
ASPSearch URL Checker=Link Checker
BlogBot=Link Checker
BMChecker=Link Checker
Bookmark Buddy=Link Checker
Check&Get=Link Checker
CheckWeb=Link Checker
CNET_Snoop=Link Checker
CSE HTML Validator=Link Checker
DRKSpider=Link Checker
DISCo Watchman=Link Checker
DoctorHTML=Link Checker
Email Extractor=Email Extractor
EmailSiphon=Email Extractor
EmailWolf=Email Extractor
FavOrg=Link Checker
Favorites Sweeper=Link Checker
FreshLinks.exe=Link Checker
Funnel Web Profiler=Link Checker
Html Link Validator=Link Checker
The Informant=Link Checker
The Intraformant=Link Checker
InternetLinkAgent=Link Checker
InternetPeriscope=Link Checker
javElink=Link Checker
jdwhatsnew.cgi=Link Checker
JRTS Check Favorites Utility=Link Checker
Lambda LinkCheck=Link Checker
LinkLint-checkonly=Link Checker
LinkAlarm=Link Checker
Linkbot=Link Checker
Linkman=Link Checker
LinkProver=Link Checker
Links=Link Checker
LinkScan Server=Link Checker
LinkSweeper=Link Checker
Link Valet Online=Link Checker
LinkVerify Spider=Link Checker
LinkWalker=Link Checker
Morning Paper=Link Checker
MoveAnnouncer=Link Checker
NetLookout=Link Checker
NetMechanic=Link Checker
www.elsop.com=Link Checker
NetMind-Minder=Link Checker
NetMonitor=Link Checker
Netprospector JavaCrawler=Link Checker
online link validator=Link Checker
Rational SiteCheck=Link Checker
Robozilla=Link Checker
RPT-HTTPClient=Link Checker
SurfMaster=Link Checker
SyncIT=Link Checker
Watchfire WebXM=Link Checker
WatzNew Agent=Link Checker
WebSite-Watcher=Link Checker
WebTrends Link Analyzer=Link Checker
Weblink Scanner=Link Checker
Xenu's Link Sleuth=Link Checker
W3C_Validator=Link Validator
WDG_Validator/=Link Validator
Tooter=Link Validator
citenikbot/=citenik.co.uk
CLIPS-index=clips-index.imag.fr/
Computer_and_Automation_Research_Institute_Crawler=Research Bot
cosmos=xyleme.com
robot@xyleme.com=xyleme.com
DiaGem/=DiaGem
Digimarc WebReader=digimarc.com
EchO!/2.0=voila.com
FinaleRobot=expressus.com
robot-master@expressus.com=expressus.com
Ideare - SignSite=ideare.com
GentleSpider=research.att.com
Gulper Web Bot=Gulper Web Bot
larbin=Unknown Spider
sebastien.ailleret@inria.fr=inria.fr
ghi@lcs.mit.edu=Unknown Spider
MultiText=MultiText
NEC Research Agent=NEC Research Agent
OntoSpider=OntoSpider
sherlock_spider=sherlock.com.cn
Steeler=Steeler
ru-robot=rutgers.edu
0.1_hseo(at)cs.rutgers.edu=rutgers.edu
WebGather=WebGather
xyro=xyro
xcrawler@inria.fr=Unknown Spider
Zao/0.2=Zao
ADSARobot=ADSARobot
AnswerChase=AnswerChase
ASPSeek=ASPSeek
AVSearch=AVSearch
Checkbot=Checkbot
DaviesBot=DaviesBot
deepweb=deepweb.com
GigaBaz=brainbot.com
GigaBazVStheWeb=brainbot.com
crawler@brainbot.com=brainbot.com
Giskard=oralco.com
InternetSeer=InternetSeer
ipiumBot=ipiumBot
InsumaScout=InsumaScout
Katriona=Katriona
LEIA=LEIA
LexiBot=lexibot.com
metabot=metabot
NetCruiser=NetCruiser
NPBot=nameprotect.com
NetZippy=NetZippy
NZBot=navigationzone.com
Opencola=opencola.com
Oxxbot1=Oxxbot
Pansophica=Pansophica
Phoaks=Phoaks
PICgrabber=PICgrabber
PictureOfInternet=PictureOfInternet
erik@malfunction.org=Unknown Spider
PintaSpider=PintaSpider
PolyBot=PolyBot
Squid=Squid
Sqworm=Sqworm
TaWWWantula=TaWWWantula
TeraCrawl=TeraCrawl
TurnitinBot=turnitin.com
UCmore=ucmore.com
UdmSearch=mnoGoSearch
unlostBot=unlost.com
URLBlaze=urlblaze.net
UrlScope=UrlScope
Vagabondo=Vagabondo
vspider=vspider
WAVETools=WAVETools
Webbandit=Webbandit
Webclipping.com=Webclipping.com
webcollage=webcollage
WebCompass=WebCompass
WebGenie=WebGenie
Web Magnet=Unknown Spider
WebMiner=Unknown Spider
Webpush=Unknown Spider
WebSymmetrix=Unknown Spider
webrank=Unknown Spider
webwasher=Unknown Spider
WhosTalking=Unknown Spider
AnzwersCrawl/2.0=Anzwers
fido/1.0 Harvest/1.4.pl2=Planet Search
GAIS Robot/1.0B2=seednet
Googlebot/1.0=Google.com
Gulliver/1.2=Northern Light
Infoseek Sidewinder/0.9=Infoseek
KIT_Fireball/2.0=Fireball
lwp-trivial/1.27=Search 4 Free
Lycos_Spider_(T-Rex)/3.0=Lycos
Scooter/1.0=AltaVista
Scooter/1.0 scooter@pa.dec.com=AltaVista
Scooter/1.1 (custom)=AltaVista
Scooter/2.0 G.R.A.B. X2.0=AltaVista
Scooter/2.0 G.R.A.B. V1.1.0=AltaVista
search.at V1.2=search.at
inktomi=Inktomi Spider
SwissSearch V1.2=SwissSearch
The Informant=The Informant
Ultraseek=Infoseek
WebCrawler/3.0 Robot libwww/5.0a=WebCrawler
WebCrawler-AddURL/2.0=WebCrawler
WiseWire=WiseWire
WiseWire-Alpha-1.0=WiseWire
WiseWire-Alpha-Spider=WiseWire
WiseWire-Alpha12-Spider971219a=WiseWire
WiseWire-Alpha12-Spider(971223a)=WiseWire
WiseWire-HotSpider-1.0=WiseWire
WiseWire-Spider=WiseWire
WiseWire-Spider-1.0=WiseWire
WiseWire-Spider2=WiseWire
WiseWire-Widow-1.0=WiseWire
WiseWire-Widow-1.0r=WiseWire
WiseWire-Widow-1.0-ALPHA12=WiseWire
CherryPickerSE/1.0=Email Extractor
CherryPickerElite/1.0=Email Extractor
Crescent Internet ToolPak HTTP OLE Control v.1.0=Email Extractor
EmailCollector/1.0=Email Extractor
EmailWolf 1.00=Email Extractor
ExtractorPro=Email Extractor
ask jeeves=Ask Jeeves
lycos=Lycos.com
whatuseek=What You Seek
wisenutbot=Looksmart
msnbot=MSN
GigaBlast=Gigablast
Gigabot=Gigablast
archive_org=Archive.org
jeeves=Ask Jeeves
Asterias=Singingfish Spider
Slurp=Inktomi Spider
ZyBorg=LookSmart Bot
baiduspider=Baidu

  是不是看得很晕,呵呵,这么多。。其实平时我们经常碰到的没这么多,就几种而已。Google、百度、雅虎、LIVE等是平常我们最经常碰到的。。上述很多都是国外的搜索引擎网站。如果你有做外贸的网站,或许碰到的机器人会更多。。

评论(0)网络
阅读(71)喜欢(0)不喜欢(0)网站排名及优化