Unknown robot (identified by 'robot')
Quite a few Unknown robots spider my sites which is usual, however last few days there is one that has spidered 10000s of pages which is somehwat unusual as the only spiders usually I get spidering everything are the larger SEs that identify themselves.
Does anyone know if this can be any kind of problem if you see an
Unknown robot (identified by 'robot')
in the stats that appears to be going thorugh everything? Or is this normal and just something I hadnt had before?
|