Gimli
0
Hi there ive been on a run around with this , ive been trying to block online backlink crawlers via Ht.access in my apache server for some of my websites... i blocked them with via user agent string and ip ranges.
Thing is it seems while ip ranges work they change all the time and user agent string the bot can call itself anything even google bot if it wants , ive looked into using a whitelist , blocking everything but google itself from my website, but that seems like a foolish idea.
Yet somehow some people manage to do it i think via 301 redirects im not sure how this process works though as bots can detect redirects as i understand and follow on, ( also this seems like alot of work to set up multiple sites ) ive tried looking at some of my competitors ranked above me and they show no links at all I don't want their links i just want to hide mine ... am i missing something here?
Any help would be appreciated
Regards
Juan Pierre
Thing is it seems while ip ranges work they change all the time and user agent string the bot can call itself anything even google bot if it wants , ive looked into using a whitelist , blocking everything but google itself from my website, but that seems like a foolish idea.
Yet somehow some people manage to do it i think via 301 redirects im not sure how this process works though as bots can detect redirects as i understand and follow on, ( also this seems like alot of work to set up multiple sites ) ive tried looking at some of my competitors ranked above me and they show no links at all I don't want their links i just want to hide mine ... am i missing something here?
Any help would be appreciated
Regards
Juan Pierre