Sponsored Links

ht.Acess file on Apache server and blocking back link tools

Hi there ive been on a run around with this , ive been trying to block online backlink crawlers via Ht.access in my apache server for some of my websites... i blocked them with via user agent string and ip ranges.

Thing is it seems while ip ranges work they change all the time and user agent string the bot can call itself anything even google bot if it wants , ive looked into using a whitelist , blocking everything but google itself from my website, but that seems like a foolish idea.

Yet somehow some people manage to do it i think via 301 redirects im not sure how this process works though as bots can detect redirects as i understand and follow on, ( also this seems like alot of work to set up multiple sites ) ive tried looking at some of my competitors ranked above me and they show no links at all :) I don't want their links i just want to hide mine ... am i missing something here?

Any help would be appreciated
Regards
Juan Pierre
 
When you say IP ranges, do you mean the full CIDR? You could also identify the common geolocation (country) and block traffic from the whole country if it's not relevant to your site. Bad traffic also tends to come from certain server farms and if you identify the primary company supporting those then you can find all of their CIDRs and impose a blanket block.

Sadly there's no fool proof way to stop everything without also impacting legitimate traffic and IPv6 creates a new headache on this front. I don't think 301s are a totally effective solution either.
 
Hi mark i seem to have made a duplicate thread , i could not find this one for some reason .... Thanks for your response .
Common geo location is india but surprisingly mostly america , ive have read a long time ago that blocking a country specifically america from your website is not a good thing with regards to google and its crawling ( even though we have google.de .za .whatever else there is.

Sadly there's no fool proof way to stop everything without also impacting legitimate traffic and IPv6 creates a new headache on this front. I don't think 301s are a totally effective solution either.

Yea this is kind of the conclusion ive come to, and having thought it over attaining my backlinks will be no pick nick either , and if the webmaster is foolish enough to try them in bulk google might flag them because ive taken months to build them and they all have unique content. I think while they do that , my time is better spent on bettering my websites position.

Although i have not been affected yet ,content scrapers and bandwidth leechers are also something i will have to learn about, do they operate in similar fashion ?
 
Sponsored Links
Depends really, a lot of content scrapers use more familiar servers and so can be easily blocked via IP / CIDRs. Bandwidth leeching is more of a problem but there are .htaccess tweaks that can be used to make that more difficult and a quick Google may offer some examples.

However I wouldn't worry too much about that unless it becomes something that's a big problem, as there will always be people remote linking to your content, which to a certain degree is par-for-the-course of running a public website.
 
Top
Cheap BIG ISPs for 100Mbps+
Community Fibre UK ISP Logo
150Mbps
Gift: None
Virgin Media UK ISP Logo
Virgin Media £24.00
132Mbps
Gift: None
Shell Energy UK ISP Logo
Shell Energy £26.99
109Mbps
Gift: None
Plusnet UK ISP Logo
Plusnet £27.99
145Mbps
Gift: None
Zen Internet UK ISP Logo
Zen Internet £28.00 - 35.00
100Mbps
Gift: None
Large Availability | View All
Cheapest ISPs for 100Mbps+
Gigaclear UK ISP Logo
Gigaclear £15.00
150Mbps
Gift: None
YouFibre UK ISP Logo
YouFibre £19.99
150Mbps
Gift: None
Community Fibre UK ISP Logo
150Mbps
Gift: None
BeFibre UK ISP Logo
BeFibre £21.00
150Mbps
Gift: £25 Love2Shop Card
Hey! Broadband UK ISP Logo
150Mbps
Gift: None
Large Availability | View All

Helpful ISP Guides and Tips

Sponsored Links
The Top 15 Category Tags
  1. FTTP (5473)
  2. BT (3505)
  3. Politics (2525)
  4. Openreach (2291)
  5. Business (2251)
  6. Building Digital UK (2234)
  7. FTTC (2041)
  8. Mobile Broadband (1961)
  9. Statistics (1780)
  10. 4G (1654)
  11. Virgin Media (1608)
  12. Ofcom Regulation (1451)
  13. Fibre Optic (1392)
  14. Wireless Internet (1386)
  15. FTTH (1381)
Sponsored

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms  ,  Privacy and Cookie Policy  ,  Links  ,  Website Rules