I’ve used Wordfence country blocking for years, but am recently changing all my sites to free Wordfence due to one of my most used and favorite features being eliminated in version 7, as well as realizing that none of the other few Premium features were necessary for me. Since shifting to free will eliminate WF country blocking, I will keep country blocking using IQ Block Country plugin, which I’ve also used for years and works fine. Whatever you do, don’t just add huge IP lists to .htaccess, major performance hit if you do so.
In terms of performance hit if using Wordfence or IQ Block Country, it’s imperceptible.
One downside of IQ Block Country is you have to manually download and install a file that contains all the IP tables. This is done by simply using FTP, and takes 5 minutes. But it is what it is. I have a calendar reminder, and do it every 8 weeks (the tables gradually change.)
In terms of country blocking being useful, I’ve found it incredibly useful both as a security measure as well as reducing bandwidth.
For example, I manage a couple of sites that are quite regional. With one we block everything but Canada and US, with another we only allow US.
Folks will chime in and point out that determined criminals will bypass country blocking using VPN. Whatever. It still works, when it works.
MTN
-
This reply was modified 7 years, 8 months ago by
mountainguy2.
Thanks a lot, @mountainguy2!
Re “In terms of performance hit if using Wordfence or IQ Block Country, it’s imperceptible.”, I am just wondering whether you blocked large countries (with a lot of IP ranges) like China or Russia?
P.S. It would also be nice to define AWS IP ranges as “Amazon” country and block them entirely – I see a lot of annoying bots coming from AWS IPs.
Yes, like I said, some of my sites have everything blocked but North America, or just US. Performance is fine. Install IQ Block and test, probably works about the same.
Funny point about making AWS into a country!
MTN
Thanks again, I will give IQ Block a try.
When I was richer I used the paid version of WF to block by Country. Now that I’m poorer I’ve found a different ‘combo’ method!
1st, I use WF to ‘Custom Block’ by Hostname. This stops the more obvious Countries AND Tor nodes.
2nd, I collect (for 4 or 5 years now) ‘naughty’ IPs and just add ’em to .htaccess and, like @mishash, I too am only concerned with the NA continent.
My IP Blocklist linked below (updated daily!!) is (very) effective for a huge range of Chinese, Eastern Bloc & Russian IPs. Guaranteed!
Slap this IP Blocklist in your .htaccess and guaranteed you will be happier 🙂
You’re all welcome!
-
This reply was modified 7 years, 8 months ago by
Brian North. Reason: 1st Post is always Beta - and non-subscription
-
This reply was modified 7 years, 8 months ago by Brian North.
-
This reply was modified 7 years, 8 months ago by Brian North. Reason: 1st Post is always Beta - and non-subscription
Thanks @aspasa. I actually took another approach and installed IQ Block Country plugin –
as @mountainguy2 recommended – (it seems to work fine though I didn’t measure performance penalty yet), and I blacklisted using Wordfence firewall few most annoying custom ranges at AWS – the list is below.
54.193.0.0-54.193.255.255
52.0.0.0-52.31.255.255
54.200.0.0-54.203.255.255
54.72.0.0-54.95.255.255
54.160.0.0-54.175.255.255
54.72.0.0-54.95.255.255
And here is the list of bots I block by name patterns using vhosts (I run my site on VPS):
#Blocking bad bots
RewriteEngine On
RewriteOptions inherit
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !/error_docs/forbidden.html
RewriteCond %{HTTP_USER_AGENT} (MJ12bot|BLEXBot|Barkrowler|DnyzBot|ExtLinksBot|SiteExplorer|HTTrack|clshttp|archiver|loader|email|nikto|miner|python|Java|Ruby|scalaj) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Go-http-client|winhttp|libwww\-perl|curl|wget|harvest|scan|grab|extract|linkchecker|test%20bot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (BUbiNG|GozaikBot|EveryoneSocialBot|Kraken|meanpathbot|OpenHoseBot|PaperLi|SeznamBot|TweetedTimes|Superfeedr) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (ZoominfoBot|GarlikCrawler|Slackbot|Yandex|SocialRankIOBot|BoogleBot|uMBot-LN|FlipboardProxy|TweetmemeBot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (ShowyouBot|QuerySeekerSpider|LivelapBot|TurnitinBot|YisouSpider|CCBot|Sogou|Crowsnest|imgsizer|AhrefsBot|Baiduspider) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (SEOkicks|Yeti|BuzzSumo|Buzzbot|AdlesseBot|RSSingBot|Climatebot|semanticbot|Plukkie|Feedspotbot|Veooz|LinkpadBot|Climatebot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Fyrebot|GnowitNewsbot|Leikibot|Wotbox|Mediatoolkitbot|MetaURI|tweetedtimes|PaperLiBot|panscient|rogerbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Hivemind|coccoc|Diffbot|semrush|Findxbot|betaBot|Re-Animator|GimmeUSAbot|Mail.RU|Exabot|Traackr.com|WBSearchBot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Ezooms|CCBot|TalkTalk|Ahrefs|James%20BOT|jetsli|AndersPinkBot|Python-urllib|Dow%20Jones%20Searchbot) [NC]
RewriteRule ^(.*)$ – [F,L]
@mishash – Interesting that we achieve the same goals through similar means.
For example my ‘deny from’ line for your IPs in the ‘Fifties’ above looks like:
deny from 51.15 52.0.0.0/10 52.208.0.0/12 54 58 59 60 61
And many thanks for the additional user-agents (missing from my .htaccess above) that I, clearly, have yet to come across. Thanks to you, now I won’t!
(And, you are way savvier than I – who has no idea what your lines below mean
RewriteOptions inherit
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !/error_docs/forbidden.html)
I have weekly cron task that runs shell script parsing through access logs and identifying offenders, any non-essential bots that hit my site more than 300 times per week are added there. I should probably implement another script that checks “outdated offenders” i.e. if listed bots didn’t hit my site for 3 months I will remove them from the list to save performance. In my TO DO list 🙂
Isn’t that the kind of stuff Wordfence is supposed to do for us?
@mountainguy2 – to what kind of ‘stuff’, specifically, do you refer. I’m not fully understanding 🙁
Stuff like what Mishash wrote. More proactive stuff, but programatic instead of needing user intervention. Wordfence does some of that, but I’m always wanting more more more. MTN
Hi @mishash Country Blocking is a premium feature, we can’t discuss any of the premium feautre here on the forums as it’s dedicated only for supporting the free version, if you have any question regarding the premium version feel free to contact us at “presales [at] wordfence [dot] com”.
I’ve noted all the suggestions mentioned in this thread and will pass them onto the team, however I thought it might be helpful to know that some of the features you mentioned are already covered by a widely known server script called “fail2ban“, many server admins configure it on their servers during the first server setup.
Thanks.
Hi @wfalaa, I think some of these features (e.g. blocking AWS IP ranges or maintaining an up-to-date list of “bad” bots/crawlers may be a good addition to paid version. I am sure many of us wouldn’t mind paying for a premium version if the features offered there would be worth it. Wordfence is one of the best plugins out there but, to be honest, I am a bit disappointed with the changes in Wordfence, removal of Falcon caching, interface changes, and memory problems with scans in the latest version (see this link) were quite unpleasant surprises.
Speaking of fail2ban, I have it running on my server… but I didn’t have much success there blocking bots by name. I tried it and using vhosts (i.e. Apache and Nginx settings) seems to be more efficient.
