Skip to content

Add ClaudeBot to bad_bots + reg.ru to spam_ref#202

Open
ShaiMagal wants to merge 4 commits into
mitchellkrogza:masterfrom
ShaiMagal:master
Open

Add ClaudeBot to bad_bots + reg.ru to spam_ref#202
ShaiMagal wants to merge 4 commits into
mitchellkrogza:masterfrom
ShaiMagal:master

Conversation

@ShaiMagal
Copy link
Copy Markdown

@ShaiMagal ShaiMagal commented May 13, 2024

Match "ClaudeBot" user-agent. Very annoying flood/crawl bot.

cat apache-access.log | grep ClaudeBot | wc -l
185447570

Match "reg.ru" referer. Spamming every day.

cat apache-access.log | grep reg.ru | wc -l
10779030

@ShaiMagal
Copy link
Copy Markdown
Author

ShaiMagal commented May 14, 2024

@mitchellkrogza Can we approve it please? :) This bota are crazy, still spamming :/

@ShaiMagal ShaiMagal changed the title Add ClaudeBot to bad_bots Add ClaudeBot to bad_bots + reg.ru to spam_ref May 14, 2024
@ShaiMagal
Copy link
Copy Markdown
Author

ShaiMagal commented May 23, 2024

UPDATE:

still very hard spamming:

domain.tld 111.222.333.444 - - [23/May/2024:19:00:12 +0200] "GET /XXXy?p=110&q=Farba-Biela-Ru%C5%BEov%C3%A1%2FV%C3%BDkon-35+W+3+2242424+asdasd HTTP/2.0" 200 41382 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"

And a lot of random strings...

@mitchellkrogza

UPDATE2:
more people have same problem (Google is full of this stories)
https://www.phpbb.com/community/viewtopic.php?t=2652265
https://www.reddit.com/r/singularity/comments/1cdm97j/anthropics_claudebot_is_aggressively_scraping_the/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant