Quantcast
Channel: phpBB.com
Viewing all articles
Browse latest Browse all 2802

[3.3.x] Support Forum • Re: apache block crawlers

$
0
0
Disallow is a robots.txt directive not a htaccess.
You also have a duplicate of the SemrushBot.

Code:

SetEnvIfNoCase User-Agent "SemrushBot" bad_botSetEnvIfNoCase User-Agent "Amazonbot" bad_botSetEnvIfNoCase User-Agent "Alexabot" bad_botSetEnvIfNoCase User-Agent "MJ12bot" bad_botSetEnvIfNoCase User-Agent "AhrefsBot" bad_botSetEnvIfNoCase User-Agent "AhrefsSiteAudit" bad_bot<IfModule mod_authz_core.c>  <Limit GET POST>    <RequireAll>      Require all granted      Require not env bad_bot    </RequireAll>  </Limit></IfModule>
Make sure both mod_authz_core and mod_setenvif are enabled.

Statistics: Posted by Anișor — Tue Apr 29, 2025 8:39 am



Viewing all articles
Browse latest Browse all 2802

Trending Articles