SK-REG BLOG

Dieser Block befindet sich derzeit im Aufbau und wird
ständig erweitert. Daher bitten wir um Verständnis, falls noch
nicht alle Funktionalitäten zur Verfügung stehen.
Dieses Projekt sehe ich als Terapie, um nicht zu vergessen
SK Backup für Joomla 5/6 .skb-doc{--a:#B1976B;--b:#d8c3…
SK Sicherheits-Check Das System analysiert die…
5 Karten Poker Beim klassischen Draw-Poker…
Bubble Shooter Feuern Sie die Projektile…
SK Mailformular Das SK Mailformular…
SK Gallery Integration von Bildgalerien…
SK Umfrage Joomla 6 · Version 1.0.2…
Patientenverfügung, Testament und Vollmacht für Angehörige Erstellung einer Patientenverf…
Drei Festnahmen: Austausch der Deutschlandflagge dokumentiert POLITIK · BERLIN · PROVOKATION…
SK Forum Ein ressourceneffizientes Web…
Senso · Simon Vier leuchtende Farbfelder…
Beiträge nach Kategorie Alle Beiträge einer oder…
SK Zwei Klick Video Plugin für Joomla 6. Es…
Module direkt im Beitrag auswählen Modulkonfiguration auf…
Gericht kippt Grenzkontrollen POLITIK Gericht kippt…
SK Brute Force Guard Joomla 6 System Plugin zur…
SK Statistik Besucherstatistik für Joomla…
SK Shop Starter Entdecke den schlanken Shop…
SK Kalender Ein schlanker Terminkalender…
Passwortschutz für Beitragsinhalte aktivieren Das Funktionsmodul SK Text…
SK Content Restriction System-Plugin für Joomla 6…
Geheimnisvolle Zeichen an Türen, Wänden oder Laternen? SICHERHEIT Achtung…
SK Autorenbox Joomla 6 · Plugin-Paket Eine…
SK Artikel Module Zuweisung von Modulen auf Beitragsbasis 🧩 SK Artikel-Module – Module…
Schiebepuzzle Ein echtes Meisterwerk des…
🌐 Translated automatically
JOOMLA 6 · SECURITY

SK Scraping Protection

Protects your website’s content from being automatically extracted by external computers, bots and scrapers. Genuine search engines are recognised and always allowed through, so your SEO remains unaffected.

Key points at a glance

  • Search engines remain unrestricted: Google, Bing, Yandex, Baidu, Apple and Seznam are verified via DNS reverse lookup and are never restricted.
  • Scrapers are blocked: download programmes, scripts, headless browsers, AI crawlers and SEO analysis tools are redirected to a block page.
  • Automatically active: Once installed, the protection works immediately with sensible default settings.
  • With statistics: An admin view shows who was blocked, when and why.

Features

1
Search engine verification

Anyone claiming to be Googlebot is verified via reverse DNS (forward-confirmed). Genuine bots are allowed through without restriction; fakes receive a 403 error.

2
Bot filter

Blocks scraping programmes (curl, wget, Python, Scrapy, HTTrack, Selenium, Puppeteer and more), AI crawlers (GPTBot, ClaudeBot, CCBot, Bytespider, etc.), SEO analysis tools (Semrush, Ahrefs, MJ12, etc.) and requests without a user-agent. Each group can be enabled or disabled individually, and each has its own block list.

3
Permitted bots

Custom exceptions, such as DuckDuckBot or Claude users. These take precedence over all blocklists and are only throttled to one-third of the standard limit. Link preview bots (Facebook, WhatsApp, LinkedIn, Slack, etc.) can be permitted separately.

4
Rate limit

Anyone who accesses a conspicuously large number of pages in a short space of time will be throttled and blocked for a configurable period (Error 429, default: 60 requests per minute, 30-minute block).

5
Honeypot trap

A link invisible to humans, which only bots follow. Anyone who accesses it will be immediately blocked for 24 hours. Search engines do not receive this link.

6
Copy protection (optional)

Makes it more difficult for visitors to copy content: either for images only (right-click, drag) or for images and text (select, copy). Input fields remain usable.

7
Exceptions

IP whitelist with individual IP addresses and networks (CIDR, IPv4 and IPv6); logged-in users are never restricted. The IP header can be specified for Cloudflare and other proxies.

8
Custom block message

The text on the blocked page can be freely edited. Blocked pages are served with the noindex tag and never appear in search engine results.

Statistics in the admin panel

Under Components → SK Scraping Protection, you’ll find the statistics:

  • Key figures: blocks in the last 24 hours, in the last 7 days and in total; different IP addresses and currently blocked IP addresses
  • Bar chart for the last 14 days
  • Analysis by reason (scraper programme, AI crawler, honeypot, rate limit, etc.)
  • The most notable IPs over the last 30 days
  • Currently blocked IPs with block end date and Unblock button
  • The last 100 blocks with time, IP, reason, user-agent and URL, filterable by reason
  • Clear statistics with a single click; old entries are automatically deleted (default: after 30 days)

The view adapts to both the light and dark admin themes.

Setup

  1. 1
    Install the pkg_skscrapeguard package in the Joomla admin panel under System → Install → Upload Extensions. The plugin will be activated automatically.
  2. 2
    Add the following line to the robots.txt file Disallow: /skguard-falle so that legitimate bots never trigger the trap.
  3. 3
    Add your own IP address to the whitelist if you’re testing the site using scripts or tools.
  4. 4
    If you’re using Cloudflare or a proxy, set the header name (e.g. CF-Connecting-IP), otherwise the plugin will only see the proxy IP.

Good to know

  • A scraper that masquerades as a normal browser and fetches data very slowly can never be completely ruled out. The protection keeps the vast majority of bots and tools out and slows down anything that looks suspicious.
  • The User-Agent can be spoofed. Bots in the ‘Allowed Bots’ list are therefore only subject to the rate limit. For maximum security, it is better to use the IP whitelist.
  • The copy protection in the browser is merely a hurdle for casual users and offers no protection against scrapers.
  • Only front-end requests are blocked. The administrator area and logged-in users remain unaffected, as do robots.txt and the sitemaps.
  • Blocked access attempts are also recorded in the Joomla log skscrapeguard.log.php.
Download under Plugins:

Wie bewertest du diesen Beitrag?

– (0 Bewertungen)

0 Kommentare

Noch keine Kommentare. Schreib den ersten!

Deine E-Mail-Adresse wird nicht veröffentlicht.

Anmelden

sk-reg.de