Technical Information
Understanding how our image discovery technology works
The Alttagger crawling bot is an automated system that discovers and processes images on your website to enable our alt text generation service. This page explains how our bot works, what data it collects, and how it interacts with your website infrastructure.
Our bot is designed to be respectful of your website's resources while effectively finding all the images that need alt text. It follows industry best practices for web crawling and is continuously improved to provide the best service possible.
The Alttagger crawling bot performs several functions to discover and process images on your website:
The bot only crawls websites that have been registered with Alttagger by an authorized user. It won't crawl your website unless you've specifically set up our service and implemented our code snippet.
Many websites use Content Delivery Networks (CDNs) and Web Application Firewalls (WAFs) for improved performance and security. These systems may restrict or block crawler access, including Alttagger's bot.
We provide specific configuration instructions for common CDN and security platforms to ensure Alttagger can properly crawl your website:
Configure Cloudflare WAF to allow Alttagger's crawler
Allow Alttagger in AWS CloudFront and WAF
Configure Akamai Bot Manager to allow Alttagger
Allow Alttagger in Fastly WAF
Configure Vercel Edge to allow Alttagger
If you use a different CDN or security platform and are experiencing issues with our crawler, please contact our support team for assistance.
Alttagger uses two primary methods to crawl websites, depending on the technical requirements of your site:
For static websites and websites with server-rendered content, our standard crawler efficiently processes pages to discover images. This method is lightweight and minimizes server load.
For websites with JavaScript-rendered content (SPAs, React, Vue, Angular, etc.), we use a headless browser approach that fully executes JavaScript to ensure we discover all dynamically loaded images. Our system automatically detects when this method is required.
Both methods are designed to be efficient and respectful of your server resources. We employ intelligent crawling patterns to minimize the impact on your website's performance.
Our crawling bot operates from the following IP addresses:
You can recognize our bot by its user agent string, which includes: AltTagger/1.0 (+https://alttagger.com/bot)
If you notice unusual crawling behavior from these IPs or our user agent, please contact our support team.
To ensure our bot doesn't overload your website, we implement several rate-limiting measures:
These limits ensure our crawler is a good citizen on your website while still effectively discovering all relevant images.
Our crawler is designed with privacy in mind:
Alttagger respects the robots.txt file by default. However, you can override this setting in the Alttagger dashboard.
Our bot initially performs a comprehensive crawl of your website to discover all existing images. After that, it switches to an incremental approach, checking for new or modified content at regular intervals based on your website's update frequency and your subscription plan.
Our bot is designed to have minimal impact on your website's performance. We use adaptive crawling speeds and respect server response times. Most websites experience no noticeable performance impact from our crawling activities.
Our JavaScript-enabled crawler can detect and process dynamically loaded content, including images loaded via AJAX or other JavaScript methods. It waits for the page to fully render before analyzing the content.
Yes, you can view all discovered images in your Alttagger dashboard. This gives you complete visibility into what our bot has found and allows you to manage alt text for each image.
If you have additional questions about our crawling bot or need technical assistance, please contact our support team.
