WebMendBot

WebMendBot sends this User-Agent:

WebMendBot/1.0 (+https://webmend.app/bot)

Every request also carries a contact address, per RFC 9110:

From: contact@webmend.app

Nearly all of our requests come from a single, fixed IP address: 46.224.40.254. We don't rotate it, so it's stable enough to allow- or block-list.

That address has a canonical hostname, bot.webmend.app, and the two point at each other. A User-Agent header costs nothing to forge; that pair does. To check a specific request, start from the source address on that request — not from the address we publish, or you'll only be confirming that our records exist:

dig -x THE-IP-IN-YOUR-LOGS +short @1.1.1.1   # → bot.webmend.app.
dig bot.webmend.app +short @1.1.1.1          # → 46.224.40.254

The request is ours only if the first command returns bot.webmend.app. and the second returns the same address you started from. Anything else claiming to be WebMendBot isn't us — with one exception, below.

The @1.1.1.1 is worth keeping: these records have a long cache lifetime, so a local resolver can hand you a stale answer for up to a day after a change.

The exception, so that check stays trustworthy: a small share of JavaScript-heavy pages are fetched for us by a third-party rendering service rather than by our own server. Those requests still carry the WebMendBot user agent and still honor your robots.txt, but they arrive from that service's addresses, so they won't pass the reverse-DNS check and blocking our IP won't stop them. A robots.txt rule covers both paths — see section 4.

• Reads and honors your robots.txt, matching rules against the WebMendBot token — for both passes described below.
• The initial crawl respects a declared Crawl-delay, up to a ceiling of 2 seconds so one site can't stall a scan; absent one, we self-limit to roughly one request per second per host. That pacing is for top-level page fetches during the crawl — a page's own subresources (images, fonts, scripts) are not throttled the same way, in either pass.
• Fetches only public pages it discovers via normal links and your sitemap — no forms, no login flows.
• Runs a small number of scans per site, not a continuous or recurring crawl.

Before a finding is reviewed, we may re-open its page in a real browser to confirm it's still there and capture a screenshot. That pass is different from the crawl:

• It executes the page's JavaScript, the way a visitor's browser would — the initial crawl does not.
• It loads the page's normal subresources (images, fonts, stylesheets, scripts) so the screenshot renders correctly.
• It looks for a cookie-consent dialog and, if one is clearly present, clicks a single reject/decline/"necessary only" action — never an accept action, and never anything else on the page.
• It blocks known third-party analytics and advertising requests on a best-effort basis, so a screenshot doesn't inject a fake visit into your analytics or fire a real ad impression. This isn't guaranteed for every tracker.

A confirmed finding can become a public report, credited to WebMendBot, visible the same way a report a site visitor filed would be. See webmend.app/findings for real examples.

Separately, a link we're about to report as broken gets one more check first: before filing, we reopen it through Cloudflare's managed Browser Rendering service rather than WebMendBot itself. That service sends a standard Chrome user agent, not WebMendBot/1.0 — so a link that only fails for our own crawler, and works fine for an ordinary visitor, is caught here and never reported.

To block us entirely, add this to your robots.txt:

User-agent: WebMendBot
Disallow: /

To slow us down instead, a Crawl-delay directive works too, up to a 2-second ceiling — we won't wait longer than that between requests to your site. We'll pick up either the next time we visit.

You're also welcome to block 46.224.40.254 at your firewall or CDN. That stops our own crawler, but not the third-party rendered fetches described in section 1, which come from other addresses. The robots.txt rule above is the one that covers everything.

If you'd rather not be crawled at all, or have any other question about this bot, email contact@webmend.app and we'll sort it out.

If crawling is fine but you'd specifically rather we not file automated reports about your site, email the same address and say so. We'll verify you control the domain, then apply that as its own setting — separate from blocking the crawler entirely, and separate from any email or outreach preferences on your account.