Features
Scheduled crawls
Updated 2026-05-18
A scheduled crawl runs your audit configuration on a fixed cadence: hourly, daily, weekly, or monthly. Combine with webhooks to get a Slack/Discord ping the moment any score drops below a threshold or a CRITICAL issue appears. Pro = weekly cadence, Agency+ = daily, Enterprise = hourly.
Creating a schedule
Open a Website → Scheduling → New schedule. Pick:
- Cadence:
HOURLY/DAILY/WEEKLY/MONTHLY - Time of day: when within the cadence to fire (UTC)
- Render mode: inherits from the Website default; can override per schedule
- Max pages: useful to cap a daily run at fewer pages than a weekly one for cost control
The scheduled-crawls scheduler ticks every 60s and dispatches due jobs via BullMQ: never on the request thread.
Best-practice cadences
- Marketing site, weekly product updates:
WEEKLYMonday 09:00 UTC, full pages - E-commerce, daily inventory:
DAILY03:00 UTC, lighter (50 pages) - Pre-launch / staging: manual or
HOURLYuntil cutover - Enterprise SLA-tracked surfaces:
HOURLYwith alert rules on any drop > 5 points
Pause / disable
Schedules can be paused without deletion: useful during a planned content migration when scores will deliberately wobble. Paused schedules don't consume crawl quota and don't fire alerts.
Related docs
Ready to try it?
Free tier: 5 crawls / month, no credit card.