We respect your privacy.

We use strictly necessary cookies to keep you signed in and to protect against CSRF. With your permission we also use a small amount of first-party analytics to improve the product. We do not sell your data and we do not use third-party advertising trackers. See our cookie policy and privacy policy.

Crawlmind

Legal

Security overview

Last updated: 2026-05-11.

This page describes the security posture of Crawlmind as deployed today. We are an early-stage product — what's listed here is what's actually wired in the codebase, not what we plan to ship later. Items we haven't built yet are called out explicitly.

Architecture

  • Containerised stack (Docker Compose) running on a hardened Linux VPS in a single region.
  • Caddy reverse proxy terminates TLS and is the only service exposed to the public internet. The application, database, and Redis live on an internal Docker network and are not reachable from outside the host.
  • Postgres 16 (with the pgcrypto + citext extensions) holds all tenant data on a host-attached volume.
  • Tenant isolation is enforced at the application layer: every read and write is scoped by organizationId, plus role-based access checks on every API route.
  • Background work (crawling, AI enrichment, reports, webhooks) runs as separate BullMQ workers backed by Redis.

Encryption

  • In transit: TLS 1.2+ everywhere, automatically provisioned and renewed by Caddy via Let's Encrypt. HSTS is enabled with a 180-day max-age.
  • At rest: data lives on the host's disk volume. We rely on the underlying provider's volume encryption when available; we do not currently run a KMS-backed envelope-encryption layer for the database.
  • Application-layer encryption: SSO client secrets and SAML keys are AES-256-GCM-encrypted at rest using a configured key (SSO_ENCRYPTION_KEY) before they are written to Postgres.
  • Outbound webhook payloads are signed with HMAC-SHA256 so receivers can verify the body wasn't tampered with.
  • Public API tokens are stored as SHA-256 hashes — the plaintext token is shown to the user once at creation and never logged.
  • User passwords are hashed with Argon2id (m=19456, t=2, p=1).

Access control

  • Operator (root) SSH to the host is disabled; admin access is via a dedicated non-root user with key-based authentication and a firewall (UFW) limited to SSH, 80, and 443.
  • Application admin actions go through a typed platform-admin role and are written to an AuditLog table.
  • End-user organizations use role-based access (Owner / Admin / Manager / Analyst / Viewer) on every protected endpoint.
  • Not yet implemented: end-user multi-factor authentication (MFA / TOTP / WebAuthn). On the roadmap; until then we recommend SSO for accounts that need a second factor.

Application security

  • Strict Content-Security-Policy, HSTS, X-Frame-Options, Referrer-Policy, and X-Content-Type-Options headers, applied by both Caddy and the application middleware.
  • A CSP-violation reporting endpoint (POST /api/v1/security/csp-report) captures any policy breaches.
  • SSRF guard on every outbound URL the crawler fetches: private, link-local, loopback, and metadata-service IP ranges are blocked, and we re-verify the resolved IP after redirects to prevent DNS-rebinding tricks.
  • State-changing routes require a CSRF token; sessions use HttpOnly + Secure + SameSite cookies.
  • Single Sign-On for the Enterprise plan: SAML 2.0 + OIDC, with claim-mapping and configurable enforcement per organisation.
  • Self-service GDPR data export and account deletion endpoints (POST /api/v1/me/data-export, POST /api/v1/me/account-delete).

Operations

  • Structured JSON logs with request correlation IDs; sensitive fields (passwords, tokens, cookies, Stripe signatures) are redacted by the logger.
  • Prometheus-format /metrics endpoints on every service.
  • Optional Sentry integration for error tracking (gated on user cookie consent in the browser).
  • Database backups via the host's automated daily snapshots; manual pg_dump on demand for point-in-time exports.

Testing + assurance

  • Static type checking and unit tests run on every push (GitHub Actions).
  • Containerised images are scanned with Trivy for known CVEs before promotion.
  • A CycloneDX SBOM is generated for each release.
  • Not yet: external penetration test, SOC 2 Type II, ISO 27001. We will commission an external pen-test before a "1.0 / general availability" milestone and publish the date here when it happens.

Roadmap (committed but not yet shipped)

  • End-user MFA (TOTP and WebAuthn).
  • Off-host encrypted database backups to S3-compatible object storage.
  • External pen-test report (under NDA).

Reporting a vulnerability

Please report security issues to [email protected]. We acknowledge reports within 1 business day. Do not publicly disclose before we have confirmed a fix.