Anycast DNS and edge for a SaaS company, absorbing attacks close to where they start
A business-software company serving customers in North America and Europe, a leader in its segment.
- Scale
- 9 points of presence, own ASN and address space, anycast DNS and HTTPS edge, DNSSEC
- Engagement
- Design, deployment and 24x7 operation of a global edge
The challenge
The company’s authoritative DNS and login endpoints were prime targets for volumetric attacks, and one provider’s outage had previously taken its customers offline. It needed an edge that was fast everywhere and could take a punch.
The company's authoritative DNS and login endpoints were prime targets for volumetric attacks, and one provider's outage had previously taken its customers offline. Its DNS lived with a single vendor, its web front door with another, and neither could be moved without a long change window. It wanted an edge that was fast everywhere, owned by the company and able to take a punch.
Anycast turns a weakness into a strength: the same IP address is announced from many places, so each client reaches the nearest one and an attack is spread across all of them rather than concentrated on one. The company operates its own ASN and IPv4 and IPv6 blocks, so every site announces them using BIRD 2 over BGP (RFC 4271), with RPKI route-origin authorizations published for its prefixes and received routes validated (RFC 6811). Each point of presence runs authoritative DNS, signed with DNSSEC and protected by response-rate limiting against reflection abuse, and the HTTPS front door. Two independent DNS server implementations run side by side, so a single software bug cannot silence every answer.
The rollout started with a quiet prefix. A test prefix was announced from all nine points of presence and checked from outside vantage points, then a small share of real zones was delegated to the new nameservers while the old ones stayed in the delegation set. Over six weeks the old nameservers were removed from the delegation, and the HTTPS front door moved last, one customer-facing hostname at a time, with a low DNS TTL so a rollback took minutes. Each site announces its routes only while its local health checks pass.
Defences are layered. XDP/eBPF programs drop junk at line rate on every edge server, FlowSpec (RFC 8955) pushes filters to carriers, communities trigger remotely triggered blackholing of a single targeted address as a last resort, and an upstream scrubbing provider takes over when an attack outgrows local capacity. In a planned test with the carriers informed in advance, a 240 Gb/s mixed UDP and SYN flood was aimed at a single point of presence. BFD detected failing sessions in under a second, traffic shifted to neighboring sites, and query latency stayed flat for customers outside the targeted region.
The outcome is DNS answered from the nearest point of presence worldwide and a login path that no longer depends on one vendor. The company can withdraw a site from the internet in seconds for maintenance, absorb an attack without a customer-visible change, and prove ownership of its address space with signed route-origin records. On-call engineers get a runbook for each escalation tier, and the team rehearses them quarterly.
What our team delivered
- Its own autonomous system number and IPv4 and IPv6 allocations, announced by BIRD 2 from nine points of presence
- Anycast authoritative DNS with DNSSEC and response-rate limiting, served by multiple independent resolver software stacks
- An anycast HTTPS front door with health-checked withdrawal of a failing site’s announcement
- XDP/eBPF packet filtering to drop floods before they reach the kernel stack
- BGP FlowSpec, RTBH communities and upstream scrubbing as escalation tiers
- RPKI route-origin authorizations for the company’s prefixes and validation of received routes
Result
DNS answered from the nearest point of presence worldwide, and a 240 Gb/s volumetric attack in testing was absorbed with no customer-visible change. In the modeled design, the nine points of presence announced the same prefixes over BGP with health-checked withdrawal, so a failing site left rotation in seconds. Quarterly drills withdrew a site on purpose, and the 240 Gb/s test was staged with the upstream providers under an agreed window. Escalation ran from XDP filtering at the edge, to FlowSpec and RTBH, to upstream scrubbing, with the 24x7 team following a written runbook.
Get this for your company
Related services with fixed prices and a written scope.
Emergency package: security engineer in 1 hour
Is your company in a cybersecurity crisis? Purchase this package and a security engineer will call you within 1 hour.
$1,950 per incident
Initial assessment of public endpoints
What an attacker can see and reach from the internet: your domains, mail, websites, remote access and cloud exposure.
$1,000 per assessment
Cloud security review
A review of one Microsoft 365, Azure, AWS or Google Cloud tenant against recognized benchmarks.
$2,800 per tenant
Show 1 more
Cloud management
We run your Microsoft 365, Azure, AWS or Google Cloud environment: access, cost, patching, backup and security settings. Priced per month for one tenant.
$1,900 per tenant, per month
After you purchase, we contact you within 1 hour, any time of day or night, to start onboarding.
No payment is taken online. When we call you within the hour we confirm the scope and send an invoice (ACH, card or crypto).
One address, three places, no outage
The same /24 is announced from three points of presence. Each user reaches the closest one. When a PoP fails its health check it withdraws the route, and BGP sends its users to the next closest PoP with no change on their side.
The shortest path is the one you peer for
Without peering, traffic crosses paid transit through several networks. Once the enterprise joins an internet exchange and peers with the content network through the route server, the same traffic takes one AS hop, at lower latency and with no transit bill.
Detecting a dead neighbour
How long a BGP router waits before it declares a silent neighbour dead, on a log scale. Hold 180 s with keepalive 60 s is the Cisco IOS default and a common default on other vendors (RFC 4271 suggests 90 s and 30 s; some stacks use other values, so check the platform). Tuned timers of 9 s and 3 s are aggressive but valid. BFD (RFC 5880) at 3 missed packets of 50 ms detects loss in 150 ms and tells BGP to tear the session down. Detection time only: route withdrawal and FIB update come after.
A hijacked prefix rejected by RPKI
A rogue network announces a more-specific slice of someone else's prefix, which normally wins because longer prefixes are preferred. A transit router doing route origin validation checks the announcement against signed ROAs, marks it RPKI-invalid and never installs it, so traffic keeps flowing to the real origin.
Facing something similar?
Tell us about your environment. We will tell you plainly what we would do and what it would take.