When datacenters lose power, generators should kick in. That didn’t happen for Google in London

Google’s October 20 power outage at its europe‑west‑2‑b zone in London was a textbook case of how even the biggest hyperscalers can trip over their own wiring. The root cause? A mis‑configured programmable logic controller (PLC) that refused to tell the on‑site generators to fire when the UPSs ran out of juice. The fallout hit big‑name customers—Lloyds, HSBC, Royal Mail and Revolut among them—showing that “2N” redundancy on paper does not automatically translate into real‑world resilience.

What actually went wrong?

The facility runs a “2N” power architecture, meaning duplicate power paths for every critical load, backed by a central high‑voltage generator bank managed by a dual‑redundant PLC. In theory that should survive a single feeder loss. In practice, an external underground cable feeding primary feeder “A” snapped just as feeder “B” was already offline for corrective repairs following a quarterly generator load test on 14 Oct 2026. With feeder B’s circuit breaker physically opened, feeder A was left carrying the entire critical load.

Google’s UPSs, which are rated to keep the load alive for five minutes, did their job. But a “voltage‑sensing design flaw and PLC misconfiguration” prevented the generator plant from receiving the automatic start signal. The PLC logic still believed a healthy utility source was present, so it never issued the start command. The UPSs drained, the generators stayed silent, and the data centre went dark.

Why the “2N” claim didn’t save the day

“2N” is a design philosophy, not a guarantee. It assumes that each redundant path can operate independently and that the control systems will correctly detect a loss and switch over. Google’s own admission shows the control layer—its PLCs—was not set up to handle the scenario where both primary and secondary feeds are de‑energized but the logic still reports a healthy source. That’s a classic “single point of failure” hidden in software, not hardware.

For independent hosting providers, the lesson is stark: redundancy is only as good as the monitoring and automation that governs it. If your failover logic can be fooled by a false‑positive health indication, you’ve built a house of cards. The cost of a mis‑configured PLC is far cheaper than the reputational damage when customers like HSBC or Revolut lose connectivity.

Google’s response and why it matters

After the outage, Google rewrote the PLC logic, re‑programming it to recognise the loss of both feeds and to fire the generators automatically. It also announced two action items: a “Communications Enhancement” to speed up alerts to customers, and an expansion of platform‑wide testing for infrastructure‑level events. Those are sensible moves, but they come after the fact.

What’s missing is any indication of a post‑mortem that looks beyond the PLC to the broader change‑management process. The feeder B breaker was opened for repairs after a quarterly test—yet the test itself exposed a weakness that wasn’t addressed before the October incident. Independent operators can’t afford to wait for a vendor’s “post‑mortem” to tighten their own processes; they must audit their own control logic continuously.

What this tells us about hyperscaler pricing

Google charges premium rates for “enterprise‑grade” availability, promising “five‑nine” uptime backed by massive redundancy. Yet the outage shows that the promised architecture can be undermined by a software bug that costs nothing to fix but can cause massive downstream disruption. When you’re paying per‑hour for cloud services, you’re essentially buying the confidence that the provider’s internal controls won’t fail.

For a founder running a boutique data centre, the takeaway is simple: you can offer comparable redundancy at a fraction of the price if you keep the control plane lean, transparent, and manually auditable. Over‑engineered PLC logic that you can’t easily inspect is a risk you can’t pass on to your customers without a discount.

War stories from the trenches

In my own decade of running production servers, I’ve seen similar “logic‑layer” failures that crippled services despite robust hardware. One client’s UPS fleet lasted the full five minutes, but a mis‑configured SNMP alarm prevented the NOC from seeing the loss, delaying manual generator start. Another time, a power‑cable fault coincided with a scheduled maintenance window, and the redundant feed was left offline because the change‑request paperwork missed a step. Those incidents cost the businesses hours of downtime and forced us to re‑engineer the manual fallback procedures.

The Google case is just a larger‑scale version of the same problem: a single point of failure in the control logic that should have been caught in routine testing. It reinforces the importance of “what‑if” drills that simulate the loss of both primary and secondary feeds while the control system is under test.

Actionable advice for independent providers

First, audit your PLC or equivalent control logic. Verify that a loss of all utility feeds triggers an automatic start command, even if the system incorrectly reports a healthy source. Second, implement a manual override that can be executed within the UPS runtime window—five minutes is not a lot of time to scramble.

Third, integrate power‑failure alerts directly into your incident‑response platform, not just into a separate communications channel. Google’s promised “automated alert‑to‑communication mechanism” is a step in the right direction, but you can do it now with a simple script that pushes a pager‑duty alert the moment the UPS switches to battery.

Finally, schedule regular “dual‑feed loss” drills. During a quarterly generator load test, deliberately isolate both feeds and confirm that the generators start without operator intervention. Document the results and keep them on a public status page—transparency builds trust.

Bottom line for founders

The Google outage is a reminder that even the most capital‑rich hyperscalers can stumble over a mis‑configured PLC. Redundancy on paper is meaningless unless the control logic is bullet‑proof and the testing regime actually validates the worst‑case scenario. Independent hosting providers can turn this into a competitive advantage: keep your power architecture simple, your control logic auditable, and your testing rigorous. When a customer asks why they should trust you over the Big G, point to a documented, regularly exercised manual override and a transparent incident‑response pipeline. That’s the kind of practical resilience that saves money and reputation.

— Allan Ali, Founder

This article was produced with AI-assisted research and editorial support. Reporting is based on the source material cited below. Sources: The Register; theregister.com; Global1.News (05 November 2026).

By Allan Ali, Global1.News