Alibaba cloud servers being ‘carefully dried’ after firefighter drenching last week

In a status update on Monday, Alibaba Cloud services said it was still gradually restoring access to servers and data affected by an explosion and fire last week. Specifically, servers were being “carefully dried” after sprinklers, and firefighters drenched them to quench a fire. We hope that works well, as unintentional liquid cooling can harm electronics.

Alibaba Cloud monitoring first flagged “network access anomalies” at its Zone C Singapore facilities last Tuesday, September 10. It quickly became aware of a fire in progress and updated customers to reassure them that firefighters were on the scene to quench the reported lithium battery explosion and fire. It seems to have taken just 30 minutes for Alibaba to switch its cloud network and security products to use data centers elsewhere. However, it asked customers who had control over such things to migrate their production workloads ASAP.

Several hours later, on the day of the explosion and subsequent fire, Alibaba updated customers to announce some hardware was being affected by some “abnormalities in high-temperature environment.” Over 12 hours after the first reports on the network abnormalities, Alibaba implemented an emergency power shutdown in the affected building zones. It said that water spray from firefighting was “posing a risk of electrical short circuits.”

The Register revealed further details about the data center disaster via local Singaporean media. It recounted reports that firefighting robots were deployed – thus keeping personnel safe from any potential further explosions and toxic fumes. It also pointed out that service providers like Lazada and Bytedance (TikTok) have experienced significant disruptions while cloud resources were shuffled around.