Earlier this week a cabinet in one of the data centers I work in suffered a power failure when a fuse melted.

Please note, that’s melt not burn. This tiny little fuse had clearly been slowly getting warmer. Over the several years, its been in operation and the metal had finally separated and seeped out the fuse caps.

The whole cabinet was dead. 

It didn’t take long to diagnose.

Thankfully it was out of hours.

So the service was returned to normal before anyone really noticed.  Thing is though – it was avoidable.  Almost all of the equipment in all the cabinets I work with are either redundant or have many redundant components.  Yet this one 20mm x 5mm tube managed to kill the lot.

I can quite happily point out that I was not the designer of this cabinet. Also, I have been pushing to get two independent power rails put in all of the cabs.  It wouldn’t be too difficult to have one power rail support one half of the redundant switches etc and one of the two PSU’s that’s in all the servers.  I guess it’s down to personal taste and experience. But I can’t help but think that power is a pretty significant part of computing infrastructure. therefore we should be considering it just as significant.

You will be pleased to know that we are in the process of addressing this to prevent it from being an issue in the future.  Anyway – that’s a lesson learnt!