- Delta Air Lines CEO Ed Bastian said the massive IT outage earlier this month that stranded thousands of customers will cost it $500 million.
- The airline canceled more than 4,000 flights in the wake of the outage, which was caused by a botched CrowdStrike software update and took thousands of Microsoft systems around the world offline.
- Bastian, speaking from Paris, told CNBC’s “Squawk Box” on Wednesday that the carrier would seek damages from the disruptions, adding, “We have no choice.”
First, multiple business systems are already being supported, and the OS only incidentally. Assuming double or triple IT costs is very unlikely, but feel free to post evidence to the contrary.
The suggestion the poster made was that ALL 3rd party services need to have an additional counterpart for redundancy. So we’re not just talking about a second AV vendor. We have to duplicate ALL 3rd party services running on or supporting critical workloads to meet what that poster is suggesting.
- inventory agents
- OS patching
- security vulnerability scanning
- file and DB level backup
- monitoring and alerting
- remote access management
- PAM management
- secrets management
- config managment
…the list goes on.
Anyone that’s been paying attention to gouging and shrinkflation of the past few years of record profits, or the doomsaying virtually anywhere the minimum wage has increased and businesses haven’t been annihilated, would know this is nonsense.
You’re suggesting the companies simply take less profits? Those company’s board of directors will get annihilated by shareholders. The board would be voted out with their IT improvement plans, and replace with those that would return to profitability.
And yes, taking less profits to distinguish your product as a prestige brand is fairly common.
Even load-balancing multiple servers in a homogenous network, where patches are only deployed in phases is better (and a best practice) than what, to outside observers, appears to have been everything going down due to a mass update everywhere, all at once.
Even load-balancing multiple servers in a homogenous network, where patches are only deployed in phases is better (and a best practice) than what, to outside observers, appears to have been everything going down due to a mass update everywhere, all at once.
This is where reason gets subjective. If you’re solving for resiliency against a bad patch, then absolutely, do a small test deployment before pushing everywhere. This is a balance that whatever is being patched is less of a risk than the patch itself.
However, look at what is being patched in this case: AV/malware protection. In this case, you’re knowingly leaving large portions of your fleet open to known, documented, and in-the-wild, vulnerabilities. In the past 10 years we’ve seen headlines littered with large organizations being downed by cryptolocker style malware. Only doing a partial deployment of this AV/malware protection means you’re intentionally leaving yourself open to the latest and greatest crytolocker (among other things). This is a balance where the risk of whatever being patched is more of a risk than the patch itself.
Seeing as we’ve only really had this AV/malware scanner problem hit the headlines in the last 10 or 15 years, and cryptolocker/malware nearly monthly for the last 10 to 15 years, it would appear on the surface that pushing the patches immediately actually the better idea.
Which of the things you listed have kernel-level access?
Kernel level access isn’t a requirement the poster @Th4tGuyII@fedia.io placed on their suggestion that all 3rd party services should have at least one duplicate 3rd party service serving each function.
