Here’s a likely scenario for what caused this outage:
1- Most managers have no expertise or practical experience in the department they’re managing and therefore cannot accurately assess the competency of their direct reports
2- They have experienced people on their teams who go about doing their job quietly because it’s no longer new to them and no longer a big deal so they don’t talk it up
3- They also have newbies who talk up what they’re doing because it’s new and challenging to them
4- RIF time comes and they decide to RIF the quiet experienced person because in their mind the newbie is some superhero who told them how they overcame all the challenges.
5- New challenging work enters the queue but the experienced person is no longer there so it gets assigned to the newbie
6- Oopsie! We have an outage!
24 replies (most recent on top)
My first thought was that VZ will not provide information on the root cause of the outage, But as a 40 year employee who recently left, I know that they do not know how or why the outage occurred. Sell any stock or might be crazy enough to still own.
.... just make sure you use the Verizon approved Neue Haas Grotesk for your post mortem root cause analysis.
We always performed major software and equipment changes during a specific period of time (window we used to say) in the middle of the night. Everyone was in sync to perform these "out-of-hours" changes verbally on one conference call. Nothing happened without synchronization.
@as Amazing (in a bad way) if true! HSS was also behind the massive 4G outage in 2011. DRA was added back then to prevent this from happening again.
@az FALSE. Don't lay '08 at the feet of bankers. Seeds for that disaster were sown by the late '70s "Community Reinvestment Act" proposed by a democrat, passed by the democrat-controlled congress and signed into law by Carter. Full stop.
@ab You would not be correct at all. It was the Verizon wireless employees that were here and might still be here that didn't test for this. I guess you work in sales since you don't know what happened. It was posted once in here. If you know anything about equipment and so on you will see the correct post
@as Winner! winner!
@a9 snort. The dreaded ngh font and updated templates w/ latest logo hail mary.
Theory number two:
They kept the su-k-ups employed, the ones that did not threaten them.
Like the young-blood bankers responsible for the 2008 economic collapse.
The local banks, with their seasoned leaders, and their understanding of economic basic rules, survived, while the big firms led by loudmouth kids failed.
HSS box in Chubb failed and the backup didnt do its job
@a6 While Verizon India slept…
@ah No need for a name change. Just another new logo.
They have to change name of company now after this debacle
My company phone is dead, but my ATT personal PHONE is working perfectly
Core, but they will never tell anyone. Back in the glory days it was unheard of to do work during the day, now they shut off sites and do all kinds of work to meet a metric on a spreadsheet for a PM. Unless of course Root is driving then you better not touch a site during the day and mess up those results by having a site down. We need those TV bragging rights and I mean only the paying customer gets affected right?
We lost a lot of senior people in our organization. Nothing suprises me anymore with this company. They call it customer centric. I call it stockholder centric. The customer was lost several rifs/layoffs ago. This is the "risk" the company was comfortable with. Curious to see how many subs they lose as a result to this wreckless behavior.
Whatever it is today brought a huge smile to my face. Enjoy your post mortem decks. Remember don’t forget your Verizon NGH font…
@a7 move along if you already left the company, nothing to see here
@a4 I have no first hand knowledge since I’m no longer at the company but a hardware failure wouldn’t cause a nationwide outage and furthermore would be quickly resolved so it’s more plausibly a soft misconfiguration.
You’re reading and commenting on a layoff board so telling me to quit is redundant (no pun intended)
Finally, I am gruntled but I observed a lot of disfunction on the part of Verizon management over the years and I’m now free to say it.
They will say it was a software issue
@OP Step 7- The manager makes a lateral move to another department and lets someone else deal with the consequences of their bad decisions
- Outages happen, doesn’t matter the business or the level of expertise. Do you know if this was caused by something internally, something by a third party provider, or something cause by a hack? Do you know if it was caused by hardware failure? If you’re this disgruntled, you don’t have to stay.
@OP it was the same story. Verizon engineers touched what they knew they shouldn't during the day. Not the ceo, not a contractor not a manger