I know someone here knows? I'm reading something about a "new jersey server failure" in some news articles.
14 replies (most recent on top)
@bw They should put you in charge!
@ay The Nokia HSS software runs on a cluster of servers. There’s a standby HSS at another location for geo-redundancy. So in theory it is not a single point of failure.
In practice this is the second time it causes a nationwide outage lasting 10 hours+. Nokia should have designed it to be more robust and more resilient.
@az it was verizon wireless engineers that failed us again. Learn the network. AI is a step up
@ay it's not
How is AI management of network software working out for you?
@ak but why is the whole nation reliant on one server in one location?
@OP According to an anonymous person on this board, the HSS (Home Subscriber Server) crapped out and the switchover to a standby HSS didn’t work as it should have. The people who were directly involved are the only ones who will know the full details.
https://www.thelayoff.com/post/@as+1kezdn885
Each time there's a large number of employees removed from the payroll there are multiple outages over the following months.
Custom Delight is program recycled from the 90's.
Without a doubt this is Dan’s fault. Trying outsourcing, automate and A.I. the technicians caused this. Where did he get this idea? Oh from the board of directors.
Verizon has now made the national news 3 times in the last 5 months that was not positive news.
Make no mistake it is ultimately the board of directors decisions that have caused all of this not listening to their experienced technicians that understand the network
Of course they will never accept responsibility for it, and point the finger at everyone else down the line.
Rest assured this will happen again.
the cause was Dan doesn’t give a sh-t. write it down, breathe it, know it, worship it
@a5 They touched it in the back?
@a4 touched what shouldn't be touched and the backups again didn't work like it should
I know, but I might get laid off if I tell...oh wait I already did along with 12,999 of my fellow co workers. Just think back to last years outage and that will give you a clue.
Heard it was just an esim software glitch that normally should have been corrected in minutes but between laying off critical people and having most network people in India where it was a holiday, we didn't react fast enough and it cascaded. You can only cut and outsource so much before things start crumbling. This is what you get when you chase pennies.