#rootcause

Posts mentioning hashtag #rootcause

Below are all the posts — topics as well as replies — that mention the hashtag #rootcause.

Mention #rootcause in your post to continue the discussion!

So, this is what happened for real…

Microsoft 365 Principal SRE here, and I can provide some context regarding the outage…

At approximately 21:40 UTC, an engineer deployed a routine configuration change to Azure Front Door that inadvertently redirected our authentication traffic to a SharePoint site last updated in 2017. We immediately initiated a rollback, but the rollback pipeline requires MFA approval through Microsoft Authenticator, which is currently waiting on a Conditional Access policy evaluation from the same infrastructure we're trying to restore.

We have a break-glass admin account, but its password is stored in a OneNote notebook on the affected tenant. Our on-call engineer attempted a workaround using PowerShell, but the required module was deprecated last Tuesday, and its replacement requires a P2 license. We then opened a Sev A ticket with Microsoft Premier Support, who responded by asking us to run sfc /scannow and provide a screenshot. The prolonged recovery is primarily due to our change advisory board requiring a Teams meeting to authorize the rollback. The meeting organizer is stuck in the lobby, and the only person authorized to admit him was laid off in June.

Current mitigation involves migrating affected users to Microsoft 365 E5, which should resolve the issue from a revenue perspective. We have identified the root cause as customer misconfiguration. Next update in 30 minutes, assuming someone can get Outlook working.


Ford Investigates Stadium Injury

An incident at Ford Field resulted in injuries to six workers. The cause of the event is currently under investigation by Ford. Further details regarding the nature of the injuries or the specific circumstances remain undisclosed. The investigation aims to determine the root cause and prevent future occurrences. This situation highlights a serious workplace safety concern.

Detroit, Michigan

https://www.freep.com/videos/news/local/2026/09/09/rugged-liner-plans-to-cut-jobs-in-owosso-how-many-layoffs-planned/91678466007/


VZ outage - Aug 8, 2026 - I wonder what could be the root cause :)

https://www.foxbusiness.com/fox-news-tech/verizon-suffers-outage-some-parts-us-thousands-reports

https://www.cnbc.com/2026/08/08/verizon-mobile-outage-downdetector.html

https://thepcenthusiast.com/verizon-outage-disrupts-voice-calls-across-the-us-service-now-restored/