Verizon Wireless had a tough December to say the least. There were three major outages, and of course, with all the holiday buying frenzy, it couldn’t have come at a worse time. Should we panic? Absolutely not. Verizon has the best network in the U.S. and they are the leaders in 4G LTE. With that comes growing pains.
Verizon Wireless VP of network engineering Mike Haberman recently spoke with GigaOm to explain what’s been happening and what they’re doing about it.
“Being the pioneers, we’re going to experience some growing pains,” Haberman said. “These issues we’ve been experiencing are certainly regrettable but they were unforeseeable.”
All three outages were caused by the IP Multimedia Subsystem (IMS). IMS is the delivery core and it replaced the old signaling architectures used in 2G and 3G networks. IMS has been around for a while, but this is the first time its been implemented in an LTE network. Unfortunately it has been a problem since April when a software bug led to a complete failure, which kicked customers off both 4G and 3G networks.
Verizon fixed that bug, but new glitches showed up this month. The first outage was on December 7 and was a result of a back-up communications database. The second was last week and that was a result of an IMS element not responding properly. Then there was yesterday, which was caused by two IMS elements not communicating properly.
Basically the LTE radio network was working fine, but customers couldn’t connect to it since the IMS network didn’t recognize them. Verizon attempted to force phones to stop trying to access the 4G network and instead stay on the 3G network. Unfortunately some customers were left without 3G before the switch-over could take place.
Each event was resolved and never re-occurred, but each event was different. To minimize future issues, they are geographically segmenting the LTE network so if something does happen, they will easily be able to isolate it to a particular region instead of it affecting the nationwide network. They are also upgrading software.
Is this the end? Probably not, but lets face it, Verizon has done a fantastic job in growing their LTE network. These kinds of issues are going to happen when you’re the top dog. I have full confidence that Verizon will rectify or minimize any future issues.
[via gigaom]