It’s not the network! Or is it? It depends whom you ask.

Perception may be reality, but it depends on whom you ask. Network practitioners and managers/executives have quite different ideas of the extent and impact of routing issues in the WAN according to our latest survey. This is one of the most interesting findings of the independent research we asked TechValidate, a California-based research firm now part of SurveyMonkey, to conduct for us. The company surveyed network professionals and executives in 27 countries on five continents. Here are the results (we also asked about the current adoption of and expectations for WAN SDN, which we’ll cover in another blog post).

Note: The target group excluded Packet Design customers to avoid skewing the results (we like to think that they are better equipped than others to deliver superior service quality across the WAN). Given that most of the world’s tier-1 network operators are Packet Design customers, their exclusion also meant that most respondent organizations were regional/tier-2 network operators with smaller WANs (and presumably, fewer routing-related issues).

Q: What percentage of service delivery problems are caused by routing issues in the wide area network?

The good news is that routing issues are generally considered to be a relatively small component of overall service delivery issues. However, there is a variance in this perception between network practitioners (network engineers, architects, planners, operations technicians, etc.) and their managers: All of the surveyed practitioners claim that routing issues account for 50% or less of service delivery issues, whereas a fifth of the managers and executives estimate that between 51% and 75% of such issues are caused by routing conditions. This discrepancy was even more marked in the responses to the following question.

Q: What percentage of routing issues are intermittent and cannot be easily recreated?

Here, the results reveal that only a third of the practitioners but half of the managers reckon that more than 50% of routing-related service delivery issues are intermittent in nature and difficult to isolate and diagnose (many of these trouble tickets are closed with no problem resolution). Could this be because managers must interact more frequently with irate customers?

Q: What are the most common causes of routing problems that impact service delivery?

Not surprisingly, accidental router misconfigurations topped the list. This is something we hear frequently in our sales engagements. Being able to diagnose these issues quickly is one of the primary drivers of Packet Design sales. So too is the ability to predict the impact of maintenance changes to avoid unintended consequences. Somewhat surprising was the high percentage of service-impacting problems associated with network equipment: failures or bugs (51%) and interoperability issues (20%).

The practitioner and manager responses were fairly consistent with one notable exception: Thirty-nine percent (39%) of practitioners compared with only 13% of managers said CE configuration changes are a common cause of service delivery problems. Could this again be a reflection of the latter group’s more direct and frequent contact with customers?

Q: On average, how long does it take to identify and resolve a routing-related service delivery problem?

This question generated a wide variety of answers. Most routing-related issues are resolved in less than two hours but more than a third take on average more than two hours to resolve. There were not significant differences based on the size of the company, although large (Global 500 and S&P 500) companies did report a higher percentage of routing-related service-impacting issues that take more than six hours to resolve (32%).

Again, there were differences in perspectives between practitioners and managers, with the latter generally indicating higher average MTTR. Regardless, for service providers concerned about brand reputation and subscriber churn (see below), these responses suggest an opportunity for improvement.

Q: Which impacts do service delivery problems have on your business?

Brand/reputation damage was listed as the number one business impact of service delivery issues across all job types. I expected practitioners to rate SLA subscriber churn or even SLA penalties higher, but clearly, competitive pressure and the importance of maintaining a good reputation are widely appreciated by everyone at these service providers.

Here’s what I think based on these results and others from the survey (my opinions don’t reflect those of Packet Design and cannot be substantiated by the data):

1. Generally, the surveyed managers and executives reported a higher number of routing-related incidents, intermittent issues, and longer MTTR than the practitioners. These discrepancies suggest that there is a lack of data to support them, or at least, that the relevant data is not available to all internal stakeholders.
2. It is natural for people on the front lines of network service assurance to downplay issues for which they are directly responsible, and they may have unwittingly skewed their answers toward those things that are beyond their control. It is impossible to assess if and how this affected the survey results, but it is one possible explanation for the differences between the practitioner and manager responses.
3. Executives and managers at service provider organizations of various types are more likely to interact with their customers, especially when there are service level breaches. They must appease irate customers and understand the circumstances on both sides surrounding specific issues. As a result, they likely develop closer business relationships and may have a more balanced perspective.

What do you think? Are there discrepancies between your perception of network issues and those of your peers/bosses/subordinates? I would appreciate your comments as well as any data or observations you can share from your organizations.