Customer Centric Metrics for Duke Email, DNS lookup, and Wifi Login Services

Abstract

This thesis introduces a number of customer-centric metrics to evaluate the experience of users of Duke’s information technology (IT) resources. Real-time monitoring and quantitative analysis are deployed for three critical Duke IT services: email, Domain Name Service (DNS) lookup, and wifi login. Although the Office of Information Technology (OIT) at Duke can track the functionality of these services, no good metrics presented a clear vision about the performance of these systems as perceived by actual end users in the real time. This thesis shows the methodology to obtain the customer centric metrics, results, and corresponding evaluations of these results. The measures proposed here were used both to quantify the users’ experience and to identify and help diagnose problems with these systems that are difficult to identify from inside the systems themselves. As an example, the author found that Duke Email system had a significant problem with delays in May 2016. The reason was that a long latency often occurred in one of the four stages of the email routing process. Once this issue was pointed out, Duke OIT solved this problem by adding six virtual machines and reduced the email delay time in the Duke Email Pre-Processing System by a half.Some of the highlights of the email-specific work include: (1) determining that the latency difference between emails containing Uniform Resource Locators (URLs) and emails with only plain-text content is discovered as very small, (2) the diversity of network paths (emails sent from the Duke CoLab machine and emails sent from Amazon EC2) can affect the differences in email delivery latencies, (3) emails with attachments experienced much longer delivery latency than plain-text emails and emails containing URLs and the differences in the type and size of email attachments can affect the email delivery latencies, and (4) the pattern of the delivery latencies of emails with an attachment was changed by the update of Proofpoint Target Attack Protection (TAP)[1].To quantify in part users’ experiences in visiting widely dispersed parts of the Internet, the author developed a real-time DNS lookup latency monitor where sample domains include the Duke domain, another education domain, and external commercial domains in and outside the U.S.. Samples were taken from both the Duke CoLab virtual machine and Amazon Elastic Compute Cloud (EC2). With the data from this latency monitor, the author discovered important information about the frequency of peaks in the latency on the network.Finally, the author began to explore monitoring the wifi login process, and specifically how long it takes a user to connect to the wifi network, by using a Raspberry Pi[2]. The experimental methodology was very similar to the login process which Duke users usually use on their personal computer (PC) and mobile devices. The author include both samples of (1) the latency of connecting wifi adapter to the access point (AP) and (2) the latency of being ready to use the Internet. The results that the author obtained till March 15 showed that the types of the wifi adapter and the signal strength of the connected AP are two possible factors that greatly affect the wifi login time.All the tools and data analysis techniques developed for monitoring these three IT processes are described in detail in the document.