CASOPIS REPUBLICKE AGENCIJE ZA TELEKOMUNIKACIJE

Internet usage is evolving, from the traditional WWW usage (i.e. downloading web pages), to triple-play usage where households may have all their communication services (telephony, data, TV) through their broadband access connection. The challenge is to design IP access networks so that they can deliver services with strict QoS demands such as IPTV while the same time having capacity for (from the operator's perspective) unwanted traffic, for example file sharing, demanded by the users.

One important part in meeting this research challenge is to identify and monitor Internet usage. Traffic modelling is tightly coupled both to traffic measurements and to engineering and techno economics. The focus of the measurements in this paper lies on analyzing parameters of interest for network management decisions, i.e. traffic volume, application usage, locality of content and popularity of content. For locality analysis, the geographical end points of data flows are studied. For content popularity analysis, YouTube videos are analyzed looking at the number of parallel sessions and content popularity distributions. Independent of the type of model, traffic measurements are the common denominator that provide input for the model parameters. In this paper, detailed traffic measurements performed as a part of the Celtic TRAMMS project are presented.

1. INTRODUCTION

The Internet, as we know it today, is a network of networks that has continuously evolved over the years as an effect of rapid innovation and development. This innovation and development is driven by various actors with differing agendas. These agendas can be commercial, political, social, developmental, etc., and with the support of a rapid technology development, new services and new ways of communicating emerge. Due to the complexity in the driving forces and the complexity in the underlying technologies that make up the backbone of the Internet, it is increasingly important to monitor and analyze the traffic in the networks and to stay up to date with trends and paradigm shifts.

The main goal of the TRAMMS project is to provide data and analysis for the above mentioned purposes. In short, the project measures and analyzes IP traffic in access networks. The analysis strives to address issues like:

-user behavior,

-user characterization,

-trend analysis,

-bottleneck analysis.

2. ABOUT TRAMMS

The TRAMMS project is a part of the Celtic framework, a EUREKA cluster focusing on telecommunications. The project (Traffic Measurements and Models in Multi-Service Networks) [1] is a three year project with partners from Sweden, Spain and Hungary:

The main objective of TRAMMS is to model traffic in multi-service IP networks, and to use the models as input for capacity planning of tomorrow's networks. The models will be built upon data acquired by advanced traffic measurements at application level with deep packet/deep flow inspection in different parts of Europe, combined with bottleneck analysis and interdomain routing analysis.

Parameters such as applications used, trends in application usage, penetration of applications, peak hours, peak rates, traffic volume, uplink/downlink ratios, network traffic locality, service specific user behavior are analyzed at different time scales, and typical user types are defined. The influence on user behavior from different first mile technologies is studied as well as the difference in user behavior between different regions in Europe.

3. TRAFFIC MEASUREMENTS

A large focus of the project and a particular strength is the direct access to measurements in live networks. In the commercial networks, the measurement equipment is installed near the end-users in order to ensure a high level of detail for the analysis. Traffic measurements in the following fixed metro/access and wireless access networks are performed within the project:

-Swedish municipal network 1: an open fibre based network with FTTH and DSL customers. The FTTH customers represent many social and ethnic groups, while the DSL customers constitute a more homogeneous group of Swedish middle class living in single family houses.

-Swedish municipal network 2: a FTTH network with IPTV users. Measurements from this network were used for studying IPTV user behavior.

-Spanish commercial network: a network containing both fixed and wireless access networks. The wireline part consists of a fibre network to the cabinet (FTTC) and the last mile consists of Cable Modem Termination System (CMTS) and ADSL. The wireless access is a combined GPRS and UMTS system.

-Spanish university network RedIRIS: a network that interconnects and allows Internet access to more than 300 institutions with 2.7 million users. The network is SDH-based with link speeds from 2.5 Gbps up to 10 Gbps.

The measurements in the residential networks have been performed using PacketLogic [2], a commercial real-time hardware/software solution used mainly for traffic surveillance, traffic shaping or as a firewall. Traffic is identified based on packet content and flow behavior using deep packet inspection and deep flow inspection. PL uses the self-developed Datastream Recognition Definition Language (DRDL) [3] to identify different application protocols. The PL can identify more than 1000 application protocols.

The measurements in the RedIRIS network have been performed with Cisco NetFlow [4]. NetFlow is a proprietary network protocol developed by Cisco Systems to run on their routers which is implemented by other vendors as well. This protocol is used to monitor the traffic that traverses a router and to keep statistics of the performance by sampling some of the packets. Cisco defines a flow as a unidirectional sequence of packets sharing all the following 7 values, commonly referred as 7-tuple: Source and Destination IP addresses, IP protocol, Source and Destination ports (when the IP protocol is TCP or UDP), Ingress interface and IP Type of Service.

4. TRAFFIC ANALYSIS

The results in this paper are taken from the TRAMMS public deliverable D3.2 [5], available at [1] on request.

4.1. Daily profiles

An overview of the normalized average daily profiles in the networks is shown in Figure 1.

The figure shows that the Spanish fixed network and the Swedish network, despite using different access technologies (CMTS, DSL, FTTH), have similar daily traffic patterns in the downlink direction. The same similarity is seen for the uplink traffic as well. However, the RedIRIS academic network shows a very different daily downlink traffic pattern. The amount of downlink traffic is less constant in the RedIRIS network than in the other networks (10% of the maximum traffic at 5 a.m.) and from 1 p.m. to 9 p.m. it decreases while it increases in the other networks. The main conclusion is that the shape of the daily traffic patterns depends on the subscriber type of the network such as residential, enterprise or academic. There is also a common daily traffic pattern for the networks that have mainly residential users which is the same for both the Swedish and the Spanish residential networks.

Figure 1. Normalized daily downlink traffic pattern for different networks and normalized average daily downlink traffic pattern

4.2. Comparison of application usage in different networks and technologies

A comparison of application usage in different networks was performed in order to see differences between user groups and countries. This comparison uses data from the Swedish municipal network 1 and the Spanish commercial networks. In order to avoid differences in the amount of unrecognized traffic between the networks, only the traffic from the following application groups has been considered: web browsing, P2P file sharing and multimedia streaming. These application groups are seen to have the largest traffic volumes.

Concerning uplink traffic, P2P file sharing is responsible for more than 97% of the considered traffic in fixed networks regardless of the technology (FTTH, DSL and CMTS). In the case of the downlink traffic in the fixed networks (FTTH, DSL and CMTS), the P2P file sharing generates an important amount of traffic depending on the technology (from 66% to 88%). Approximately 60% of the rest of the traffic corresponds to web browsing and 40% to multimedia streaming.

Regarding the mobile network (GGSN), the amount of web browsing traffic is five times higher than the multimedia streaming. Compared to the fixed networks, the P2P file sharing traffic in the mobile network is lower in uplink (77% of the considered traffic) and much lower in downlink (27% of the considered traffic). In downlink, the mobile network traffic belongs mainly to web browsing (61% of the traffic).

Table 1. Volume share of application groups in different networks

4.3. Content locality

Analyzing the geographical locality of end-to-end flows is increasingly important in order to understand the trends in Internet application usage. This knowledge is also important for e.g. network performance management and optimization. For operators, traffic exchange has a direct impact on their business, making this kind of analysis a crucial part of the business intelligence for network management.

In this paper, network traffic locality measurements in the RedIRIS network are presented. Traffic sent to and from six universities within the RedIRIS network was collected using NetFlow [4] and analyzed to find the traffic source or destination IP address. The IP addresses were mapped with their related countries using MaxMind [6], a public database giving the geographic localization of IP addresses with an accuracy of 99.5%. The analysis was done both for incoming traffic, i.e. traffic from the Internet to RedIRIS entities, and outgoing traffic, i.e. the traffic from the RedIRIS universities to the Internet, as well as for the total traffic in both directions. Results concerning the total traffic are shown in Figure 2. Spain and the United States have been excluded as they dominate the graph. The results for incoming and outgoing traffic are quite similar to the result for the total traffic. More details can be found in the TRAMMS public deliverable D3.2 [5].

Figure 2. Percentages of traffic sent and received by each country (Spain and the US are not included in the figure)

In this study, the majority of packets, around 40%, are sent and received within Spain. This is not unexpected since all of the studied traffic comes from Spanish users and they naturally connect mainly to Spanish sites or to sites with content in Spanish. The United States comes in second place with 20% of the sent and received packets. As the United States is a leading nation in research and IT, this is of course expected. Moreover, the majority of the most visited web pages are hosted in the United States. In the third place we find some of the most important countries of the European Community such as United Kingdom, Germany, France, to mention a few, see top graph in Figure 2. They account for between 2.5 and 6% of the total amount of packets per country. In the fourth position we encounter Latin American countries such as Argentina, Chile, etc., accounting for between 0.5 and 1.5% of the total share. In these countries the main spoken language is Spanish, so it is common to get redirected to a Web page in Latin America when browsing. Also, there are a lot of people from those countries involved in research in Spain thanks to travel grants or other arrangements. Finally, we find that there is traffic going to or from nearly all countries although their percentages of the total are very small (less than 10^-3 % of the traffic). However, their combined traffic accounts for nearly 5% of the total amount.

Figure 3. Traffic in the incoming direction (Spain and the US are excluded from the figure)

We have also studied two more metrics, namely flows and bytes counts. The results are very similar to those presented for the packet count, see second and third graphs of Figure 2. There is, however, one difference that can be seen for the incoming traffic when counting the packets and the bytes. Although more packets were received from Spain than from the United States, the number of bytes received from the United States was greater than that received from Spain which is shown in Figure 3. This is a common phenomenon on the Internet, known as "the elephants and mice phenomenon" [7], where a small percentage of the flows (about 10%) account for a high percentage of the traffic (80%).

The most likely explanation is that the type of traffic to and from the US differs from that to and from Spain. Since the measurements do not reveal the applications used, we can not analyze the application mix in these two cases, but it is likely that in the former case (US) the traffic is dominated by streaming traffic and downloads, while in the later case there is a larger share of instant communication which gives rise to smaller packets.

4.4. YouTube content popularity analysis

One important network design parameter is the usage of caching of popular content. This may help eliminate bottlenecks and reduce traffic in links with limited capacity. There is also an important business aspect here in that traffic exchange may be costly for operators, and it may be in the operator's interest to keep the volume of exchanged traffic down.

The aim of the YouTube content popularity analysis is to investigate popularity distributions for a widespread and popular video sharing network. The purpose of the analysis is to be able to draw conclusions on caching gain, built on real user statistics. The results so far are preliminary, and the final conclusions on caching gain can not be drawn from these initial measurements, but they are very promising and they provide a detailed picture of user activity, user behavior, traffic intensity and content popularity. Naturally, the focus is not on the content of the video itself but on determining properties such as intensity, popularity distribution, content lifetime, etc, of a collection of videos. Preliminary results from this study are shown below.

To achieve the measurements, a firewall rule was set up using the PacketLogic measurement equipment filtering out all HTTP GET requests containing the following query string:

http://www.youtube.com/watch?v=

The equation sign is followed by the 11 character long YouTube content ID, which is the subject of the investigation. The content ID is used to get statistics of the video usage. For integrity reasons, video content and individual usage is not analyzed. The PacketLogic logged and dumped all IP packets containing the given pattern. By processing the trace, it is possible to extract the content IDs and time stamps when the videos were viewed. The latter is determined by the packet arrival time. The packet dump is anonymized and processed to obtain a text file containing only the content IDs and times. This text file is later loaded into a database system for further analysis.

Figure 4. shows the number of viewed videos per hour throughout the 16 day long measurement, which is an estimation of the user activity (and the traffic intensity as well). The user activity seems more intense on weekdays and lower on the weekend (confirmed by Figure 5. as well). In Figure 5. the final day of the measurement is cut so that it contains samples of exactly two weeks; this way no distortion is introduced in the sampling. The same technique is applied to calculate the daily distribution (see Figure 6. below) of access times.

Figure 4. Number of viewed YouTube videos per hour (Swedish network measurement (DSL) from 2008-11-17 to 2008-12-02)

Figure 5. Weekly distribution of viewed YouTube videos per day (Swedish network measurement (DSL) from 2008-11-17 16:31 to 2008-12-01 16:34)

Figure 6. shows the daily distribution of the videos; apparently, user activity is higher in the afternoon and evening hours. The busy hours start at around 4 PM, which is the typical time when people return home from work. The peak hours can be observed around 6-7 PM which is consistent with the daily traffic pattern as shown above in Figure 1.

Figure 6. Daily distribution of viewed YouTube videos per hour (Swedish network measurement (DSL) from 2008-11-17 to 2008-12-02)

Finally, Figure 7. shows the popularity distribution of the YouTube videos. The videos were ranked according to the number of times they had been watched. The figure shows the popularity distribution on a linear-logarithmic scale. It suggests an exponential-like decrease in popularity. Consequently, the popularity of the content is definitely not even; a limited number of videos are extremely popular, while others are watched rarely.

Figure 7. Ranking of YouTube videos according to popularity, Swedish network measurement (DSL) from 2008-11-17 to 2008-12-02

5. CONCLUSIONS AND OUTLOOK

In conclusion, results from the TRAMMS project have been presented. It is seen that there is a common daily traffic pattern for both Swedish and Spanish residential users. The traffic pattern is, however, found to vary between residential and non-residential users. P2P file sharing applications dominate the traffic volumes in the fixed networks followed by web browsing and multimedia streaming. In the mobile networks the volume of the file sharing traffic is smaller than in the fixed networks especially in the downlink direction mainly in favor of web browsing. The content locality analysis performed for the Spanish RedIRIS academic network shows that most of the traffic is sent to and from Spain followed by USA and Europe, the exception being that more packets were received from Spain than from the United States, although the number of bytes received from the United States was greater than that received from Spain. Concerning the YouTube content analysis, it is found that the peak hour is consistent with the general daily traffic pattern. Also, the video content popularity is uneven with a few video clips that are very popular, while the rest are seldom watched.

TRAMMS is an ongoing project and measurement data is being collected continuously. The analysis of these measurements will, of course, go on and be deepened to gain more specific knowledge concerning content and user behavior. The content locality studies and YouTube analysis previously described are preliminary and will be further explored within the project. In the future, the aim is to include more of end user behavior and user quality of experience.

Acknowledgements

The authors would like to acknowledge the Swedish Agency for Innovation Systems, VINNOVA, for funding the Swedish part of the TRAMMS project, and the Spanish Ministry of Industry, Tourism and Trade for funding the Spanish part of the project. Maria Kihl is financed in the VINNMER programme at VINNOVA. Moreover, parts of this work were performed within the VINNOVA project Broadband Behaviour. UAM would also like to acknowledge the support of the DIOR project that provided the RedIRIS measurements.

References

[1]Traffic measurements and models in multi-service networks, http://projects.celtic-initiative.org/tramms/

[2]Procera networks.http://www.proceranetworks.com

[3]Packetlogic drdl signatures and properties 10340 (beta), tech. rep., 2007.

[4]Cisco netflow. http://www.cisco.com/go/netflow

[5]Deliverable d3.2 - traffic models, tech. rep., 2009.

[6]http://www.maxmind.com/app/geolitecountry

[7]K.Papagiannaki, N. Taft, S. Bhattacharyya, P. Thiran, K. Salamatian, and C. Diot: “A pragmatic definition of elephants in internet backbone traffic”,”, ACM SIGCOMM Internet Measurement Workshop, Marseilles, France, November, 2002.

Authors

Andreas Aurelius received his MSc degree in Engineering Physics in 2000, from studies at Chalmers University of Technology in Gothenburg and the Optical Sciences Center at Australian National University, Canberra. Andreas joined the Optical Networks Research lab at Ericsson Telecom in 2000. Here, he pursued research on high-speed optical transmission systems, including experiments, modelling and simulations. In 2002 he joined Acreo AB, working with research on high-speed optical communication. He has been involved in several national and European research projects focused on access networks. In 2005 he initiated the traffic measurements activities at Acreo, and he is now the project coordinator of the Celtic project TRAMMS (Traffic Measurements and Models in Multi-Service Networks). His main research interests are IP traffic modelling and measurements in access networks.

Christina Lagerstedt was born in 1974. She performed her undergraduate studies at the University of Stockholm from which she received her master's degree in physics in 2002, majoring in particle physics and at Mälardalen University from which she received a bachelor's degree in fine arts in chamber music in 2000. In 2003 she joined the Department of Nuclear and Reactor Physics at the Royal Institute of Technology as a graduate student where her research concerned computer simulation of radiation damage in reactor materials. She received her Ph. lic. degree in 2007 after which she joined Acreo AB. At Acreo AB she has performed research in the field of traffic measurements and modelling. Her main interests are traffic measurement and end user behavior analysis.

Maria Kihl received her M.Sc. in Computer Science and Engineering at Lund University, Sweden, in 1993. In 1999 she received her Ph.D. in Communication Systems from the same university. Since 2004, she has been an Associate Professor. During 2005-2006 she was a visiting researcher at NC State University. Her main research area is the performance of distributed telecommunication applications. She has worked on service oriented architectures, web server systems, vehicular networks, and she is currently working on multimedia applications in IP-access networks.

Marcell Perényi received his M.Sc. degree in Computer Science from the Budapest University of Technology and Economics (BUTE), Hungary, in 2005. He is currently a Ph.D. candidate at the Department of Telecommunication and Media Informatics. He has participated in several research projects supported by the EU and the Hungarian government. He is a member of IEEE and the secretary of the HTE Education Committee. His research interests include simulation, algorithmic optimization and planning of optical networks, as well as measurement, identification, analysis and modelling of traffic of IP networks, especially P2P, VoIP and other multimedia applications. He has experience in planning and maintaining database systems, web services and Microsoft infrastructures.

Iñigo Sedano received his M.Sc in telecommunications engineering from University of Deusto, Bilbao, in 2005. Between 2005 and 2006 he worked as a researcher in the Mobility Research Lab group (MoreLab) at Tecnológico Fundación Deusto - University of Deusto. In September 2007 he was awarded a master’s degree in Information Technologies and Communications in Mobile Networks by the University of the Basque Country. Since 2006 he has worked as a researcher at Robotiker-Tecnalia Technology Centre in the field of broadband networks. He has expertise in the development and configuration of network prototypes for access networks and in traffic measurement and modelling. He has participated in research and development projects related to broadband networks in European R&D projects such as PlaNetS (Eureka Medea+), BANITS2 (Eureka Celtic) and TRAMMS (Eureka Celtic).

Felipe Mata Marcos completed his M.Sc. degree in Telecommunications Engineering with Honors at Universidad Autónoma de Madrid (UAM), Spainin 2007, where he was given a scholarship for his good academic performance. Nowadays he is combining his studies in Mathematics and a Master in Computer and Electrical Science in the UAM, where he is currently doing research in networking monitoring and measuring under the F.P.U. fellowship program of the Spanish Ministry since October of 2008 when he joined the Networking Research Group.