| P2PRG | S.K. Kamei |
| Internet-Draft | T.M. Momose |
| Intended status: Informational | Cisco Systems |
| Expires: April 16, 2013 | T.I. Inoue |
| NTT Communications | |
| T.N. Nishitani | |
| NTT Communications | |
| October 15, 2012 |
ALTO-Like Activities and Experiments in P2P Network Experiment Council
draft-kamei-p2p-experiments-japan-09
This document introduces experiments to clarify how ALTO-like approach was effective to reduce network traffic made by a Council in Japan to harmonize P2P technology with the infrastructure. And this also provides some suggestions that might be useful for ALTO architecture learned through our experiments. Especially, experiment for application independent ALTO-like server operation.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http:/⁠/⁠datatracker.ietf.org/⁠drafts/⁠current/⁠.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 16, 2013.
Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http:/⁠/⁠trustee.ietf.org/⁠license-⁠info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
An overlay network, which is used by P2P and other applications, offers the advantage of allowing flexible provision of services while hiding the lower layer network. The downside is that inefficient routes are often taken in the lower IP network, thereby increasing the network load. Several proposals have been made to build an overlay network that takes account of the information about the lower layer network. [INFOCOMM2009] [IEEEJOURNAL2004] Since the management of the Internet is highly distributed, it is difficult to implement such proposals and thus optimize a network without the cooperation of network providers.
Recently, the controversy between the overlay network and the network providers about network resource wastefulness has been rekindled. Under these circumstances, some researchers have studied overlay network control technology that takes account of the network topology information obtained from network providers.
One of the activities concerning this issue has been made by the P2P Network Experiment Council in Japan. We planned some experiments in the council. This document reports on the issues addressed and experiments being made by the council.
This experiment made from 2007 to 2008, because P2P traffic had reduced due to rivision of copyright law in Japan. And recently, dominant traffic is HTTP based flash streaming in Japan, U.S, and so on. However, P2P traffic remains in large traffic, like PPStreaming, in Asia (except Japan)[sandvine], and P2P technology is very userful in such a realtime streaming area.
Our experience in this experiment might be useful for ALTO architecture, especially for application independent and multi application ALTO-like server operations. We suggest that generic measurement mechanism is important because each application has different mechanism and it is difficult to compare the effectiveness.
This document is a product of the P2P RG. The views in this document were considered controversial by the P2P RG but the RG reached a consensus that the document should still be published.
As of 2008, the world most popular P2P file sharing application, Bittorrent, isn't widely deployed in Japan. Instead, other Japan specific file sharing P2P applications such as Winny [winny], Share [share], and so on, still occupy 40% of the Internet traffic in Japan even though many those P2P users were arrested for sharing illegal files with these P2P apps.
Each P2P file sharing application has a unique protocol and none of them have a large market share therefore making it hard to effectively control.
One of the advantage of using P2P technology for content delivery is that peers exchange content directly among themselves without server bottleneck. This reduces the load on servers. Also, P2P applications can reduce upstream traffic from an origin content server. This reduce server cost dramatically.
It is also known that server cost could be reduced with P2P technology. However, the story is quite different for network providers. From the viewpoint of network providers, the traffic that content servers generate has shifted to the edge network and the amount of traffic has not necessarily been reduced with using P2P technology for reducing server cost. Another problem for network providers that an extremely inefficient routing may be selected has been raised. It is because overlay network systems are configured without any regard to the structure of the lower layer network or network geometry.
In some cases, the total amount of traffic on the Internet used to be limited by the capacity of servers. For those cases, P2P technology can improve the scalability of servers , however it may exhaust network resources. Moreover, using P2P applications increases the volume of traffic per user remarkably.
Faced with increase in the load on network infrastructure, network providers are compelled to take actions to overcome the sudden increase in facilities' cost. Representative actions include placing content in internet exchanges or data centers, introducing bandwidth control, and raising the access fees [mic].
However, current dominant traffic is HTTP based flash streaming in Japan, U.S, and so on. However, P2P traffic remains in large traffic, like PPStreaming, in Asia (except Japan)[sandvine], and P2P technology is very userful in such a realtime streaming area. The increase in traffic arising from such a shift may be a great threat to the network.
In order to reduce Internet traffic and encourage legitimate use of P2P technologies, the Japanese government led to establish a new council called P2P Network Experiment Council conjunction with commercial P2P application vendors and ISPs in 2006.
Then the council had started to develop regulations that include several guidelines such like an advance notice to restrict bandwidth to heavy traffic users. In accordance with the regulations, some ISPs introduced solutions that reduce traffic caused by P2P file sharing applications.
Besides this activity, the council also looked for new ways to control traffic by commercial P2P applications with ISPs, carriers, contents providers and P2P system vendors. In this work, the council had experiments that introduced ALTO-like system and observed how the traffic was reduced by redirecting to proper peers on the real Internet in Japan.
In our experiment, the council deployed hint servers, which are described in section 4. Hint servers have a protocol offering network distance to peers, which is disclosed to P2P application vendors.
Using hint server, P2P application vendors can introduce ALTO concepts easily to their P2P distribution systems. Because the protocol provided of hint servers is independent on specific P2P application vendors like Bittorrent, the council defines the protocol to be able to use any P2P application vendors. It needs to gather network information from ISPs to offer network distance to peers, however many ISPs dislike to disclose such information to others. Therefore, hint servers are designed to offer little information about ISPs' network architecture to P2P application vendors.
To monitor traffic of peers, the council also deployed dummy node, which are described in section 3.1.
This memo describes the overview of the experiments.
Ministry of Internal Affairs and Communications Japan, which has jurisdiction over information and communication systems in Japan, held meetings of an advisory panel on network neutrality from 2006 to 2007 in order to study issues related to next generation networks, such as how to ensure fairness in the use of networks and how to define fairness in cost burden. The panel took an interest in P2P technology as a solution to the impending traffic saturation in the backbone network resulting from the rapid expansion of broadband access in Japan, and formed a "Working Group on the P2P Network", which carried out an intensive study of P2P networks.
The Working Group reported that it is necessary to undertake the following four activities, which are intended to encourage the government to adopt relevant policies [4]:
The first two proposals led to the establishment of the P2P Network Experiment Council supported by Ministry of Internal Affairs and Communications Japan [5]. The Council, with membership from P2P delivery providers, content holders, and network providers, began a variety of delivery experiments, which were expected to strengthen cooperative control between different layers. In contrast to P4P, which takes a relatively top-down approach of adopting architecture based on a proposal from a university, the Council is characterized by its bottom-up approach. The aim of establishing the Council has been described as follows.
The activities of the P2P Network Experiment Council can be classified into two categories. The first is activities to formulate guidelines for the promotion of the commercial use of P2P technology. These will enable users to use P2P technology safely, and providers to have clear rules they must observe. The other is feasibility tests of P2P technology. The next section describes mainly experiments conducted from 2007 to 2008.
The council has already learned that the server cost could be reduced with using P2P technology for contents delivering by investigating data offered by the members of the council. For example, the data brought by the vendors shows as follows:
On the other hand, these reduced server costs may affect network load. One of the goals of our experiments was to visualize the impacts and propose an architecture to reduce network load caused by these new technologies.
In order to visualize the reduction of network cost, we had to modelize P2P applications and multi-ISP environment. It will also needed for visualizing the effectiveness of ALTO-like approach.
As mentioned before, while the effect of delivery using P2P technology on reducing the traffic and the load on servers is well known, traffic behavior in the inter-ISP is not known. In Japan, there is a backbone traffic report cooperated with ISPs and IXes [impact_user_traffic]. However, this measurement requires to capture packets on subscribers line to know end user's activity. It is not realistic to measure the behavior of P2P applications at user terminals connected to the Internet because that would require a large-scale arrangement for measurement, such as using Deep Packet Inspection (DPI) on aggregated lines.
To solve these problems, we put several nodes called 'dummy nodes' in the ISP's networks. The dummy nodes emulate an end user's PC and P2P applications are running on the nodes. Every P2P node provided by participating vendors in the experiment was configured so it always contacted the hint server.
By introducing dummy nodes, we can observe and evaluate how much P2P applications have affected networks by measuring the traffic on dummy nodes. Since this method can't measure every subscriber's traffic, the accuracy would be less than other methods. But this make it possible to adapt to situations many different P2P applications coexist on a network. We can say this is suitable for these experiments.
A dummy node consists of Intel PC server, Linux(CentOS), VMWare and Windows XP works on VMWare. With this configuration, all packets can be captured without any impacts to the network, nodes and application behaviors. And it enable us to use different P2P applications for windows and evaluate them generally.
To see behaviors of the node, incoming and outgoing packets are captured on Linux because every packets are transmitted through it. In these experiments, we captured source/destination address, port number, amount of traffic and start/end time to see flow information.
60 Dummy nodes are put on access networks that are closest subscriber as possible in different 40 ISP networks.
+----------------------+
|+--------------------+|
||+------------------+||
||| P2P Application |||
||| WindowsXP |||
||| +--+ |||
||+--------|N |------+||
|| VMware |e | ||
|+---------|t |-------+|
| Linux |IF| capture|
+----------| |--------+
+--+
Dummy nodes
Figure 1
In Japan, bottleneck in IP networks have been shifting from access networks to backbone networks and equipments, such as bandwidth between ISPs and capacity in IXs, since FTTH has rapidly spread all over Japan. Under these circumstances, the Council proposed a less restrictive and more flexible cooperation between ISPs than existent P4P experiments [p4p]. The proposed method consists of the following elements: (1) P2P clients, (2) P2P control servers, and (3) a hint server: a peer selection hint server. (1) and (2) are existing systems but whether (2) exists depends on each application. (3) is a server that provides a hint as to the selection of a peer, and plays a role equivalent to that of ALTO Server. Note that this proposal was based on results of experiments using dummy nodes. The results showed that it was possible to reduce unnecessary traffic that flows across the boundaries of geographical districts or ISPs through providing information about the physical network to P2P applications.
When a peer joins the network, it registers its location information (IP address) and supplementary information (line speed, etc.) with the hint server. The hint server calculate network distance between peers (P2P client) based on network topology information obtained from the ISP and generates a priority table for peer selection. The hint server returns the table to the peer.
If all information can be made publicly, the above procedure can produce a result which is close to overall optimization. However, some information held by ISPs can often be confidential. Besides, in some cases, the volume of calculation required to process all information can be excessive. To avoid these problems, it is planned to conduct experiments with a limited set of functions, analyze experiments results, and gradually expand the scope of optimization.
A control mechanism that makes use of all possible information is difficult not only technically but also difficulties to achieve coordination among providers. In consideration of these difficulties, the council has been limiting the implementation and experiments to the following scope since 2006.
Figure 2 shows an outline of the hint server.
+---------+ GetLocation +-------------GeoIP DB Server---------+
| | +-----------+ | +----------+ +-----------+ |
| |--|IP Address |-->| | GeoIP DB | |Quagga etc | |
| | +-----------+ | +----------+ +-----------+ |
| | | +-------------+ +----------------+ |
| | +-----------+ | | District | | Routing | |
| |--|AS Code: |---| | information | |information(DGP)| |
| | |Regional | | | | | | |
|P2P Peers| |Information| | | Range of | |AS Code(origin) | |
| or | +-----------+ | | IP address | | | |
| Contro| | | +-------------+ +----------------+ |
| Server | +-------------------------------------+
| | | ^
| | PeerSelection v |
| | +-----------+ +--------------------------------------+
| |--|IP Address |-->| +--Priority Node Selection System--+ |
| | | List | | | | |
| | +-----------+ | | Peer candidate ranking | |
| | +-----------+ | | | |
| |--| Ranking |-->| +----------------------------------+ |
| | +-----------+ +--------------------------------------+
+---------+
Peer selection hint server
Figure 2
The network information used by the hint server is not information solicited from individual ISPs but the AS number and district information, which are more or less already public. Routing tables are not generated. Instead, peers within the same ISP or the same district are selected with higher priority in order to confine traffic to within the same ISP or the same district.
When the hint server receives an IP address, it returns its attribute information, to achieve the above. A peer can select a peer based on the returned information. This operation is called GetLocation. However, in preparation for the time when it becomes necessary to hide topology information, an interface is provided through which a priority order is returned in response to an input of a list of candidate peers. This operation is called PeerSelection.
Although the target node is selected based on the criterion that it is within the same ISP or the same district, this type of selection is not very effective if the number of participating peers is small. Table 1 shows ratio of peers within the same AS or the same prefecture calculated from the distribution of ASs and prefectures in the IP address space from one-day data on a Winny network.
| Conditions | ratio |
|---|---|
| AS matches | 6.70% |
| Prefecture matches | 12.76% |
| Both match | 2.09% |
| Neither match | 78.45% |
Since, in addition to the above, the presence/absence of content affects the result, the control of selecting a peer within the same district may be inadequate. Therefore, it is necessary to introduce the weight of a continuous quantity that reflects the physical distance or the AS path length as an indicator of the proximity of the areas involved.
In consideration of the above, the following two measures are used for the evaluation of proximity between peers in a hint server.
An example of the request and the response
o Request
POST /PeerSelection HTTP/1.1
Host: ServerName
User-Agent: ClientName
Content-Type: text/plain; charset=utf-8
v=Version number
[application=Application identifier]
ip=IP address of physical interface
port=Port number of physical interface
[nat={no|upnp|unknown}]
[nat_ip=Global IP address using UPnP]
[nat_port= Global port number using UPnP]
[trans_id=transcation ID]
[pt=Flag of port type]
[ub=upload bandwidth]
[db=download bandwidth]
o Response
HTTP/1.1 200 OK
Date: Timestamp
Content-Type: text/plain; charset=utf-8
Cache-control: max-age=max age
Connection: close
v=Version number
ttl=ttl
server=hint server name
...
trans_id=transaction ID
pt=Flag of port type
client_ip=Peer IP address observed from server
client_port=Peer port number observed from server
numpeers=number of respond peer
n=[src address] dst address / cost / option
Table 2 shows the result of the analysis of communication in a node of an ISP installed in Tokyo, as an example of measurement results.
In these two experiments we evaluate different P2P applications, in 1st experiment, the P2P topology is generated by tree algorithm, and in 2nd experiment, it is generated by mesh algorithm. Both of them result in similar performance.
| Conditions | Experiment 1 | Experiment 2 |
|---|---|---|
| *Peers selected within the same ISP | 22% | 29% |
| *Peers selected within the same district | 19% | 23% |
| *Peers selected within the same district and the same ISP | 5% | 7% |
The table shows that the probability of communication with peers in the same ISP is proportional to the number of population and the share of the ISP in each district. The data show that peers were selected at random. Note that the vendor of a P2P application used in these experiments explained that the mechanism of selection a peer using network information can be implemented. However, peer selection is normally based on past information because users often cannot actually perceive the effect of using network information.
The main objective of these experiments was to verify the operations of the hint server and P2P applications. The distances between a dummy node and a peer were obtained from data on the dummy nodes. An examination of the distances between a dummy node and a peer revealed that mean value of distance after the hint server was introduced was reduced by 10% and that 95th percentile was reduced by 5%. The results show introducing hint server can reduce network loads by P2P applications.
We clarified followings throughout our experiments.
In the experimental result of peer selection control, it is smaller in intra-ISP traffic than other experiments [comcast] We think that it is because there are smaller peers in each area of traffic control. When there are many peers in one ISP, it is easy to select peers in the same ISP. However, when there are small peers in one ISP, it is difficult to select peers in the same ISP. In the situation of our experiments, there are many ISPs of peers belonging, and there are relatively smaller peers exist in same ISP.
Moreover, we didn't force P2P vendors to limit their implementation policy, therefore we can observe differences how each implementations weigh the information from the hint servers. Especially, in tree overlay topology P2P applications, such mechanism is very effective, on the other hand, in mesh overlay system, less effective.
Recent research, we've changed the communication protocol to hint servers to ALTO based because it is nearly standardized. In our implementation, PIDs and the value of cost are mapped to ISP subnets, and ISP distance respectively. We also implement services for compatibility required by ALTO such as Service Capability and Map Services. But the Endpoint Cost Service is mainly used because of backward compatibility of our experiments.
We also study hierarchical hint server structure, in order to control in coarse inter-ISPs and in detail intra-ISP. It is also effective for limiting the area of information disclose.
This section describes what the authors learned with these experiments would be useful for the ALTO WG.
In our experiments, we present the possibility of traffic control among multi-ISPs and multi-P2P applications using ALTO mechanism. On the other hand, we found several problems in ISP operations to adapt the mechanism. One is the granularity of network information from council members. Among inter-ISP area, it is relatively easy to treat information for public purpose using BGP full route. On the other hand, among intra-ISP area, it may be difficult to disclose private information of each ISP. [I-D.kiesel-alto-h12] propose some modification for ALTO protocol in order to hide ISP information. We propose hierarchical structures. From the viewpoint of cooperation between ISPs, fine-grained information is not necessarily required and moreover it is difficult to exchange the fine-grained information between ISPs. Considering this situation, the authors use only coarse-grained information to control backbone traffic in this experiments, though demand of controlling traffic within an ISP using fine-grained information may arise in future. Therefore it led us that introducing hierarchical structure into ALTO is necessary to cope with both situations. Actually, to adapt a hierarchical control mechanism which include the following two steps will be useful.
In the experiments, there were two difficulties as follows:
From these experiences, the authors consider that clarifying requirements about measurement mechanisms for P2P applications are necessary also in ALTO.
This document does not propose any kind of protocol, practice or standard.
No need to describe any request regarding number assignment.
Thanks to strong support by MIC (Ministry of Internal Affairs and Communications of Japanese government), the council was established. These experiments were performed under cooperation among P2P Network Experiment Council members, and DREAMBOAT co.,ltd., Bitmedia Inc., Utagoe. Inc. and Toyama IX have especially supported analyses of the experiments. The authors appreciate Tohru Asami, Hiroshi Esaki and Tatsuya Yamshita for their constructive comments.
And the authors would like to thank Martin Stiemerling, Stefano Previdi and Vijay K.Gurbani for the comments on this document.