Network Working Group J. Fu
Internet-Draft X. Zhu
Intended status: Informational M. Ramalho
Expires: January 7, 2016 W. Tan
Cisco Systems
July 6, 2015

Evaluation Test Cases for Interactive Real-Time Media over Wi-Fi Networks
draft-fu-rmcat-wifi-test-case-01

Abstract

An increasing proportion of multimedia communication applications, including real-time interactive voice and video, are transported over Wi-Fi networks (i.e., wireless local area networks following IEEE 802.11 standards) today. It is therefore important to evaluate candidate congestion control schemes designed in the RMCAT Working Group over test cases that include Wi-Fi access links. This draft serves such a purpose, and is complementary to [I-D.ietf-rmcat-eval-test] and [I-D.draft-sarker-rmcat-cellular-eval-test-cases]

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on January 7, 2016.

Copyright Notice

Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1. Introduction

Given the prevalence of Internet access links over Wi-Fi, it is important to evaluate candidate RMCAT congestion control solutions over Wi-Fi test cases. Such evaluations should also highlight the inherent different characteristics of Wi-Fi networks in contrast to Wired networks:

As we can see here, presence of Wi-Fi network in different different network topologies and traffic arrival can exert different impact on the network performance in terms of video transport rate, packet loss and delay that, in turn, effect end-to-end real-time multimedia congestion control.

Currently, the most widely used IEEE 802.11 standards are 802.11g and 802.11n. The industry is moving towards 802.11ac and, potentially, 802.11ad. IEEE 802.11b is legency standard, and 802.11a has not been widely adopted. Throughout this draft, unless otherwise mentioned, test cases are described using 802.11g mostly due to its wide availability both in test equipments and network simulation platform. Whenever possible, it is recommended to extend some of the experiments to 802.11ac so as to reflect a more mordent Wi-Fi network setting.

Since Wi-Fi network normally connects to a wired infrastructure, either the wired network or the Wi-Fi network could be the bottleneck. In the following section, we describe basic test cases for both scenarios separately.

2. Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described RFC2119 [RFC2119].

3. Test Scenarios with Wired Bottlenecks

The test scenarios below are intended to mimic the set up of video conferencing over Wi-Fi connections from the home. Typically, the Wi-Fi home network is not congested and the bottleneck is present over the wired home access link. Although it is expected that test evaluation results from this section are similar to those from the all-wired test cases (draft-sarker-rmcat-eval-test), it is worthwhile to run through these tests as sanity checks.

3.1. Single RMCAT Flow over Wired Bottleneck

This test case is designed to measure the responsiveness of the candidate algorithm when the Wi-Fi hop of the connection is uncongested.

Figure 1 illustrates topology of this test.


                           uplink
                      +-------------->+
    +----+          +----+          +----+         +----+ 
    | S  |))))))))))| AP |==========|  B |=========|  R | 
    +----+          +----+          +----+         +----+

Figure 1: Single Flow over Wired Bottleneck

Testbed attributes:

Expected behavior: the candidate algorithm is expected to detect the path capacity constraint, converges to bottleneck link's capacity and adapt the flow to avoid unwanted oscillation when the sending bit rate is approaching the bottleneck link's capacity. Oscillations occur when the media flow(s) attempts to reach its maximum bit rate, overshoots the usage of the available bottleneck capacity, to rectify it reduces the bit rate and starts to ramp up again.

3.2. Bidirectional RMCAT Flow over Wired Bottleneck

This test case is designed to evaluate performance of the candidate algorithm when lack of enough feedback.


   +----+                                         +----+ 
   | R1 |))))))                             /=====| S1 |
   +----+      ))         uplink           //     +----+ 
                ))    +-------------->+   //                         
    +----+          +----+          +----+        +----+ 
    | S2 |))))))))))| AP |==========|  B |========| R2 | 
    +----+          +----+          +----+        +----+
                      +<--------------+
                          downlink

Figure 2: One Bi-directional Flow over Wired Bottleneck

Testbed attributes:

  • Test duration: 100s
  • Path characteristics:
    • uplink capacity: 1Mbps
    • One-Way propagation delay: 50ms.
    • Maximum end-to-end jitter: 30ms
    • Bottleneck queue type: Drop tail.
    • Bottleneck queue size: 300ms.
    • Path loss ratio: 0%.

  • Application characteristics:
    • Media Traffic:
      • Media type: Video
      • Media direction: forward and backward.
      • Number of media sources: Two (2)
      • Media timeline:
        • Start time: 0s.
        • End time: 99s.
    • Competing traffic:
      • Number and Types of sources : zero (0)

Expected behavior: It is expected that the candidate algorithms is able to cope with the lack/noise of feedback information and adapt to minimize the performance degradation of media flows in the forward channel.

3.3. RMCAT Flow Competing with Long TCP over Wired Bottleneck

This test case is designed to measure the performance of the candidate algorithm when lack of enough feedback.


   +----+                                         +----+ 
   | S1 |))))))                             /=====| R1 |
   +----+      ))         uplink           //     +----+ 
                ))    +-------------->+   //                         
 +-------+          +----+          +----+        +-------+ 
 | S_tcp |))))))))))| AP |==========|  B |========| R_tcp | 
 +-------+          +----+          +----+        +-------+
                      +<--------------+
                          downlink

Figure 3: RMCAT vs. TCP over Wired Bottleneck

Testbed attributes:

  • Test duratiion: 100s
  • Path characteristics:
    • uplink capacity: 1Mbps
    • One-Way propagation delay: 50ms.
    • Maximum end-to-end jitter: 30ms
    • Bottleneck queue type: Drop tail.
    • Bottleneck queue size: 300ms.
    • Path loss ratio: 0%.
  • Application-related:
    • Media Traffic:
      • Media type: Video
      • Media direction: forward.
      • Number of media sources: One (1)
      • Media timeline:
        • Start time: 0s.
        • End time: 99s.
    • Competing traffic:
      • Types of sources : long-lived TCP
      • Number of sources: One (1)
      • Traffic direction : forward
      • Congestion control: Default TCP congestion control [TBD].
      • Traffic timeline:
        • Start time: 0s.
        • End time: 119s.

Expected behavior: the candidate algorithm should be able to avoid congestion collapse, and get fair share of the bandwidth. In the worst case, the media stream will fall to the minimum media bit rate.

4. Bottleneck over Wireless Network

These test cases assume that the wired portion along the media path are well-provisioned. The bottleneck is in the Wi-Fi network over wireless. This is to mimic the enterprise/coffee-house scenarios.

4.1. Adaptive rate selection with single RMCAT flow

Since morden IEEE 802.11 standards supports far higher data rates than the maximum requirements of individual RMCAT flows, in this test the legacy standard 802.11b is chosen to test the single RMCAT flow case. 802.11b Adaptive rate selection can operate at 11 Mbps in terms of PHY-layer transmission rate, and falls back to 5.5 Mbps, 2 Mbps, and 1 Mbps when the wireless client moves away from the access point.

[Editor's Note: we may want to move this section to Section 5.4 instead.]


           uplink
      +-------------->+
    +----+          +----+          +----+         +----+ 
    | S  |))))))))))| AP |==========|  B |=========|  R | 
    +----+          +----+          +----+         +----+

Figure 4: One RMCAT Flow over Wireless Bottleneck

Testbed attributes:

  • Test duratiion: 100s
  • Path characteristics:
    • Wired path capacity: 100Mbps
    • Wi-Fi PHY Rate: 1Mbps (PHY rate)
    • One-Way propagation delay: 50ms.
    • Maximum end-to-end jitter: 30ms
    • Bottleneck queue type: Drop tail.
    • Bottleneck queue size: 300ms.
    • Path loss ratio: 0%.
  • Application characteristics:
    • Media Traffic:
      • Media type: Video
      • Media direction: forward and backward.
      • Number of media sources: One (1)
      • Media timeline:
        • Start time: 0s.
        • End time: 99s.
    • Competing traffic:
      • Number and Types of sources : zero (0)
  • Test Specific Information:
    • This test will change the distance between station and AP (need some experiment), and incur the adaptive rate selection variation as listed in Figure 5. 

   +---------------------+----------------+------------+----------------+
   | Variation pattern   | Path direction | Start time | PHY-layer rate |
   | index               |                |            |                |
   +---------------------+----------------+------------+----------------+
   | One                 | Forward        | 0s         |  5.5 Mbps      |
   | Two                 | Forward        | 40s        |  2 Mbps        |
   | Three               | Forward        | 60s        |  1 Mbps        |
   | Four                | Forward        | 80s        |  2 Mbps        |
   +---------------------+----------------+------------+---------------+

Figure 5: Adaptive rate variation pattern for uplink direction

Expected behavior: The rate adaptation algorithm run at application level should follow the adaptation in 802.11 mac layer.

4.2. Multiple RMCAT Flows Sharing the Wireless Downlink

This test case is for studying the impact of contention on competing RMCAT flows. Specifications for IEEE 802.11g with a physical-layer transmission rate of 54 Mbps is chosen. Not that retransmission and MAC-layer headers and control packets may be sent at a lower link speed. The total application-layer throughput (reasonable distance, low interference and small number of contention stations) for 802.11g is around 20 Mbps. Consequently, a total of 16 RMCAT flows are needed for saturating the wireless interface in this experiment.