Signaling Solution for HP-WAN

Introduction Data-intensive applications, such as scientific research, academia, and education, always demand high-speed data transmission over WANs, as discussed in and other applications in public networks as per . HP-WAN applications mainly focus on job-based timely transmission over long-distance WANs. High throughput with efficient use of capacity is the fundamental requirement for HP-WAN. However, HP-WAN faces challenges such as poor convergence speed, long feedback loop, unscheduled traffic and multi-flow concurrent transmission as per . defines a framework to enable the host-and-network collaboration for high-speed and high-throughput data transmission. This document proposes a technical solution for the host-network collaboration signaling to enhance the congestion control in HP-WAN. It also describes the RSVP extensions as an instantiation of the signaling solution.

Conventions used in this document

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology This document uses the terms defined in and .

Job-based Host-network Collaboration The requirement for Host-network collaboration originates from the transmission demands of intelligent computing jobs over WANs. The network can establish the tunnels based on hosts' demands, reserve resources, and ensure high-throughput transmission of jobs' workloads within their completion deadlines. The definitions of Job and Task are as follows.

Job: An intelligent computing service and a single Job is decomposed into multiple tasks for parallel transmission over the WAN.
Task: A peer-to-peer (P2P) network transmission task triggered by intelligent computing or data transmission. Each task corresponds to a long-lived traffic flow.

For HP-WAN, the host-network collaboration can be divided into two phases: job-based tunnel establishment and task-based resource scheduling. Job-based tunnel establishment indicates that the per-job traffic engineering (TE) tunnel setup is triggered by the intelligent computing Job from the hosts. For example, a Job scheduler triggers one or more hosts to initiate tunnel establishment toward the network based on job's requirements. Task-based resource scheduling refers to the dynamic tunnel resource utilization for concurrent tasks based on rate control.

The Rate Negotiation Policy for Host-network Collaboration As per , HP-WAN framework proposes to enhance the congestion control for the host and network collaboration especially the rate negotiation. It is required to guarantee the completion time of the traffic based on different rate policies. The host-network collaboration includes:

The host needs to send job-based requests to the network to negotiate the sending rate before data transmission;
The network needs to schedule the requests and perform the admission control based on available capacity. The network performs dynamic resource reservation across different time frames defined by quota. For example, the nodes need to subtract the resource quota within the time frames, but not allocate it while traffic in the network is still sharing the resources.
The subtraction result will be viewed as the resource constraints for admission control. The request will be accepted when the rate-based resource quota reservation succeeds; otherwise, it will be rejected.
After the acknowledgement of the rate negotiation, the client could transmit the job-based flows at a sending rate based on the rate policy.

Minimum Rate Negotiation As per , the host will request the network to provide the minimum resource guarantee in minimum rate negotiation policy. The network implements the admission control based on the minimum resource quota reservation at the nodes along the path. After the acknowledgement of the minimum rate negotiation, the client could transmit the job-based flows at a sending rate not less than the minimum rate. The minimum rate can be computed as follows:

Min-rate = Flowsize/(CompletionTime-StartTime)

For example, the client requests to transfer the job with 10G data volume from 10s to 20s. The minimum rate will be 10G/(20s-10s)=1Gbps. The network will subtract 1G bandwidth quota from 10s to 20s and the admission of this job is accepted. The client could transfer this job with rate no less than 1Gbps.

Maximum Rate Negotiation As per , the host will request the network to provide an upper limit for resource guarantee in maximum rate negotiation policy. The network implements the admission control based on the maximum resource scheduling at the edge node of the path. After the acknowledgement of the maximum rate negotiation, the client could transmit the job-based flows at a sending rate not greater than the maximum rate. The maximum rate of a flow on a specific link is related to the link bandwidth and the number of aggregated traffic flows. When more flows are aggregated, the maximum rate of each individual flow decreases. Conversely, with fewer aggregated flows, each flow can achieve a higher maximum rate, ensuring the buffer does not overflow and congestion is mitigated. Multiple hops along the path could calculate a set of maximum rates, the negotiated maximum rate for a flow transmitting along the path is the minimum value within the maximum rates. The maximum rate can be computed as follows:

Max-rate = a*Bandwidth/FlowNumber

a is an expansion coefficient which is set based on network buffer information. For example, if the the output bandwidth of the edge node is 10G, a is 1.5, and two flows aggregate through this node, the maximum rate will be 1.5*10G/2=7.5Gbps. The client could transfer this job with rate no greater than 7.5Gbps within the time frame.

Constant Rate Negotiation As per , the host will request the network to provide constant resource reservation for high-speed data to guarantee optimal rate transmission in constant rate negotiation policy. The network implements the admission control based on the constant resource quota reservation at the nodes along the path. After the acknowledgement of the constant rate negotiation, the client could transmit the job-based flows at a sending rate according to the multiple negotiated constant rates within corresponding time frames. The constant rates can be computed as the minimum rate or the controller could perform high-level resources planning and allocate optimal rates for the time frames with multiple intervals. For example, the constant rates could be like {(10s~20s,10Gbps), (20s~30s,2Gbps)}.

Host-network Collaboration signaling As per , the negotiated rate-based congestion control can be enabled through host-network collaboration signaling. There are several options for the signaling procedures as described in the following sections.

Stitching signaling The host-network collaboration signaling can be implemented as stitching signaling between host-network and in-network. The P2P signaling (e.g., GRASP and RSVP) can be provisioned between the client and the network edge node. Dynamic resource
quota reservation signaling along network nodes can be achieved within the network. The workflow is shown in Figure 1.

The requests of jobs will be signaled through P2P signaling from the client to the edge node carrying the job-based requirements.
The edge node and perform admission control while triggering the network to establish the TE tunnel for the job. The edge node will send the success or failure for the acknowledgement result.
The requests of scheduled traffic will be signaled through P2P signaling from the client to the edge node carrying the traffic requirements of tasks.
The edge node will configure the rate policy, compute the negotiated rate, and initiate the rate control and resource scheduling based on the resources of the established TE tunnel and the edge nodes. It will reply with the rate information for the task when the resource quota is successfully reserved. It will also notify the client when the resource quota is updated.
The client will notify the edge node when the data transmission is completed. The network resources will be released and the acknowledgement is cancelled.

|*Admission control | |<--------------------|<...*Job-based traffic engineering...>| |JobAcknowledgement | | |(success or failure) | | | | | |TaskRequest | | |(traffic pattern) | | |-------------------->|*Rate negotiation | |TaskResponse |*Rate control and resource scheduling | |(negotiated rate) |<...*Reserve the resource quota......>| |<--------------------| | |TaskUpdate | | |(negotiated rate) |<...*Update the resource quota.......>| |<--------------------| | | | | |JobNotify(completion)| | |-------------------->| | |JobCancel(cancel) |<...*Release the network resources...>| |<--------------------| | | | | V V V |<------------------->|<....................................>| P2P signaling between client Rate-based traffic engineering and network edge node along network nodes Figure 1 Stitching signaling ]]>

Overlay signaling The host-network collaboration signaling can be implemented as overlay signaling. It can be end-to-end signaling (e.g., RSVP) provisioned along the path from client, network nodes, and server. The rate-based traffic engineering along network nodes will be triggered as underlay signaling. The workflow is shown in Figure 2. |*Admission control | | | |<...*Job-based traffic engineering...>| | |JobAcknowledgement | |JobRequest | |(success or failure) | |<---------------->| |<--------------------| |JobAcknowledgement| | | | | |TaskRequest | | | |(traffic pattern) | | | |-------------------->|*Rate negotiation | | |TaskResponse |*Rate control and resource scheduling | | |(negotiated rate) |<...*Reserve the resource quota......>|<---------------->| |TaskResponse | |<--------------------| |TaskRequest | |TaskUpdate | | | |(negotiated rate) |<...*Update the resource quota.......>| | |<--------------------| | | | | | | |JobNotify(completion)| | | |-------------------->| | JobNotify | |JobCancel(cancel) |<...*Release the network resources...>|<---------------->| |<--------------------| | JobCancel | | | | | V V V V |<....................................>| Rate-based traffic engineering along network nodes |<------------------------------------------------------------------------------>| Overlay signaling along the client, edge nodes and server Figure 2 Overlay signaling ]]>

Instantiation with RSVP Extensions RSVP protocols can be instantiated to implement the host-network collaboration signaling. This document proposes the RSVP extensions to achieve the provisioning of the job-based request and acknowledgement, task-based request and response, task-based update and job-based notification and cancellation. This document focuses on the host-network collaboration signaling including the P2P signaling between client and network edge node and the overlay signaling along the client, edge nodes, and server. The procedures and extensions for rate-based traffic engineering within the network will be in a separate document as per .

Objects Extensions

Job Object This document proposes the Job Object to carry the job-based parameters and requirements, such as job ID, job size, job type, job models and other parameters. The format of Job Object is shown in Figure 3. Job ID: 32 bits, indicates the identifier of the job. Job Type: 32 bits, indicates the type of the job. Job Size: 32 bits, indicates the data size of the job. Job Model: variable length, indicates the job model parameters such as model type, model complexity and so on.

Traffic Pattern Object This document proposes the Traffic Pattern Object to carry the traffic pattern and task-based requirements, such as traffic characteristic parameters. The format of Traffic Pattern Object is shown in Figure 4. Job ID: 32 bits, indicates the identifier of the job. Task ID: 32 bits, indicates the identifier of the task. Start Time: 32 bits, indicates the time when it starts to transmit the flows. Completion Time: 32 bits, indicates the deadline time when it requires to complete the transmission. Data Volume: 32 bits, indicates the data volume of the job which needs to transfer.

Rate Object This document proposes the Rate Object to carry the negotiated rate which is acknowledged. The format of Rate Object is shown in Figure 5. Job ID: 32 bits, indicates the identifier of the Job and the task. Task ID: 32 bits, indicates the identifier of the task. Rate Policy: 32 bits, indicates the type of the rate policy such as minimum, maximum, optimal rate policy and range of minimum and maximum. optional TLVs: variable length and multiple TLVs can be carried based on the value of rate policy. When the optimal rate policy is selected, Time-Frame TLV is carried and shown in Figure 6. Multiple Time-Frame TLVs can be carried when multiple intervals are computed with some particular rates. Rate: 32 bits, indicates the optimal rate for the job or task to transmit flows. Time Unit Type: 32 bits, indicates the type of time unit, including second, microsecond, millisecond and minute. Start Time: 32 bits, indicates the start time of the time frame. End Time: 32 bits, indicates the end time of the time frame. When the minimum rate policy is selected, Min-Rate TLV is carried and shown in Figure 7. Min-Rate/Min-Bandwidth: 32 bits, indicates the minimum rate or bandwidth for the job. When the maximum rate policy is selected, Max-Rate TLV is carried and shown in Figure 8. Max-Rate/Max-Bandwidth: 32 bits, indicates the maximum rate or bandwidth for the job. When the range of minimum and maximum rate policy is selected, Min-Rate TLV and Max-Rate TLV should be both carried.

Message Extensions This document proposes new messages or reuses the existing messages to achieve the signaling communication. The format of the new messages is shown in Figure 9.

JobRequest Message The JobRequest message with new Message Type (TBD1) can be used for job-based request. The client can send the JobRequest message and the Job Object must be included. It will trigger the edge node to perform the job-based traffic engineering within the network such as establishing the TE tunnel and reserving the TE resources.

JobAcknowledgement Message The JobAcknowledgement message with new Message Type (TBD2) can be used for job-based acknowledgement. The edge node can send JobAcknowledgement message with successful result when the job is acknowledged.

TaskRequest Message The TaskRequest message with new Message Type (TBD3) can be used for task-based request. The client can send the TaskRequest message and the Traffic Pattern Object must be included. It will trigger the edge node to perform the rate control and resource scheduling based on the TE tunnel.

TaskResponse Message The TaskResponse message with new Message Type (TBD4) can be used for task-based response. The edge node can send the TaskResponse message conveying success when the network reserves the resource quota for the task. The Rate Object must be included to control the rate of the traffic.

TaskUpdate Message The TaskUpdate message with new Message Type (TBD5) can be used for task-based update with the Rate Object included with new rate related parameters.

JobNotify Message The JobNotify message with new Message Type (TBD6) can be used for job-based notification to indicate the job transmission is completed.

JobCancel Message The JobCancel message with new Message Type (TBD7) can be used for job-based cancellation to indicate the acknowledgement is cancelled.

Considerations for the Centralized Solution As per , the host and the network could interact and negotiate the sending rate due to the predictability of jobs. The client will send the request with the job parameters and traffic patterns of high-speed flows and the network will reserve the corresponding resource quota and perform the admission control based on the capacity of network nodes. If the network deployment is Software-Defined Network (SDN) centralized architecture, the controller will perform resource quota reservation and admission control. For instance, for the SDN for End-to-end Networked Science at Exascale (SENSE) system in Research and Education (R&E) networks, the orchestrator and resource Manager (RM) have the capability of hierarchical planning and resource reservation in the network. The orchestrator communicates the requests from applications and interacts with the RM for resource reservation.

Security Considerations The security considerations specified in and apply to this document. In addition, and provide useful guidance on RSVP security mechanisms. It is also required to create the trusted relationship between the clients and the network based on authentication (e.g., and ) and authorization (e.g.,).

IANA Considerations TBA.