Internet Engineering Task Force | J. Hadi Salim |
Internet-Draft | Mojatatu Networks |
Updates: 7121,5810 (if approved) | July 30, 2014 |
Intended status: Standards Track | |
Expires: January 31, 2015 |
ForCES Protocol Extensions
draft-ietf-forces-protoextension-04
Experience in implementing and deploying ForCES architecture has demonstrated need for a few small extensions both to ease programmability and to improve wire efficiency of some transactions. This documents updates both RFC 5810 and RFC 7121 semantics to achieve that end goal.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 31, 2015.
Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Experience in implementing and deploying ForCES architecture has demonstrated need for a few small extensions both to ease programmability and to improve wire efficiency of some transactions. This document describes a few extensions to the ForCES Protocol Specification [RFC5810] semantics to achieve that end goal.
This document describes and justifies the need for 2 small extensions which are backward compatible. The document also clarifies details of how dumping of a large table residing on an FE is achieved. To summarize:
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
This document reiterates the terminology defined in several ForCES documents [RFC3746], [RFC5810], [RFC5811], and [RFC5812] for the sake of contexual clarity.
In this section we present sample use cases to illustrate each challenge being addressed.
Consider, for the sake of illustration, an FE table with 1 million reasonably sized table rows which are sparsely populated. Assume, again for the sake of illustration, that there are 2000 table rows sparsely populated between the row indices 23-10023.
Implementation experience has shown that existing approaches for retrieving or deleting a sizeable number of table rows is at the programmatically level (from an application point of view) unfriendly, tedious, and abusive of both compute and bandwidth resources.
By Definition, ForCES GET and DEL requests sent from a controller (or control app) are prepended with a path to a component and sent to the FE. In the case of indexed tables, the component path can either point to a table or a table row index.
As an example, a control application attempting to retrieve the first 2000 table rows appearing between row indices 23 and 10023 can achieve its goal in one of:
As argued, while the above options exist - all are tedious.
[RFC5810] has defined a generic set of error codes that are to be returned to the CE from an FE. Deployment experience has shown that it would be useful to have more fine grained error codes. As an example, the error code E_NOT_SUPPORTED could be mapped to many FE error source possibilities that need to be then interpreted by the caller based on some understanding of the nature of the sent request. This makes debugging more time consuming.
This section describes proposals to update the protocol for issues discussed in Section 2
We define a new TLV, TABLERANGE-TLV (type ID 0x117) that will be associated with the PATH-DATA TLV in the same manner the KEYINFO-TLV is.
+---------------------+---------------------+ | Type (0x117) | Length | +---------------------+---------------------+ | Start Index | +-------------------------------------------+ | End Index | +-------------------------------------------+
Figure 1: ForCES table range request Layout
Figure 1 shows how this new TLV is constructed.
OPER = GET PATH-DATA: flags = F_SELTABRANGE, IDCount = 2, IDs = {1,6} TABLERANGE-TLV content = {11,23}
Figure 2: ForCES table range request
Figure 2 illustrates a GET request for a range of rows 11 to 23 of a table with component path of "1/6".
Path flag of F_SELTABRANGE (0x2 i.e bit 1, where bit 0 is F_SELKEY as defined in RFC 5810) MUST be set to indicate the presence of the TABLERANGE-TLV. The pathflag bit F_SELTABRANGE can only be used in a GET or DEL and is mutually exclusive with F_SELKEY. The FE MUST enforce the path flag constraints and ensure that the selected path belongs to a defined indexed table component. Any violation of these constraints MUST be rejected with an error code of E_INVALID_TFLAGS with a description of what the problem is when using extended error reporting (refer to Section 3.2).
The TABLERANGE-TLV contents constitute:
The response for a table range query will either be:
We define several things:
EXTENDEDRESULT-TLV Result Value is 32 bits and is a superset of RFC 5810 Result TLV Result Value. The new version code space is 32 bits as opposed to the RFC 5810 code size of 8 bits. The first 8 bit values are common to both old
Code | Mnemonic | Details |
---|---|---|
0x18 | E_TIMED_OUT | A time out occured while processing the message |
0x19 | E_INVALID_TFLAGS | Invalid table flags |
0x1A | E_INVALID_OP | Requested operation is invalid |
0x1B | E_CONGEST_NT | Node Congestion notification |
0x1C | E_COMPONENT_NOT_A_TABLE | Component not a table |
0x1D | E_PERM | Operation not permitted |
0x1E | E_BUSY | System is Busy |
0x1F | E_EMPTY | Table is empty |
0x20 | E_UNKNOWN | A generic catch all error code. Carries a string to further extrapolate what the error implies. |
Codes 0x100-0x200 are reserved for use as private codes. Since these are freely available it is expected that the FE and CE side implementations will both understand/interpret the semantics of any used codes and avoid any conflicts.
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = EXTENDEDRESULT-TLV | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Result Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Optional Cause content | . . | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3: EXTENDEDRESULT-TLV
To support backward compatibility, we update and the FEPO LFB Appendix A version to 1.2. We also add a new component ID 16 (named EResultAdmin) and a capability Component ID 32 (named EResultCapab).
An FE will advertise its capability to support extended TLVs via the EResultCapab table. When an FE is capable of responding with both extended results and older result TLVs, it will have two table rows one for each supported value. By default an FE capable of supporting both modes will assume the lowest common denominator i.e EResultAdmin will be EResultNotSupported; and will issue responses using RESULT-TLVs. It should be noted an FE advertising FEPO version 1.2 MUST support EXTENDEDRESULT-TLVs at minimum.
On an FE which supports both RESULT-TLVs and EXTENDEDRESULT-TLVs, a master CE can turn on support for extended results by setting the value to 2 in which case the FE MUST switch over to sending only EXTENDEDRESULT-TLVs. Likewise a master CE can turn off extended result responses by writting a 1 to the EResultAdmin. An FE that does not support one mode or other MUST reject setting of EResultAdmin to a value it does not support by responding with an error code of E_NOT_SUPPORTED.
Imagine a GET request to a path that is a table i.e a table dump. Such a request is sent to the FE with a specific correlator, say X. Imagine this table to have a large number of entries at the FE. For the sake of illustration, lets say millions of rows. This requires that the FE delivers the response over multiple messages, all using the same correlator X.
The protocol document [RFC5810] does not adequately describe how a GET response to such a large message is delivered. The text in this section clarifies. We limit the discussion to a table object only.
Implementation experience of dumping large tables indicates we can use the transaction flags to indicate that a GET response is the beginning, middle or end of a multi-part message. In other words we mirror the effect of an atomic transaction sent by a CE to an FE.
CE PL FE PL | | | (0) Query, Path-to-a-large-table, OP=GET | |----------------------------------------------------->| | correlattor = X | | | | (1) Query-Response, SOT,AT, OP=GET-RESPONSE, DATA | |<-----------------------------------------------------| | correlattor = X | | DATA TLV (SPARSE/FULL) | | | | (2) Query-Response, MOT,AT, OP=GET-RESPONSE, DATA | |<-----------------------------------------------------| | correlattor = X | | DATA TLV (SPARSE/FULL) | | | | (3) Query-Response, MOT,AT, OP=GET-RESPONSE, DATA | |<-----------------------------------------------------| | correlattor = X | | DATA TLV (SPARSE/FULL) | . . . . . . . . | | | (N) Query-Response, MOT,AT, OP=GET-RESPONSE, DATA | |<-----------------------------------------------------| | correlattor = X | | DATA TLV (SPARSE/FULL) | | | | (N) Query-Response, EOT,AT, OP=GET-RESPONSE | |<-----------------------------------------------------| | correlattor = X | | RESULT TLV (SUCCESS) | | |
Figure 4: EXTENDEDRESULT-TLV
The last message which carries the EOT flag to go the CE MUST NOT carry any data. This allows us to mirror ForCES 2PC messaging [RFC5810] where the last message is an empty commit message. GET response will carry a result code TLV in such a case.
The author would like to thank Evangelos Haleplidis and Joel Halpern for discussions that made this document better. Adrian Farrel did an excellent AD review of the document which improved the quality of this document.
This document registers two new top Level TLVs and two new path flags and updates an IANA registered FE Protocol object Logical Functional Block (LFB).
The Appendix A defines an update to the FE Protocol Object LFB to version 1.2. XXX: comment to IANA: The IANA registry https://www.iana.org/assignments/forces/forces.xml sub-registy "Logical Functional Block (LFB) Class Names and Class Identifiers" will need to be updated for FE Protocol Object LFB version from 1.1 to 1.2 and this document reflected in the reference column.
XXX: comments to IANA - updates required to the "TLV types" subregistry for the TLVs below.
The following new TLVs are defined:
XXX: Comment to IANA, section below affects subregistry "RESULT-TLV Result Values"
The Defined RESULT-TLV Result Values are changed:
XXX: Note to IANA - codes 0x18-0x20 need approval of the designated expert (In this case Joel Halpern since the author is the other expert).
A a new sub-registry for EXTENDEDRESULT-TLV Result Values needs to be created. The codes 0x00-0xff are mirrored from the RESULT-TLV Result Values sub-registry and must not be allocated. The codes 0x100-0x200 are reserved for private use as described earlier and the codes 0x200-0xffffffff are reserved for future use; these codes will be allocated on First Come First Served basis and require specification as well approval of an expert review.
The security considerations that have been described in the ForCES protocol [RFC5810] apply to this document as well.
[RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. |
[RFC5810] | Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, W., Dong, L., Gopal, R. and J. Halpern, "Forwarding and Control Element Separation (ForCES) Protocol Specification", RFC 5810, March 2010. |
[RFC5811] | Hadi Salim, J. and K. Ogawa, "SCTP-Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation (ForCES) Protocol", RFC 5811, March 2010. |
[RFC5812] | Halpern, J. and J. Hadi Salim, "Forwarding and Control Element Separation (ForCES) Forwarding Element Model", RFC 5812, March 2010. |
[RFC7121] | Ogawa, K., Wang, W., Haleplidis, E. and J. Hadi Salim, "High Availability within a Forwarding and Control Element Separation (ForCES) Network Element", RFC 7121, February 2014. |
[RFC3746] | Yang, L., Dantu, R., Anderson, T. and R. Gopal, "Forwarding and Control Element Separation (ForCES) Framework", RFC 3746, April 2004. |
<LFBLibrary xmlns="urn:ietf:params:xml:ns:forces:lfbmodel:1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="lfb-schema.xsd" provides="FEPO"> <!-- XXX --> <dataTypeDefs> <dataTypeDef> <name>CEHBPolicyValues</name> <synopsis> The possible values of CE heartbeat policy </synopsis> <atomic> <baseType>uchar</baseType> <specialValues> <specialValue value="0"> <name>CEHBPolicy0</name> <synopsis> The CE will send heartbeats to the FE every CEHDI timeout if no other messages have been sent since. </synopsis> </specialValue> <specialValue value="1"> <name>CEHBPolicy1</name> <synopsis> The CE will not send heartbeats to the FE </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> <dataTypeDef> <name>FEHBPolicyValues</name> <synopsis> The possible values of FE heartbeat policy </synopsis> <atomic> <baseType>uchar</baseType> <specialValues> <specialValue value="0"> <name>FEHBPolicy0</name> <synopsis> The FE will not generate any heartbeats to the CE </synopsis> </specialValue> <specialValue value="1"> <name>FEHBPolicy1</name> <synopsis> The FE generates heartbeats to the CE every FEHI if no other messages have been sent to the CE. </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> <dataTypeDef> <name>FERestartPolicyValues</name> <synopsis> The possible values of FE restart policy </synopsis> <atomic> <baseType>uchar</baseType> <specialValues> <specialValue value="0"> <name>FERestartPolicy0</name> <synopsis> The FE restart restats its state from scratch </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> <dataTypeDef> <name>HAModeValues</name> <synopsis> The possible values of HA modes </synopsis> <atomic> <baseType>uchar</baseType> <specialValues> <specialValue value="0"> <name>NoHA</name> <synopsis> The FE is not running in HA mode </synopsis> </specialValue> <specialValue value="1"> <name>ColdStandby</name> <synopsis> The FE is running in HA mode cold Standby </synopsis> </specialValue> <specialValue value="2"> <name>HotStandby</name> <synopsis> The FE is running in HA mode hot Standby </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> <dataTypeDef> <name>CEFailoverPolicyValues</name> <synopsis> The possible values of CE failover policy </synopsis> <atomic> <baseType>uchar</baseType> <specialValues> <specialValue value="0"> <name>CEFailoverPolicy0</name> <synopsis> The FE should stop functioning immediate and transition to the FE OperDisable state </synopsis> </specialValue> <specialValue value="1"> <name>CEFailoverPolicy1</name> <synopsis> The FE should continue forwarding even without an associated CE for CEFTI. The FE goes to FE OperDisable when the CEFTI expires and no association. Requires graceful restart support. </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> <dataTypeDef> <name>FEHACapab</name> <synopsis> The supported HA features </synopsis> <atomic> <baseType>uchar</baseType> <specialValues> <specialValue value="0"> <name>GracefullRestart</name> <synopsis> The FE supports Graceful Restart </synopsis> </specialValue> <specialValue value="1"> <name>HA</name> <synopsis> The FE supports HA </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> <dataTypeDef> <name>CEStatusType</name> <synopsis>Status values. Status for each CE</synopsis> <atomic> <baseType>uchar</baseType> <specialValues> <specialValue value="0"> <name>Disconnected</name> <synopsis>No connection attempt with the CE yet </synopsis> </specialValue> <specialValue value="1"> <name>Connected</name> <synopsis>The FE connection with the CE at the TML has been completed </synopsis> </specialValue> <specialValue value="2"> <name>Associated</name> <synopsis>The FE has associated with the CE </synopsis> </specialValue> <specialValue value="3"> <name>IsMaster</name> <synopsis>The CE is the master (and associated) </synopsis> </specialValue> <specialValue value="4"> <name>LostConnection</name> <synopsis>The FE was associated with the CE but lost the connection </synopsis> </specialValue> <specialValue value="5"> <name>Unreachable</name> <synopsis>The CE is deemed as unreachable by the FE </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> <dataTypeDef> <name>StatisticsType</name> <synopsis>Statistics Definition</synopsis> <struct> <component componentID="1"> <name>RecvPackets</name> <synopsis>Packets Received</synopsis> <typeRef>uint64</typeRef> </component> <component componentID="2"> <name>RecvErrPackets</name> <synopsis>Packets Received from CE with errors </synopsis> <typeRef>uint64</typeRef> </component> <component componentID="3"> <name>RecvBytes</name> <synopsis>Bytes Received from CE</synopsis> <typeRef>uint64</typeRef> </component> <component componentID="4"> <name>RecvErrBytes</name> <synopsis>Bytes Received from CE in Error</synopsis> <typeRef>uint64</typeRef> </component> <component componentID="5"> <name>TxmitPackets</name> <synopsis>Packets Transmitted to CE</synopsis> <typeRef>uint64</typeRef> </component> <component componentID="6"> <name>TxmitErrPackets</name> <synopsis> Packets Transmitted to CE that incurred errors </synopsis> <typeRef>uint64</typeRef> </component> <component componentID="7"> <name>TxmitBytes</name> <synopsis>Bytes Transmitted to CE</synopsis> <typeRef>uint64</typeRef> </component> <component componentID="8"> <name>TxmitErrBytes</name> <synopsis>Bytes Transmitted to CE incurring errors </synopsis> <typeRef>uint64</typeRef> </component> </struct> </dataTypeDef> <dataTypeDef> <name>AllCEType</name> <synopsis>Table Type for AllCE component</synopsis> <struct> <component componentID="1"> <name>CEID</name> <synopsis>ID of the CE</synopsis> <typeRef>uint32</typeRef> </component> <component componentID="2"> <name>Statistics</name> <synopsis>Statistics per CE</synopsis> <typeRef>StatisticsType</typeRef> </component> <component componentID="3"> <name>CEStatus</name> <synopsis>Status of the CE</synopsis> <typeRef>CEStatusType</typeRef> </component> </struct> </dataTypeDef> <dataTypeDef> <name>ExtendedResultType</name> <synopsis> Possible extended result support </synopsis> <atomic> <baseType>uchar</baseType> <rangeRestriction> <allowedRange min="1" max="2"/> </rangeRestriction> <specialValues> <specialValue value="1"> <name>EResultNotSupported</name> <synopsis> Extended Results are not supported </synopsis> </specialValue> <specialValue value="2"> <name>EResultSupported</name> <synopsis> Extended Results are supported </synopsis> </specialValue> </specialValues> </atomic> </dataTypeDef> </dataTypeDefs> <LFBClassDefs> <LFBClassDef LFBClassID="2"> <name>FEPO</name> <synopsis> The FE Protocol Object, with EXtended Result control </synopsis> <version>1.2</version> <components> <component componentID="1" access="read-only"> <name>CurrentRunningVersion</name> <synopsis>Currently running ForCES version</synopsis> <typeRef>uchar</typeRef> </component> <component componentID="2" access="read-only"> <name>FEID</name> <synopsis>Unicast FEID</synopsis> <typeRef>uint32</typeRef> </component> <component componentID="3" access="read-write"> <name>MulticastFEIDs</name> <synopsis> the table of all multicast IDs </synopsis> <array type="variable-size"> <typeRef>uint32</typeRef> </array> </component> <component componentID="4" access="read-write"> <name>CEHBPolicy</name> <synopsis> The CE Heartbeat Policy </synopsis> <typeRef>CEHBPolicyValues</typeRef> </component> <component componentID="5" access="read-write"> <name>CEHDI</name> <synopsis> The CE Heartbeat Dead Interval in millisecs </synopsis> <typeRef>uint32</typeRef> </component> <component componentID="6" access="read-write"> <name>FEHBPolicy</name> <synopsis> The FE Heartbeat Policy </synopsis> <typeRef>FEHBPolicyValues</typeRef> </component> <component componentID="7" access="read-write"> <name>FEHI</name> <synopsis> The FE Heartbeat Interval in millisecs </synopsis> <typeRef>uint32</typeRef> </component> <component componentID="8" access="read-write"> <name>CEID</name> <synopsis> The Primary CE this FE is associated with </synopsis> <typeRef>uint32</typeRef> </component> <component componentID="9" access="read-write"> <name>BackupCEs</name> <synopsis> The table of all backup CEs other than the primary </synopsis> <array type="variable-size"> <typeRef>uint32</typeRef> </array> </component> <component componentID="10" access="read-write"> <name>CEFailoverPolicy</name> <synopsis> The CE Failover Policy </synopsis> <typeRef>CEFailoverPolicyValues</typeRef> </component> <component componentID="11" access="read-write"> <name>CEFTI</name> <synopsis> The CE Failover Timeout Interval in millisecs </synopsis> <typeRef>uint32</typeRef> </component> <component componentID="12" access="read-write"> <name>FERestartPolicy</name> <synopsis> The FE Restart Policy </synopsis> <typeRef>FERestartPolicyValues</typeRef> </component> <component componentID="13" access="read-write"> <name>LastCEID</name> <synopsis> The Primary CE this FE was last associated with </synopsis> <typeRef>uint32</typeRef> </component> <component componentID="14" access="read-write"> <name>HAMode</name> <synopsis> The HA mode used </synopsis> <typeRef>HAModeValues</typeRef> </component> <component componentID="15" access="read-only"> <name>AllCEs</name> <synopsis>The table of all CEs</synopsis> <array type="variable-size"> <typeRef>AllCEType</typeRef> </array> </component> <component componentID="16" access="read-write"> <name>EResultAdmin</name> <synopsis> Turn Extended results off or on. default to off </synopsis> <typeRef>ExtendedResultType</typeRef> <defaultValue>1</defaultValue> </component> </components> <capabilities> <capability componentID="30"> <name>SupportableVersions</name> <synopsis> the table of ForCES versions that FE supports </synopsis> <array type="variable-size"> <typeRef>uchar</typeRef> </array> </capability> <capability componentID="31"> <name>HACapabilities</name> <synopsis> the table of HA capabilities the FE supports </synopsis> <array type="variable-size"> <typeRef>FEHACapab</typeRef> </array> </capability> <capability componentID="32"> <name>EResultCapab</name> <synopsis> the table of supported result capabilities </synopsis> <array type="variable-size"> <typeRef>ExtendedResultType</typeRef> </array> </capability> </capabilities> <events baseID="61"> <event eventID="1"> <name>PrimaryCEDown</name> <synopsis> The primary CE has changed </synopsis> <eventTarget> <eventField>LastCEID</eventField> </eventTarget> <eventChanged/> <eventReports> <eventReport> <eventField>LastCEID</eventField> </eventReport> </eventReports> </event> <event eventID="2"> <name>PrimaryCEChanged</name> <synopsis>A New primary CE has been selected </synopsis> <eventTarget> <eventField>CEID</eventField> </eventTarget> <eventChanged/> <eventReports> <eventReport> <eventField>CEID</eventField> </eventReport> </eventReports> </event> </events> </LFBClassDef> </LFBClassDefs> </LFBLibrary>