Registration Data Access Protocol (RDAP) Query Parameters for Result Sorting and Paging

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 3 June 2021.¶

Copyright Notice

Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶

Table of Contents

2. RDAP Query Parameter Specification

The new query parameters are OPTIONAL extensions of path segments defined in [RFC7482]. They are as follows:¶

• "count": a boolean value that allows a client to request the return of the total number of objects found;¶

• "sort": a string value that allows a client to request a specific sort order for the result set;¶

• "cursor": a string value representing a pointer to a specific fixed size portion of the result set.¶

Augmented Backus-Naur Form (ABNF) [RFC5234] is used in the following sections to describe the formal syntax of these new parameters.¶

{
  "rdapConformance": [
        "rdap_level_0",
        "paging"
  ],
  ...
  "paging_metadata": {
    "totalCount": 43
  },
  "domainSearchResults": [
    ...
  ]
}

{
  "rdapConformance": [
    "rdap_level_0",
    "sorting"
  ],
  ...
  "sorting_metadata": {
     "currentSort": "name",
     "availableSorts": [
       {
       "property": "registrationDate",
       "jsonPath": "$.domainSearchResults[*]
          .events[?(@.eventAction==\"registration\")].eventDate",
       "default": false,
       "links": [
         {
         "value": "https://example.com/rdap/domains?name=example*.com
                   &sort=name",
         "rel": "alternate",
         "href": "https://example.com/rdap/domains?name=example*.com
                  &sort=registrationDate",
         "title": "Result Ascending Sort Link",
         "type": "application/rdap+json"
         },
         {
         "value": "https://example.com/rdap/domains?name=example*.com
                   &sort=name",
         "rel": "alternate",
         "href": "https://example.com/rdap/domains?name=example*.com
                  &sort=registrationDate:d",
         "title": "Result Descending Sort Link",
         "type": "application/rdap+json"
         }
       ]
       },
       ...
     ]
  },
  "domainSearchResults": [
    ...
  ]
}

{
  "rdapConformance": [
    "rdap_level_0",
    "paging"
  ],
  ...
  "notices": [
    {
      "title": "Search query limits",
      "type": "result set truncated due to excessive load",
      "description": [
      "search results for domains are limited to 50"
      ]
    }
  ],
  "paging_metadata": {
    "totalCount": 73,
    "pageSize": 50,
    "pageNumber": 1,
    "links": [
      {
      "value": "https://example.com/rdap/domains?name=example*.com",
      "rel": "next",
      "href": "https://example.com/rdap/domains?name=example*.com
              &cursor=wJlCDLIl6KTWypN7T6vc6nWEmEYe99Hjf1XY1xmqV-M=",
      "title": "Result Pagination Link",
      "type": "application/rdap+json"
      }
    ]
  },
  "domainSearchResults": [
    ...
  ]
}

3. Negative Answers

The constraints for the parameters values are defined by their ABNF syntax. Therefore, each request that includes an invalid value for a parameter SHOULD produce an HTTP 400 (Bad Request) response code. The same response SHOULD be returned in the following cases:¶

• if in both single and multi sort the client provides an unsupported value for the "sort" parameter, as well as a value related to an object property not included in the response;¶

• if the client submits an invalid value for the "cursor" parameter.¶

Optionally, the response MAY include additional information regarding either the supported sorting properties or the correct cursor values in the HTTP entity body (Figure 4).¶

{
    "errorCode": 400,
    "title": "Domain sorting property 'unknownproperty' is not valid",
    "description": [
        "Supported domain sorting properties are: 'aproperty', 'anotherproperty'."
    ]

}

4. Implementation Considerations

Implementation of the new parameters is technically feasible, as operators for counting, sorting and paging are currently supported by the major relational database management systems. Similar operators are completely or partially supported by the most well-known NoSQL databases (e.g. MongoDB, CouchDB, HBase, Cassandra, Hadoop). Additional implementation notes are included in Appendix C.¶

5. IANA Considerations

IANA is requested to register the following values in the RDAP Extensions Registry:¶

• Extension identifier: paging¶
• Registry operator: Any¶
• Published specification: This document.¶
• Contact: IETF <iesg@ietf.org>¶
• Intended usage: This extension describes best practice for result set paging.¶

• Extension identifier: sorting¶
• Registry operator: Any¶
• Published specification: This document.¶
• Contact: IETF <iesg@ietf.org>¶
• Intended usage: This extension describes best practice for result set sorting.¶

6. Implementation Status

NOTE: Please remove this section and the reference to RFC 7942 prior to publication as an RFC.¶

This section records the status of known implementations of the protocol defined by this specification at the time of posting of this Internet-Draft, and is based on a proposal described in [RFC7942]. The description of implementations in this section is intended to assist the IETF in its decision processes in progressing drafts to RFCs. Please note that the listing of any individual implementation here does not imply endorsement by the IETF. Furthermore, no effort has been spent to verify the information presented here that was supplied by IETF contributors. This is not intended as, and must not be construed to be, a catalog of available implementations or their features. Readers are advised to note that other implementations may exist.¶

According to RFC 7942, "this will allow reviewers and working groups to assign due consideration to documents that have the benefit of running code, which may serve as evidence of valuable experimentation and feedback that have made the implemented protocols more mature. It is up to the individual working groups to use this information as they see fit".¶

7. Security Considerations

Security services for the operations specified in this document are described in [RFC7481].¶

A search query typically requires more server resources (such as memory, CPU cycles, and network bandwidth) when compared to a lookup query. This increases the risk of server resource exhaustion and subsequent denial of service. This risk can be mitigated by either restricting search functionality or limiting the rate of search requests. Servers can also reduce their load by truncating the results in a response. However, this last security policy can result in a higher inefficiency or risk due to acting on incomplete information if the RDAP server does not provide any functionality to return the truncated results.¶

The new parameters presented in this document provide RDAP operators with a way to implement a server that reduces inefficiency risks. The "count" parameter gives the client the ability to evaluate the completeness of a response. The "sort" parameter allows the client to obtain the most relevant information at the beginning of the result set. This can reduce the number of unnecessary search requests. Finally, the "cursor" parameter enables the user to scroll the result set by submitting a sequence of sustainable queries within server-acceptable limits.¶

Appendix A. JSONPath operators

The jsonpaths used in this document are provided according to the Goessner v.0.8.0 proposal [GOESSNER-JSON-PATH].¶

Such specification requires that implementations support a set of "basic operators". These operators are used to access the elements of a JSON structure like objects and arrays, and their subelements, respectively, object members and array items. No operations are defined for retrieving parent or sibling elements of a given element. The root element is always referred to as $ regardless of it being an object or array.¶

Additionally, the specification permits implementations to support arbitrary script expressions. These can be used to index into an object or array, or to filter elements from an array. While script expression behavior is implementation-defined, most implementations support the basic relational and logical operators, as well as both object member and array item access, sufficiently similar for the purpose of this document. Commonly-supported operators/functions divided into "top-level operators" and "filter operators" are documented in Table 2 and Table 3 respectively.¶

For more information on implementation interoperability issues, see [JSONPATH-COMPARISON]. As at the time of writing, work is beginning on a standardization effort, too: see [JSONPATH-WG].¶

Appendix B. Approaches to Result Pagination

An RDAP query could return a response with hundreds, even thousands, of objects, especially when partial matching is used. For this reason, the cursor parameter addressing result pagination is defined to make responses easier to handle.¶

Presently, the most popular methods to implement pagination in a REST API include offset pagination and keyset pagination. Neither pagination method requires the server to handle the result set in a storage area across multiple requests since a new result set is generated each time a request is submitted. Therefore, they are preferred to any other method requiring the management of a REST session.¶

Using limit and offset operators represents the traditionally used method to implement result pagination. Both of them can be used individually:¶

• "limit=N": means that the server returns the first N objects of the result set;¶

• "offset=N": means that the server skips the first N objects and returns objects starting from position N+1.¶

When limit and offset are used together, they provide the ability to identify a specific portion of the result set. For example, the pair "offset=100,limit=50" returns the first 50 objects starting from position 101 of the result set.¶

Though easy to implement, offset pagination also includes drawbacks:¶

• when offset has a very high value, scrolling the result set could take some time;¶

• it always requires fetching all rows before dropping as many rows as specified by offset;¶

• it may return inconsistent pages when data are frequently updated (i.e. real-time data).¶

Keyset pagination [SEEK] adds a query condition that enables the selection of the only data not yet returned. This method has been taken as the basis for the implementation of a "cursor" parameter [CURSOR] by some REST API providers [CURSOR-API1] [CURSOR-API2]. The cursor is an opaque to client URL-safe string representing a logical pointer to the first result of the next page.¶

Nevertheless, even keyset pagination can be troublesome:¶

• it needs at least one key field;¶

• it does not allow sorting simply by any field because the sorting criterion must contain a key;¶

• it works best with full composite values support by data base management systems (i.e. [x,y]>[a,b]), emulation is possible but inelegant and less efficient;¶

• it does not allow direct navigation to arbitrary pages because the result set must be scrolled in sequential order starting from the initial page;¶

• implementing bi-directional navigation is tedious because all comparison and sort operations have to be reversed.¶

Appendix C. Additional Implementation Notes

This section contains an overview of the main choices made during the implementation of the capabilities defined above in the RDAP public test server of Registro.it at the Institute of Informatics and Telematics of the National Research Counci (IIT-CNR). The content of this section can represent a guidance for those implementers who plan to provide RDAP users with those capabilities. The RDAP public test server can be accessed at https://rdap.pubtest.nic.it/. Further documentation about the server features is available at https://rdap.pubtest.nic.it/doc/README.html.¶

Authors' Addresses