Internet Engineering Task Force H. VandeSompel
Internet-Draft Los Alamos National Laboratory
Intended status: Informational M.L. Nelson
Expires: May 20, 2013 Old Dominion University
R.D. Sanderson
Los Alamos National Laboratory
November 16, 2012

HTTP framework for time-based access to resource states -- Memento
draft-vandesompel-memento-05

Abstract

The HTTP-based Memento framework bridges the present and past Web by interlinking current resources with resources that encapsulate their past. It facilitates accessing prior states of a resource, encapsulated in archival resources in Web archives or version resources in content management systems, by leveraging the resource's URI and a preferred datetime. To this end, the framework introduces datetime negotiation (a variation on content negotiation) and new Relation Types. It also introduces discovery mechanisms to support these capabilities.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http:/⁠/⁠datatracker.ietf.org/⁠drafts/⁠current/⁠.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on May 20, 2013.

Copyright Notice

Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http:/⁠/⁠trustee.ietf.org/⁠license-⁠info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.


Table of Contents

1. Introduction

1.1. Terminology

This specification uses the terms "resource", "request", "response", "entity-body", "content negotiation", "client", "user agent", "server" as described in [RFC2616], and it uses the terms "representation" and "resource state" as described in [W3C.REC-aww-20041215].

In addition, the following terms specific to the Memento framework are introduced:

1.2. Purpose

The state of an Original Resource may change over time. Dereferencing its URI at any specific moment in time yields a response that reflects the resource's state at that moment: a representation of the resource's state (e.g. "200 OK" HTTP status code), an indication of its non-existence (e.g. "404 Not Found" HTTP status code), a relation to another resource (e.g. "302 Found" HTTP status code), etc. However, resources may also exist that encapsulate prior states of an Original Resource. Each such resource, named a Memento, has its own URI that, when dereferenced, yields a response that reflects a prior state of the Original Resource: a representation of a prior state of the Original Resource, an indication that the Original Resource did not exist at some time in the past, a relation that the Original Resource had to another resource at some time in the past, etc. Mementos exist in Web archives, Content Management Systems, or Revision Control Systems, among others. For any given Original Resource several Mementos may exist, each one reflecting a frozen prior state of the Original Resource. In terms of the Ontology for Relating Generic and Specific Information Resources [W3C.gen-ont-20090420]), a Memento is a FixedResource.

Examples are:

Mementos for Original Resource http://www.ietf.org/ :

Mementos for Original Resource http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol :

Mementos for Original Resource http://www.w3.org/TR/webarch/ :

In the abstract, Memento introduces a mechanism to access versions of Web resources that:

The core components of Memento's mechanism to access resource versions are:

1. The abstract notion of the state of a resource identified by URI-R as it existed at some time Tj. Note the relationship with the ability to identify a the state of a resource at some datetime Tj by means of a URI as intended by the proposed Dated URI scheme [I-D.masinter-dated-uri].

2. A bridge from the present to the past, consisting of:

3. A bridge from the past to the present, consisting of an appropriately typed link from a resource identified by URI-M, which encapsulates the state a resource identified by URI-R had at some datetime Tj, to the resource identified by URI-R.

Section 2 and Section 3 of this document are concerned with specifying an instantiation of these abstractions for resources that are identified by HTTP(S) URIs, whereas Section 4 details an approach to support batch discovery of TimeGates and TimeMaps that is based on well-known URI [RFC5785] and host-meta [RFC6415].

1.3. Notational Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

When needed for extra clarity, the following conventions are used:

2. The Memento Framework, Datetime Negotiation component: HTTP headers, HTTP Link Relation Types

The Memento framework is concerned with Original Resources, TimeGates, Mementos, and TimeMaps that are identified by HTTP or HTTPS URIs. Details are only provided for resources identified by HTTP URIs but apply similarly to those with HTTPS URIs.

2.1. HTTP Headers

The Memento framework operates at the level of HTTP request and response headers. It introduces two new headers ("Accept-Datetime", "Memento-Datetime"), introduces new values for two existing headers ("Vary", "Link"), and uses an existing header ("Location") without modification. All these headers are described below. Other HTTP headers are present or absent in Memento response/request cycles as specified by [RFC2616].

2.1.1. Accept-Datetime, Memento-Datetime

The "Accept-Datetime" request header is used by a user agent to indicate it wants to access a Memento that encapsulates a past state of an Original Resource. To that end, the "Accept-Datetime" header is conveyed in an HTTP request issued against a TimeGate for an Original Resource, and its value indicates the datetime of the desired past state of the Original Resource.

The "Memento-Datetime" response header is used by a server to indicate that the response is a Memento, and its value expresses the datetime of the state of an Original Resource that is encapsulated in that Memento. The URI of that Original Resource is provided in the response, as the Target IRI (see [RFC5988]) of a link provided in the HTTP "Link" header that has a Relation Type of "original" (see Section 2.2).

The presence of a "Memento-Datetime" header and associated value for a given resource constitutes a promise that the resource is stable and that its state will no longer change. This means that, in terms of the Ontology for Relating Generic and Specific Information Resources (see [W3C.gen-ont-20090420]), a Memento is a FixedResource.

As a consequence, "Memento-Datetime" headers associated with a Memento MUST be "sticky" in the following ways:

2.1.1.1. Values for Accept-Datetime

Values for the "Accept-Datetime" header consist of a MANDATORY datetime expressed according to the [RFC1123] format, which is formalized by the rfc1123-date construction rule of the BNF in Figure 1. The datetime MUST be represented in Greenwich Mean Time (GMT).

Example of an "Accept-Datetime" request header:

 
Accept-Datetime: Thu, 31 May 2007 20:35:00 GMT

The user agent uses the MANDATORY datetime value to convey its preferred datetime for a Memento.


accept-dt-value = rfc1123-date *SP
rfc1123-date = wkday "," SP date1 SP time SP "GMT"
date1        = 2DIGIT SP month SP 4DIGIT 
                  ; day month year (e.g., 20 Mar 1957)
time         = 2DIGIT ":" 2DIGIT ":" 2DIGIT 
                  ; 00:00:00 - 23:59:59 (e.g., 14:33:22)
wkday        = "Mon" | "Tue" | "Wed" | "Thu" | "Fri" | "Sat" | 
               "Sun" 
month        = "Jan" | "Feb" | "Mar" | "Apr" | "May" | "Jun" | 
               "Jul" | "Aug" | "Sep" | "Oct" | "Nov" | "Dec"

Figure 1: BNF for the datetime format

2.1.1.2. Values for Memento-Datetime

Values for the "Memento-Datetime" headers MUST be datetimes expressed according to the rfc1123-date construction rule of the BNF in Figure 1; they MUST be represented in Greenwich Mean Time (GMT).

An example "Memento-Datetime" response header:

 
Memento-Datetime: Wed, 30 May 2007 18:47:52 GMT

2.1.2. Vary

Generally, the "Vary" header is used in HTTP responses to indicate the dimensions in which content negotiation is possible. In the Memento framework, a TimeGate uses the "Vary" header with a value that includes "negotiate" and "accept-datetime" to convey that datetime negotation is possible.

For example, this use of the "Vary" header indicates that datetime is the only dimension in which negotiation is possible:

Vary: negotiate, accept-datetime

The use of the "Vary" header in this example shows that both datetime negotiation, and media type content negotiation are possible:

Vary: negotiate, accept-datetime, accept

2.1.3. Location

The "Location" header is used as defined in [RFC2616]. Examples are given in Section 3 below.

2.1.4. Link

The "Link" response header is specified in [RFC5988]. The Memento framework introduces new Relation Types to convey typed links among Original Resources, TimeGates, Mementos, Redirecting Resources, and TimeMaps. Existing Relation Types, among others, aimed at supporting navigation among a series of ordered resources may also be used in the Memento framework. This is detailed in the next section.

2.2. Link Header Relation Types

2.2.1. Memento Framework Relation Types

This section introduces the Relation Types used in the Memento framework. They are defined in a general way and their use in HTTP "Link" Headers is detailed. The use of these Relation Types in TimeMaps and for the purpose of batch discovery is described in Section 3.5 and Section 4, respectively.

The below table summarizes the use of Memento Relation Types in HTTP "Link" headers by Original Resources, TimeGates, Mementos, and Redirecting Resources.

Appendix A shows a Memento HTTP request/response cycle that uses all the Memento Relation Types.

The use of Relation Types
Relation Type Original Resource TimeGate Memento Redirecting Resource
original NA, except see Section 3.1.2.1 REQUIRED, 1 REQUIRED, 1 REQUIRED, 1
timegate RECOMMENDED, 0 or more NA RECOMMENDED, 0 or more NA
timemap OPTIONAL, 0 or more RECOMMENDED, 0 or more RECOMMENDED, 0 or more NA
memento OPTIONAL, 0 or more REQUIRED, 1 or more REQUIRED, 1 or more, except see Section 3.3.2.4 NA

2.2.1.1. Relation Type "original"

"original" -- A link with an "original" Relation Type is used to point from a TimeGate, a Memento, or a Redirecting Resource to their associated Original Resource.

Use in HTTP "Link" headers: A TimeGate, a Memento, and a Redirecting Resource MUST include exactly one link with an "original" Relation Type in their HTTP "Link" header.

2.2.1.2. Relation Type "timegate"

"timegate" -- A link with a "timegate" Relation Type is used to point from the Original Resource itself, as well as from a Memento associated with the Original Resource, to a TimeGate for the Original Resource.

Use in HTTP "Link" headers: An Original Resource and a Memento SHOULD include a link with a "timegate" Relation Type in their HTTP "Link" header. Since multiple TimeGates can exist for any Original Resource, multiple "timegate" links MAY occur, each with a distinct Target IRI.

2.2.1.3. Relation Type "timemap"

"timemap" -- A link with a "timemap" Relation Type is used to point from a TimeGate or a Memento associated with an Original Resource, as well as from the Original Resource itself, to a TimeMap for the Original Resource.

Attributes: A link with a "timemap" Relation Type SHOULD use the "type" attribute to convey the mime type of the TimeMap serialization. The "from" and "until" attributes may be used to express the start and end of the temporal interval covered by Mementos listed in the TimeMap. That is, the linked TimeMap will not contain Mementos with archival datetimes outside of the expressed temporal interval. Attemtps SHOULD be made to convey this interval as accurately as possible. The value for the these attributes MUST be a datetime expressed according to the rfc1123-date construction rule of the BNF in Figure 1 and it MUST be represented in Greenwich Mean Time (GMT).

Use in HTTP "Link" headers: A TimeGate and a Memento SHOULD include a link with a "timemap" Relation Type in their HTTP "Link" header. Multiple such links, each with a distinct Target IRI, MAY be expressed as a means to point to different TimeMaps or to different serializations of a same TimeMap. An Original Resource MAY include one or more "timemap" link in its HTTP header. In all cases, use of the "from" and "until" attributes is OPTIONAL.

2.2.1.4. Relation Type "memento"

"memento" -- A link with a "memento" Relation Type is used to point from a TimeGate or a Memento for an Original Resource, as well as from the Original Resource itself, to a Memento for the Original Resource.

Attributes: A link with a "memento" Relation Type MUST include a "datetime" attribute with a value that matches the "Memento-Datetime" of the Memento that is the target of the link; that is, the value of the "Memento-Datetime" header that is returned when the URI of the linked Memento is dereferenced. The value for the "datetime" attribute MUST be a datetime expressed according to the rfc1123-date construction rule of the BNF in Figure 1 and it MUST be represented in Greenwich Mean Time (GMT). This link MAY include a "license" attribute to associate a license with the Memento; the value for the "license" attribute SHOULD be a URI.

Use in HTTP "Link" headers: A TimeGate and a Memento MUST include links in their HTTP "Link" headers with a "memento" Relation Type and, for the most common case whereby a Memento is selected in a response to an HTTP requests, the links MUST be as follows:

  • One "memento" link MUST be included that has as Target IRI the URI of the Memento that was selected or served;
  • One "memento" link MUST be included that has as Target IRI the URI of the temporally first Memento known to the responding server;
  • One "memento" link MUST be included that has as Target IRI the URI of the temporally most recent Memento known to the responding server.
  • One "memento" link SHOULD be included that has as Target IRI the URI of the Memento that is previous to the selected Memento in the temporal series of all Mementos (sorted by ascending "Memento-Datetime" values) known to the server;
  • One "memento" link SHOULD be included that has as Target IRI the URI the Memento that is next to the selected Memento in the temporal series of all Mementos (sorted by ascending "Memento-Datetime" values) known to the server.
  • Other "memento" links MAY only be included if both the aforementioned previous and next links are provided. Each of these OPTIONAL "memento" links MUST have as Target IRI the URI of a Memento other than the ones listed above.

Note that the Target IRI of some of these links may coincide. For example, if the selected Memento actually is the first Memento known to the server, only three distinct "memento" links may result. The value for the "datetime" attribute of these links would be the datetimes of the first (equal to selected), next, and most recent Memento known to the responding server.

The use of the "memento" Relation Types for special cases whereby no Memento is selected is described in Section 3.2.2.5, Section 3.2.2.6, Section 3.3.2.4, and Section 3.3.2.5.

2.2.2. Other Relation Types

Web Linking [RFC5988] allows for the inclusion of links with different Relation Types but the same Target IRI, and hence the Relation Types introduced by the Memento framework MAY be combined with others as deemed necessary. As the "memento" Relation Type focuses on conveying the datetime of a linked Memento, Relation Types that allow navigating among the temporally ordered series of Mementos known to a server are of particular importance. With this regard, the Relation Types listed in the below table SHOULD be considered for combination with the "memento" Relation Type. A distinction is made between responding servers that can be categorized as systems that are the focus of [RFC5829] (such as version control systems) and others that can not (such as Web archives). Note that, in terms of [RFC5829], the last Memento (URI-Mn) is the version prior to the latest (i.e. current) version.

The use of Relation Types
Memento Type RFC5829 system non RFC5829 system
First Memento (URI-M0) first first
Last Memento (URI-Mn) last last
Selected Memento (URI-Mj) NA NA
Memento prior to selected Memento (URI-Mi) predecessor-version prev
Memento next to selected Memento (URI-Mk) successor-version next

3. The Memento Framework, Datetime Negotiation component: HTTP Interactions

This section describes the HTTP interactions of the Memento framework for a variety of scenarios. First, Figure 2 provides a schematic overview of a successful request/response chain that involves datetime negotiation. Dashed lines depict HTTP transactions between user agent and server. Appendix A shows these HTTP interactions in detail for the case where the Original Resource resides on one server, whereas both the TimeGate and the Mementos reside on another. Scenarios also exist in which all these resources are on the same server (for example, Content Management Systems) or on different servers (for example, an aggregator of TimeGates). Note that, in Step 2 and Step 6, the HTTP status code of the response is shown as "200 OK", but a series of "206 Partial Content" responses could be substituted without loss of generality.

1: UA --- HTTP HEAD/GET; Accept-Datetime: Tj ---------------> URI-R
2: UA <-- HTTP 200; Link: URI-G ----------------------------- URI-R
3: UA --- HTTP HEAD/GET; Accept-Datetime: Tj ---------------> URI-G
4: UA <-- HTTP 302; Location: URI-Mj; Vary; Link:
      URI-R,URI-T,URI-M0,URI-Mn,URI-Mi,URI-Mj,URI-Mk -------- URI-G
5: UA --- HTTP GET URI-Mj; Accept-Datetime: Tj -------------> URI-Mj
6: UA <-- HTTP 200; Memento-Datetime: Tj; Link:  
      URI-R,URI-T,URI-G,URI-M0,URI-Mn,URI-Mi,URI-Mj,URI-Mk -- URI-Mj

Figure 2: Typical Memento request/response chain

  • Step 1: In order to discover the URI of a TimeGate for an Original Resource, the user agent issues an HTTP HEAD/GET request against the URI of the Original Resource (URI-R).
  • Step 2: The response from URI-R includes an HTTP "Link" header with a Relation Type of "timegate" pointing at a TimeGate for the Original Resource (URI-G).
  • Step 3: The user agent starts the datetime negotiation process with the TimeGate by issuing an HTTP GET request against its URI-G thereby including an "Accept-Datetime" HTTP header with a value of the datetime of the desired prior state of the Original Resource.
  • Step 4: The response from URI-G includes a "Location" header pointing at the URI of a Memento for the Original Resource (URI-Mj). In addition, the response contains an HTTP "Link" header with a Relation Type of "original" pointing at the Original Resource, and an HTTP "Link" header with a Relation Type of "timemap" pointing at a TimeMap (URI-T). Also HTTP Links pointing at various Mementos are provided using the "memento" Relation Type, as specified in Section 2.2.1.4.
  • Step 5: The user agent issues an HTTP GET request against the URI of the Memento obtained in Step 4 (URI-Mj).
  • Step 6: The response from URI-Mj includes a "Memento-Datetime" HTTP header with a value of the datetime of the Memento. It also contains an HTTP "Link" header with a Relation Type of "original" pointing at the Original Resource, with a Relation Type of "timegate" pointing at a TimeGate associated with the Original Resource, and with a Relation Type of "timemap" pointing at a TimeMap. The state that is expressed by the response is the state the Original Resource had at the datetime expressed in the "Memento-Datetime" header. This response also includes HTTP Links with a "memento" Relation Type pointing at various Mementos, as specified in Section 2.2.1.4.

The following sections detail the specifics of HTTP interactions with Original Resources, TimeGates, Mementos, and TimeMaps under various conditions.

3.1. Interactions with an Original Resource

This section details HTTP HEAD/GET requests targeted at an Original Resource (URI-R).

3.1.1. Step 1: User Agent Requests an Original Resource

In order to attempt to discover the recommended TimeGate for the Original Resource, the user agent SHOULD issue an HTTP HEAD or GET request against the Original Resource's URI. Use of the "Accept-Datetime" header in the HTTP HEAD/GET request is OPTIONAL.

Figure 3 shows the use of HTTP HEAD to determine a TimeGate for the Original Resource http://a.example.org.

HEAD / HTTP/1.1
Host: a.example.org
Accept-Datetime: Tue, 11 Sep 2001 20:35:00 GMT
Connection: close

Figure 3: User Agent Requests Original Resource

3.1.2. Step 2: Server Responds to a Request for an Original Resource

The response of the Original Resource's server to the user agent's HTTP HEAD/GET request of Step 1, for the case where the Original Resource exists, is as it would be in a regular HTTP request/response cycle, but in addition MAY include a "timegate" link in the HTTP "Link" header that conveys the URI of the Original Resource's TimeGate as the Target IRI of the link. Multiple "timegate" links MAY be provided to accommodate situations in which the server is aware of multiple TimeGates for an Original Resource. The presence or abscence of an "Accept-Datetime" header in a user agent's HTTP request MUST NOT influence the presence or abscence of a "timegate" link in the HTTP "Link" header of the server's response. A response for this case is illustrated in Figure 4.

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Link: <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate"
Content-Length: 255
Connection: close
Content-Type: text/html; charset=iso-8859-1

Figure 4: Server of Original Resource Responds

Servers that actively maintain archives of their resources SHOULD include the "timegate" HTTP "Link" header because this link is an important way for a user agent to discover TimeGates for those resources. This includes servers such as Content Management Systems, Control Version Systems, and Web servers with associated transactional archives [Fitch]. Servers that do not actively maintain archives of their resources MAY include the "timegate" HTTP "Link" header as a way to convey a preference for TimeGates for their resources exposed by a third party archive. This includes servers that rely on Web archives such as the Internet Archive to archive their resources.

3.1.2.1. Original Resource is an Appropriate Memento

The "Memento-Datetime" header MAY be applied to an Original Resource directly to indicate it is a FixedResource (see [W3C.gen-ont-20090420]), meaning that the state of the Original Resource has not changed since the datetime conveyed in the "Memento-Datetime" header, and as a promise that it will not change anymore beyond it. This may occur, for example, for certain stable media resources on news sites. In case the user agent's preferred datetime is equal to or more recent than the datetime conveyed as the value of "Memento-Datetime" in the server's response in Step 2, the user agent SHOULD conclude it has located an appropriate Memento, and it SHOULD NOT continue to Step 3.

Figure 5 illustrates such a response to a request for the resource with URI http://a.example.org/pic that has been stable since it was created. Note the use of both the "memento" and "original" Relation Types for links that have as Target IRI the URI of the Original Resource.

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Link: 
 <http://a.example.org/pic> 
  ; rel="original memento" 
  ; datetime="Fri, 20 Mar 2009 11:00:00 GMT"
Memento-Datetime: Fri, 20 Mar 2009 11:00:00 GMT
Content-Length: 255
Connection: close
Content-Type: text/html; charset=iso-8909-1

Figure 5: Response to a request for an Original Resource that was created as a FixedResource

Cases may also exist in which a resource becomes stable at a certain point in its existence, but changed previously. In such cases, the Original Resource may know about a TimeGate that is aware of its prior history and hence MAY also include a link with a "timegate" Relation Type in addition to the previously mentioned links with "original" and "memento" Relation Types.

3.1.2.2. Server Exists and Original Resource Used to Exist

Servers SHOULD also provide a "timegate" link in the HTTP "Link" header in responses to requests for an Original Resource that the server knows used to exist, but no longer does so. This allows the use of an Original Resource's URI as an entry point to its prior states even if the resource itself no longer exists. A server's response for this case is illustrated in Figure 6.

HTTP/1.1 404 Not Found
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Link: 
 <http://arxiv.example.net/timegate/http://a.example.org/gone> 
  ; rel="timegate"
Content-Length: 255
Connection: close
Content-Type: text/html; charset=iso-8909-1

Figure 6: Response to a request for an Original Resource that not longer exists

In case the server is not aware of the prior existence of the Original Resource, its response SHOULD NOT include a "timegate" link in the HTTP "Link" header. Section 3.1.2.3 details what the user agent's behavior should be in such cases.

3.1.2.3. Missing "timegate" Link in Original Server's Response

When engaging in a Memento request/response cycle, a user agent SHOULD NOT proceed immediately to Step 3 by using a TimeGate of its own preference but rather SHOULD always start the cycle by issuing an HTTP HEAD/GET against the Original Resource (Step 1, Figure 3) as it is an important way to learn about dedicated or preferred TimeGates for the Original Resource. Also, cases exist in which the response in Step 2 will not provide a "timegate" link in the HTTP "Link" header, including:

  • The Original Resource's server does not support the Memento framework;
  • The Original Resource no longer exists and the responding server is not aware of its prior existence;
  • The server that hosted the Original Resource no longer exists;

In all these cases, the user agent SHOULD attempt to determine an appropriate TimeGate for the Original Resource, either automatically or interactively supported by the user. The discovery mechanisms described in Section 4 can support the user agent with this regard.

3.2. Interactions with a TimeGate

This section details HTTP HEAD/GET requests targeted at a TimeGate (URI-G).

3.2.1. Step 3: User Agent Negotiates with a TimeGate

In order to negotiate in the datetime dimension with a TimeGate, the user agent MUST issue an HTTP request with the "Accept-Datetime" header, as described in Section 2.1.1.1. This is illustrated in Figure 7.

GET /timegate/http://a.example.org HTTP/1.1
Host: arxiv.example.net
Accept-Datetime: Tue, 11 Sep 2001 20:35:00 GMT
Connection: close

Figure 7: User agent negotiates with TimeGate

3.2.2. Step 4: Server Responds to Negotiation with TimeGate

In order to respond to a datetime negotiation request (Step 3, Section 3.2.1), the server uses an internal algorithm to select the Memento that best meets the user agent's datetime preference, and redirects to it. The exact nature of the selection algorithm is at the server's discretion but SHOULD be consistent. A variety of approaches can be used including selecting the Memento that is nearest in time (either past or future) or nearest in the past relative to the requested datetime. The most common scenario for datetime negotiation with a TimeGate is described in Section 3.2.2.1 but special cases exist, and they are addressed in Section 3.2.2.2 through Section 3.2.2.7.

It is NOT RECOMMENDED to implement TimeGates such that they coincide with their associated Original Resource, i.e. URI-R and URI-G are the same. Doing so may cause cache invalidation problems and confuse Memento clients.

3.2.2.1. Successful Scenario

In cases where the TimeGate exists, and the datetime provided in the user agent's "Accept-Datetime" header can be parsed, the server selects a Memento based on the user agent's datetime preference. The response MUST have a "302 Found" HTTP status code, and the "Location" header MUST be used to convey the URI of the selected Memento. The "Vary" header MUST be provided and it MUST include the "negotiate" and "accept-datetime" values. The "Link" header MUST be provided and contain links with Relation Types subject to the considerations described in Section 2.2. The response MUST NOT contain a "Memento-Datetime" header. Such a response is illustrated in Figure 8.

If a user agent's "Accept-Datetime" header conveys a datetime that is either earlier than the datetime of the first Memento or later than the datetime of the most recent Memento known to the server, the server's response is as just described yet entails the selection of the first or most recent Memento, respectively.

HTTP/1.1 302 Found
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
Vary: negotiate, accept-datetime
Location: 
 http://arxiv.example.net/web/20010911203610/http://a.example.org
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="timemap"; type="application/link-format"
   ; from="Tue, 15 Sep 2000 11:28:26 GMT"
   ; until="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/web/20000915112826/http://a.example.org>
   ; rel="first memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
 <http://arxiv.example.net/web/20080708093433/http://a.example.org>
   ; rel="last memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="memento"; datetime="Tue, 11 Sep 2001 20:36:10 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="prev memento"; datetime="Tue, 11 Sep 2001 20:30:51 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org> 
   ; rel="next memento"; datetime="Tue, 11 Sep 2001 20:47:33 GMT"
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Figure 8: Server of TimeGate responds

3.2.2.2. Multiple Matching Mementos

Because the finest datetime granularity expressible using the [RFC1123] format used in HTTP is seconds level, cases may occur in which a TimeGate server is aware of multiple Mementos that meet the user agent's datetime preference. This may occur in Content Management Systems with very high update rates. The response in this case MUST be handled as in Section 3.2.2.1, with the selection of one of the matching Mementos.

3.2.2.3. Accept-Datetime and other Accept Headers Provided

When interacting with a TimeGate, the regular content negotiation dimensions (media type, character encoding, language, and compression) remain available. It is the TimeGate server's responsibility to honor (or not) such content negotiation, and in doing so it MUST always first select a Memento that meets the user agent's datetime preference, and then consider honoring regular content negotiation for it. As a result of this approach, the returned Memento will not necessarily meet the user agent's regular content negotiation preferences. Therefore, it is RECOMMENDED that the server provides "memento" links in the HTTP "Link" header pointing at Mementos that do meet the user agent's regular content negotiation requests and that have a value for the "Memento-Datetime" header in the temporal vicinity of the user agent's preferred datetime value.

3.2.2.4. Accept-Datetime Not Provided

In case, in Step 3, a user agent issues a request to a TimeGate and fails to include an "Accept-Datetime" request header, the response MUST be handled as in Section 3.2.2.1, with a selection of the most recent Memento known to the responding server.

3.2.2.5. Accept-Datetime Unparseable

In case, in Step 3, a user agent conveys a value for the "Accept-Datetime" request header that does not conform to the accept-dt-value construction rule of the BNF in Figure 1, the TimeGate server's response MUST have a "400 Bad Request" HTTP status code. The use of the "Vary" header MUST be as described in Section 3.2.2.1. The use of entries in the "Link" header with "original", "timegate", and "timemap" Relation Types MUST be as described in Section 2.2. Entries with the "memento" Relation Type MUST be provided to only point to the first and most recent Mementos.

3.2.2.6. TimeGate Does Not Exist

Cases may occur in which a user agent issues a request against a TimeGate that does not exist. This may, for example, occur when a user agent uses internal knowledge to construct the URI of an assumed, yet non-existent TimeGate. In these cases, the response from the target server MUST have a "404 Not Found" HTTP status code. The response MUST NOT include a "Link" header with any of the Relation Types introduced in Section 2.2.1, and it MUST NOT contain a "Memento-Datetime" header.

3.2.2.7. HTTP Methods other than HEAD/GET

It is intended that the safe HTTP methods GET and HEAD are used with TimeGates, and other methods are currently undefined. A TimeGate MUST respond to unsupported HTTP methods with a "405 Method Not Allowed" HTTP status code.

3.2.3. Recognizing a TimeGate

When a user agent issues a HTTP HEAD/GET request against the URI of an assumed TimeGate for an Original Resource (e.g. the URI is the Target IRI of a "timegate" link), it SHOULD NOT conclude that the targeted resource effectively is a TimeGate and hence will behave as described in Section 3.2.2.

A user agent MUST decide it has reached a TimeGate if the response to a HTTP HEAD/GET request against the resource's URI contains a "Vary" header that includes the "negotiate" and "accept-datetime" values. If the response does not, the user agent MUST decide it has not reached a TimeGate and proceed as follows:

  • If the response contains a redirection that includes an "original" link in the HTTP "Link" header, the agent MUST conclude it has reached a Redirecting Resource that is part of the Memento infrastructure and SHOULD follow the redirect. Note that a chain of such redirections is possible, e.g. URI-R -> URI-1 (with rel="original") -> URI-2 (with rel="original") -> ... -> URI-G
  • If the response does not contain a redirection, or if the redirection does not have an "original" link in the HTTP "Link" header, the user agent SHOULD attempt to determine an appropriate TimeGate for the Original Resource, either automatically or interactively supported by the user. The discovery mechanisms described in Section 4 can support the user agent with this regard.

3.3. Interactions with a Memento

This section details HTTP HEAD/GET requests targeted at a Memento (URI-M).

3.3.1. Step 5: User Agent Requests a Memento

In Step 5, the user agent issues an HTTP GET request against the URI of a Memento. The user agent MAY include an "Accept-Datetime" header in this request, but the existence or absence of this header MUST NOT affect the server's response. The URI of the Memento may have resulted from a response in Step 4, or the user agent may simply have happened upon it. Such a request is illustrated in Figure 9.

GET /web/20010911203610/http://a.example.org HTTP/1.1
Host: arxiv.example.net
Accept-Datetime: Tue, 11 Sep 2001 20:35:00 GMT
Connection: close

Figure 9: User agent requests Memento

3.3.2. Step 6: Server Responds to a Request for a Memento

This section describes possible responses to a request for a Memento. Section 3.3.2.1 discusses the most common scenario. Section 3.3.2.2 through Section 3.3.2.4 detail special cases whereby Mementos are archived copies of HTTP responses with 3xx, 4xx and 5xx status codes.

3.3.2.1. Common Scenario

If the Memento requested by the user agent in Step 5 exists, and is not a special Memento as described in Section 3.3.2.2 and Section 3.3.2.2, the server's response MUST have a "200 OK" HTTP status code or, where appropriate "206 Partial Content". It MUST include a "Memento-Datetime" header with a value equal to the archival datetime of the Memento, that is, the datetime of the state of the Original Resource that is encapsulated in the Memento. The "Link" header MUST be provided and contain links subject to the considerations described in Section 2.2.

Figure 10 illustrates the server's response to the request issued against a Memento in Step 5 (Figure 9).

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:09:40 GMT
Server: Apache-Coyote/1.1
Memento-Datetime: Tue, 11 Sep 2001 20:36:10 GMT
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="timemap"; type="application/link-format"
   ; from="Tue, 15 Sep 2000 11:28:26 GMT"
   ; until="Tue, 08 Jul 2008 09:34:33 GMT", 
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/web/20000915112826/http://a.example.org>
   ; rel="first memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
 <http://arxiv.example.net/web/20080708093433/http://a.example.org>
   ; rel="last memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="memento"; datetime="Tue, 11 Sep 2001 20:36:10 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="prev memento"; datetime="Tue, 11 Sep 2001 20:30:51 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org> 
   ; rel="next memento"; datetime="Tue, 11 Sep 2001 20:47:33 GMT"
Content-Length: 23364
Content-Type: text/html;charset=utf-8
Connection: close

Figure 10: Server of Memento responds

The server's response MUST include the "Memento-Datetime" header regardless whether the user agent's request contained an "Accept-Datetime" header or not. This is the way by which resources make explicit that they are Mementos. Due to the sparseness of Mementos in most archives, the value of the "Memento-Datetime" header returned by a server may differ (significantly) from the value conveyed by the user agent in "Accept-Datetime".

Although a Memento encapsulates a prior state of an Original Resource, the entity-body returned in response to an HTTP GET request issued against a Memento may very well not be byte-to-byte the same as an entity-body that was previously returned by that Original Resource. Various reasons exist why there are significant chances these would be different yet do convey substantially the same information. These include format migrations as part of a digital preservation strategy, URI-rewriting as applied by some Web archives, and the addition of banners as a means to brand Web archives.

3.3.2.2. Memento of a 3XX Response

Cases exist in which HTTP responses with 3XX status codes are archived. For example, crawl-based web archives commonly archive responses with HTTP status codes "301 Moved Permanently" and "302 Found" whereas Linked Data archives hold on to "303 See Other" responses.

If the Memento requested by the user agent is an archived version of an HTTP response with a 3XX status code, the server's response MUST have the same 3XX HTTP status code, and it MUST include a "Memento-Datetime" header with a value equal to the archival datetime of the original 3XX response. All other considerations, e.g. pertaining to the use of "Link" header, expressed in Section 3.3.2.1 apply.

The client's handling of an HTTP response with a 3XX status code is not affected by the presence of a "Memento-Datetime" header. The client SHOULD behave in the same manner as it does with HTTP responses with a 3XX status code that do not have a "Memento-Datetime" header.

However, the client MUST be aware that the URI that was selected from the "location" header of an HTTP response with a 3XX status code might not be that of a Memento but rather of an Original Resource. In that case it SHOULD proceed by looking for a Memento of the selected Original Resource.

For example, Figure 11 shows the response to an HTTP GET request for http://a.example.org issued on April 11 2008. This response is archived as a Memento of http://a.example.org, and this Memento's URI is http://arxiv.example.net/web/20080411000650/http://a.example.org. The response to an HTTP GET on this Memento is shown in Figure 12. In essence, it is a replay of the original response with "Memento-Datetime" and "Link" headers added, to allow a client to understand the response is a Memento. In Figure 12, the value of the "Location" header is the same as in the original response; it identifies an Original Resource. The client proceeds with finding a Memento for this Original Resource. Web archives sometimes overwrite the value that was originally provided in the "Location" header in order to point at a Memento they hold of the resource to which the redirect originally led. This is shown in Figure 13. In this case, the client may decide it found an appropriate Memento.

HTTP/1.1 301 Moved Permanently
Date: Fri, 11 Apr 2008 00:06:50 GMT
Server: Apache
Location: http://b.example.org
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Figure 11: Response to the User Agent Request is a Redirect

HTTP/1.1 301 Moved Permanently
Date: Thu, 21 Jan 2010 00:09:40 GMT
Server: Apache-Coyote/1.1
Memento-Datetime: Fri, 11 Apr 2008 00:06:50 GMT
Location: http://b.example.org
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="timemap"; type="application/link-format", 
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/web/20000915112826/http://a.example.org>
   ; rel="first memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
 <http://arxiv.example.net/web/20080708093433/http://a.example.org>
   ; rel="last memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/web/20080411000650/http://a.example.org>
   ; rel="memento"; datetime="Fri, 11 Apr 2008 00:06:50 GMT",
 <http://arxiv.example.net/web/20080410203051/http://a.example.org>
   ; rel="prev memento"; datetime="Thu, 10 Apr 2008 20:30:51 GMT",
 <http://arxiv.example.net/web/20080412204733/http://a.example.org> 
   ; rel="next memento"; datetime="Sat, 12 Apr 2008 20:47:33 GMT"
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Figure 12: Response to a User Agent Request for a Memento of a Redirect; leads to an Original Resource

HTTP/1.1 301 Moved Permanently
Date: Thu, 21 Jan 2010 00:09:40 GMT
Server: Apache-Coyote/1.1
Memento-Datetime: Fri, 11 Apr 2008 00:06:50 GMT
Location: 
 http://arxiv.example.net/web/20080411000655/http://b.example.org
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="timemap"; type="application/link-format", 
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/web/20000915112826/http://a.example.org>
   ; rel="first memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
 <http://arxiv.example.net/web/20080708093433/http://a.example.org>
   ; rel="last memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/web/20080411000650/http://a.example.org>
   ; rel="memento"; datetime="Fri, 11 Apr 2008 00:06:50 GMT",
 <http://arxiv.example.net/web/20080410203051/http://a.example.org>
   ; rel="prev memento"; datetime="Thu, 10 Apr 2008 20:30:51 GMT",
 <http://arxiv.example.net/web/20080412204733/http://a.example.org> 
   ; rel="next memento"; datetime="Sat, 12 Apr 2008 20:47:33 GMT"
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Figure 13: Response to a User Agent Request for a Memento of a Redirect; leads to a Memento

3.3.2.3. Memento of Responses with Other HTTP Status Codes

Cases exist in which responses with 4xx and 5xx HTTP status codes are archived. If the Memento requested by the user agent is an archived version of such an HTTP response, the server's response MUST have the same 4xx or 5xx HTTP status code, and it MUST include a "Memento-Datetime" header with a value equal to the archival datetime of the original response. All other considerations, e.g. pertaining to the use of "Link" header, expressed in Section 3.3.2.1 apply.

For example, Figure 14 shows the 404 response to an HTTP GET request for http://a.example.org issued on April 11 2008. This response is archived as a Memento of http://a.example.org, and this Memento's URI is http://arxiv.example.net/web/20080411000650/http://a.example.org. The response to an HTTP HEAD/GET on this Memento is shown in Figure 15. It is a replay of the original response with "Memento-Datetime" and "Link" headers added, to allow a client to understand the response is a Memento.

HTTP/1.1 404 Not Found
Date: Fri, 11 Apr 2008 00:06:50 GMT
Server: Apache
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Figure 14: Response to the User Agent Request is a 404

HTTP/1.1 404 Not Found
Date: Thu, 21 Jan 2010 00:09:40 GMT
Server: Apache-Coyote/1.1
Memento-Datetime: Fri, 11 Apr 2008 00:06:50 GMT
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="timemap"; type="application/link-format", 
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/web/20000915112826/http://a.example.org>
   ; rel="first memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
 <http://arxiv.example.net/web/20080708093433/http://a.example.org>
   ; rel="last memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/web/20080411000650/http://a.example.org>
   ; rel="memento"; datetime="Fri, 11 Apr 2008 00:06:50 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="prev memento"; datetime="Thu, 10 Apr 2008 20:30:51 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org> 
   ; rel="next memento"; datetime="Sat, 12 Apr 2008 20:47:33 GMT"
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Figure 15: Response to a User Agent Request for a Memento of a 404 Response

3.3.2.4. Mementos Without a TimeGate

Cases may occur in which a server that hosts Mementos does not expose a TimeGate for those Mementos. This can, for example, be the case if the server's Mementos result from taking a snapshot of the state of a set of Original Resources from another server at the time this other server is being retired. As a result, only a single Memento per Original Resource is hosted, making the introduction of a TimeGate unnecessary. But it may also be the case for servers that hosts multiple Mementos for an Original Resource but consider exposing TimeGates too expensive.

In cases of Mementos without associated TimeGates, responses to a request for a Memento by a user agent MUST contain a "Memento-Datetime" response header with a value that corresponds to archival datetime of the Memento. The use of an HTTP "Link" header to convey Memento Relation Types is OPTIONAL.

Figure 16 illustrates the server's response to the request issued against a Memento in Step 5 (Figure 9) for the case that Memento has no associated TimeGate. In this example, it is also assumed there is only one Memento for the Original Resource, and hence the links with Relation Types "memento", "first", "last" all point at the same - responding - Memento.

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:09:40 GMT
Server: Apache-Coyote/1.1
Memento-Datetime: Tue, 11 Sep 2001 20:36:10 GMT
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="first last memento"
   ; datetime="Tue, 15 Sep 2000 11:28:26 GMT"
Content-Length: 23364
Content-Type: text/html;charset=utf-8
Connection: close

Figure 16: Server of Memento without TimeGate responds

Note that a server issuing a response similar to that of Figure 16 does not imply that there is no server whatsoever that exposes a TimeGate; it merely means that the responding server neither provides nor is aware of the location of a TimeGate.

3.3.2.5. Memento Does not Exist

Cases may occur in which a TimeGate's response (Step 4) points at a Memento that actually does not exist, resulting in a user agent's request (Step 5) for a non-existent Memento. In this case, the server's response MUST have the expected "404 Not Found" HTTP Status Code and it MUST NOT contain a "Memento-Datetime" header. Note that the absence of a Memento in an archive is distinct from the case of an archived response with a "404 Not Found" HTTP status code as is described in Section 3.3.2.3

3.3.3. Recognizing a Memento

When following the redirection provided by a confirmed TimeGate (see Section 3.2.3), a user agent SHOULD NOT assume that the targeted resource effectively is a Memento and hence will behave as described in Section 3.3.2.

A user agent MUST decide it has reached a Memento if the response to an HTTP request against the targeted resource's URI contains a "Memento-Datetime" header with a legitimate value. If the response does not, the following applies:

  • If the response contains a redirection that includes an "original" link in the HTTP "Link" header, the user agent MUST conclude it has reached a Redirecting Resource that is part of the Memento infrastructure and SHOULD follow the redirect. Note that a chain of such redirections is possible, e.g. URI-G -> URI-1 (with rel="original") -> URI-2 (with rel="original") -> ... -> URI-M
  • If the response contains no redirection or if a redirection (chain) does not lead to a resource that provides a "Memento-Datetime" header, the user agent SHOULD attempt to determine an appropriate TimeGate for the Original Resource, either automatically or interactively supported by the user. The discovery mechanisms described in Section 4 can support the user agent with this regard.

3.4. Interactions with a Redirecting Resource

A Redirecting Resource is a resource that issues a redirect to a TimeGate, to a Memento, or to another Redirecting Resource, and thus plays an active role in the Memento infrastructure. An example of a Redirecting Resource is a resource that acts as a front-end to multiple TimeGates and redirects to an appropriate TimeGate based on the value of the user agent's "Accept-Datetime" header.

A user agent MUST decide that it has reached a Redirecting Resource if the response to an HTTP GET issued against the resource's URI has an HTTP status code indicative of redirection, has an "original" link in the HTTP "Link" header, yet does not have a "Memento-Datetime" nor "Vary" header. A user agent SHOULD follow the redirection provided by a Redirecting Resource.

If, for example, a user agent follows a link to an assumed TimeGate and attempts the datetime negotiation illustrated in Figure 7, it will expect a response from a TimeGate, which must include a "Vary" header. Instead, it receives the response shown in Figure 17. This response has an HTTP "302 Found" status code, neither a "Vary" nor "Memento-Datetime" header, yet it does have an "original" link in the HTTP "Link" header pointing at the Original Resource, indicating it is a Redirecting Resource.

HTTP/1.1 302 Found
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
Location: 
 http://arxiv.example.net/new-timegate/http://a.example.org
Link: <http://a.example.org>; rel="original"
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Figure 17: Redirecting Resource redirects to a TimeGate

3.5. Interactions with a TimeMap

A TimeMap is introduced to support retrieving a comprehensive list of all Mementos for a specific Original Resource known to a responding server. The entity-body of a response to an HTTP GET request issued against a TimeMap's URI:

  • MUST list the URI of the Original Resource that the TimeMap is about;
  • MUST preferably list the URI and datetime of each Memento for the Original Resource known to the responding server, or, alternatively support assembling such a list by following links with a "timemap" Relation Type provided in the TimeMap;
  • SHOULD list the URI of one or more TimeGates for the Original Resource known to the responding server;
  • SHOULD, for self-containment, list the URI of the TimeMap itself;
  • MUST unambiguously type listed resources as being Original Resource, TimeGate, Memento, or TimeMap.

The entity-body of a response from a TimeMap MAY be serialized in various ways, but the link-value format serialization MUST be supported. In this serialization, the entity-body MUST be formatted in the same way as the value of an HTTP "Link" header, and hence MUST comply to the "link-value" construction rule of "Section 5. The Link Header Field" of [RFC5988], and the media type of the entity-body MUST be "application/link-format" as introduced in [RFC6690]. Links contained in the entity-body MUST be interpreted as follows:

  • The Context IRI is set to the anchor parameter, when specified;
  • The Context IRI of links with the "self" Relation Types is the URI of the TimeMap, i.e. the URI of the resource from which the TimeMap was requested;
  • The Context IRI of all other links is the URI of the Original Resource, which is provided as the Target IRI of the link with an "original" Relation Type.

3.5.1. User Agent Requests a TimeMap

In order to retrieve the link-value serialization of a TimeMap, a user agent SHOULD use an "Accept" request header with a value set to "application/link-format". This is shown in Figure 18.

GET /timemap/http://a.example.org HTTP/1.1
Host: arxiv.example.net
Accept: application/link-format;q=1.0
Connection: close

Figure 18: Request for a TimeMap

3.5.2. Server Responds to a Request for a TimeMap

If the TimeMap requested by the user agent exists, the server's response MUST have a "200 OK" HTTP status code (or "206 Partial Content", where appropriate). Note that a TimeMap is itself an Original Resource for which Mementos may exist. For example, a response from a TimeMap could provide a "timegate" Link to a TimeGate via which prior TimeMap versions are available. In this case, the use of the "Link" header is subject to all considerations described in Section 2.2, with the TimeMap acting as the Original Resource.

However, in case a TimeMap wants to explicitly indicate in its response headers for which Original Resource it is a TimeMap, it MUST do so by including an HTTP "Link" header with the following characteristics:

  • The Context IRI for the HTTP Link is the URI of the Original Resource;
  • The Relation Type is "timemap";
  • The Target IRI for the HTTP Link is the URI of the TimeMap.

Because the Context IRI of this HTTP Link is not the URI of the TimeMap, as per [RFC5988], the default Context IRI must be overwritten by using the "anchor" attribute with a value of the URI of the Original Resource.

The response from the TimeMap to the request of Figure 18 is shown in Figure 19. The response header shows the TimeMap explicitly conveying the URI of the Original Resource for which it is a TimeMap; for practical reasons the entity-body in the example has been abbreviated. Note the use of the "self" Relation Type to express the URI of the TimeMap itself. Note also the use of the "from" and "until" attributes on that link to express the datetime of the earliest and most recent Memento listed in the TimeMap. And, note the use of the "license" attribute introduced in Section 2.2.1.4 on the "memento" links in the TimeMap.

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
Link: <http://arxiv.example.net/timemap/http://a.example.org> 
   ; anchor="http://a.example.org"; rel="timemap"
   ; type="application/link-format"
Content-Length: 4883
Content-Type: application/link-format
Connection: close

 <http://a.example.org>;rel="original",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="self";type="application/link-format"
   ; from="Tue, 20 Jun 2000 18:02:59 GMT"
   ; until="Wed, 09 Apr 2008 20:30:51 GMT",
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/web/20000620180259/http://a.example.org>
   ; rel="first memento";datetime="Tue, 20 Jun 2000 18:02:59 GMT"
   ; license="http://creativecommons.org/publicdomain/zero/1.0/", 
 <http://arxiv.example.net/web/20091027204954/http://a.example.org>
    ; rel="last memento";datetime="Tue, 27 Oct 2009 20:49:54 GMT"
    ; license="http://creativecommons.org/publicdomain/zero/1.0/",
 <http://arxiv.example.net/web/20000621011731/http://a.example.org>
   ; rel="memento";datetime="Wed, 21 Jun 2000 01:17:31 GMT"
   ; license="http://creativecommons.org/publicdomain/zero/1.0/",
 <http://arxiv.example.net/web/20000621044156/http://a.example.org>
   ; rel="memento";datetime="Wed, 21 Jun 2000 04:41:56 GMT"
   ; license="http://creativecommons.org/publicdomain/zero/1.0/",
 ...

Figure 19: Response from a TimeMap

Cases exist in which a TimeMap points at several other TimeMaps. In one such case, a TimeMap could merely point at other TimeMaps and not list any Mementos itself. This can happen when Mementos are spread across several archives that share a front-end. Such a TimeMap can be considered an index of TimeMaps and is shown in Figure 20. Another case is when the number of available Mementos requires introducing multiple TimeMaps that can be paged. Such a TimeMap is shown in Figure 21. Note that this TimeMap contains links to other TimeMaps but actually also lists Mementos. In both examples, the "timemap" links have the "from" and "until" attributes to express the temporal span of listed Mementos. Since the index TimeMap does not list any Mementos, these attributes are not used on the link that points at the index TimeMap itself, i.e. the link with the "self" Relation Type. Note that TimeMaps obtained by following a "timemap" link can contain links to further TimeMaps.

<http://a.example.org>;rel="original",
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="self";type="application/link-format",
 <http://arxiv1.example.net/timemap/http://a.example.org>
   ; rel="timemap";type="application/link-format"
   ; from="Wed, 21 Jun 2000 04:41:56 GMT"
   ; until="Wed, 09 Apr 2008 20:30:51 GMT",
 <http://arxiv2.example.net/timemap/http://a.example.org>
   ; rel="timemap";type="application/link-format"
   ; from="Thu, 10 Apr 2008 20:30:51 GMT"
   ; until="Tue, 27 Oct 2009 20:49:54 GMT",
 <http://arxiv3.example.net/timemap/http://a.example.org>
   ; rel="timemap";type="application/link-format"
   ; from="Thu, 29 Oct 2009 20:30:51 GMT"

Figure 20: An index TimeMap

<http://a.example.org>;rel="original",
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/timemap/1/http://a.example.org>
   ; rel="self";type="application/link-format"
   ; from="Tue, 20 Jun 2000 18:02:59 GMT"
   ; until="Wed, 09 Apr 2008 20:30:51 GMT",
 <http://arxiv.example.net/timemap/2/http://a.example.org>
   ; rel="timemap";type="application/link-format"
   ; from="Thu, 10 Apr 2008 20:30:51 GMT"
   ; until="Tue, 27 Oct 2009 20:49:54 GMT",
 <http://arxiv.example.net/timemap/3/http://a.example.org>
   ; rel="timemap";type="application/link-format"
   ; from="Thu, 29 Oct 2009 20:30:51 GMT"
   ; until="Fri, 31 Aug 2012 12:22:34 GMT"
 <http://arxiv.example.net/web/20000620180259/http://a.example.org>
   ; rel="memento";datetime="Tue, 20 Jun 2000 18:02:59 GMT", 
 <http://arxiv.example.net/web/20000621011731/http://a.example.org>
   ; rel="memento";datetime="Wed, 21 Jun 2000 01:17:31 GMT",
 <http://arxiv.example.net/web/20000621044156/http://a.example.org>
   ; rel="memento";datetime="Wed, 21 Jun 2000 04:41:56 GMT",
 ...

Figure 21: TimeMap Paging

4. The Memento Framework, Discovery of TimeGates, TimeMaps, Mementos

Section 3 describes how TimeGates, Mementos, Original Resources, and TimeMaps can be discovered by following HTTP Links with Relation Types "timegate", "memento", "original", and "timemap", respectively.

Naturally, some of these links can also be included in representations of resources that have a media type that allows embedding typed links. For example, an Original Resource that has an HTML representation can include a "timegate" link by using HTML's LINK element, e.g. <link href="http://arxiv.example.net/timegate/http://a.example.org" rel="timegate">. The use of such embedded links is also subject to the considerations of Section 2.2.

In this section an additional approach is introduced to support batch discovery of TimeGates, TimeMaps, and Mementos. The approach leverages the well-known URI [RFC5785] and host-meta [RFC6415] specifications. It can be used in cases where URIs of TimeGates, TimeMaps, or Mementos have as part of their path-component the URI of the associated Original Resource. The approach uses the following variables for URI templates used in host-meta files:

  • The "uri" variable, as reserved by [RFC6415], which stands for any URI in the server's document scope.
  • The "anyuri" variable, introduced here, which stands for any URI, i.e. not just URIs in the server's document scope.

A server SHOULD make TimeGates discoverable in the following way:

  • As per [RFC5785] and [RFC6415], the server publishes a document with the name "host-meta" in its /.well-known/ path.
  • As per [RFC6415], the host-meta document uses the XRD 1.0 document format.
  • The host-meta document includes one or more "Link" elements to support discovery of TimeGates. Each such Link element has a value of "timegate" for its "rel" attribute, and it has a "template" attribute that has a URI template as its value. The URI template MAY either use the "uri" or "anyuri" variable.
  • The URI template MUST be such that, when applying the URI of an Original Resource to the "uri" or "anyuri" variable the URI of a potential TimeGate for that Original Resource results.

There are no guarantees that these resulting TimeGates exist, but URI templates should be created to try and maximize chances they do.

The considerations described for TimeGate discovery also apply to TimeMap discovery. However, for TimeMaps, each Link element has a value of "timemap" for its "rel" attribute.

Figure 22 illustrates how a server makes TimeGates associated with its own Original Resources discoverable, whereas Figure 23 illustrates how a web archive that hosts Mementos for Original Resources originating from various domains makes its TimeGates and TimeMaps discoverable.

Figure 22 shows a host-meta document published at http://a.example.org/.well-known/host-meta by a wiki that has http://a.example.org as its base URI. The document supports discovery of TimeGates that allow accessing prior versions of the wiki's Original Resources. The URI template uses the "uri" variable to stand for a document in the wiki's document range. Assuming http://a.example.org/w/Clock is the URI of an Original Resource on the wiki, applying the template to that URI yields http://a.example.org/Special:TimeGate/http://a.example.org/w/Clock as its corresponding TimeGate.

<?xml version='1.0' encoding='UTF-8'?>
 <XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'
       xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'>
  <Link rel='timegate' 
        template='http://a.example.org/Special:TimeGate/{uri}'>
  </Link>
 </XRD>

Figure 22: A host-meta Document Supporting TimeGate Discovery for a Wiki's Original Resources

Figure 23 shows a host-meta document published at http://arxiv.example.net/.well-known/host-meta by a web archive that has http://arxiv.example.net/ as its base URI. The document supports discovery of TimeGates and TimeMaps that the archive exposes, and that are associated with Original Resources for which the archive holds Mementos. The URI templates use the "anyuri" variable, which is an indication that the web archive holds Mementos for a wide variety of domains. Note the use of the "from" and "until" attributes to express the time range for which the archive has Mementos.

<?xml version='1.0' encoding='UTF-8'?>
 <XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'
      xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'>
  <Link rel='timegate' 
        template='http://arxiv.example.net/timegate/{anyuri}'>
  </Link>
  <Link rel='timemap' 
        template='http://arxiv.example.net/timemap/{anyuri}'
        from='Tue, 20 Jun 2000 18:02:59 GMT'
        until='Wed, 09 Apr 2008 20:30:51 GMT'>
  </Link>
 </XRD>

Figure 23: A host-meta Document Supporting Discovery of TimeGates and TimeMaps Exposed by a Web Archive

In some cases, Mementos can be made discoverable via a host-meta document. Consider, for example, a server with base URI http://a.example.org/ that archives its own content once a year and makes the result available in a snapshot archive with base URI http://arxiv.example.net/. The server can publish a host-meta document at http://a.example.org/.well-known/host-meta that points to Mementos contained in the annual snapshots. This is shown in Figure 24. Note also the use of the "license" attribute that associates a Creative Commons license with the Mementos in the archive.

<?xml version='1.0' encoding='UTF-8'?>
 <XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'
      xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'>
  <Link rel='memento' 
   template='http://arxiv.example.net/2010/{uri}'
   license='http://creativecommons.org/publicdomain/zero/1.0/'>
  </Link>
  <Link rel='memento' 
   template='http://arxiv.example.net/2011/{uri}'
   license='http://creativecommons.org/publicdomain/zero/1.0/'>
  </Link>
 </XRD>

Figure 24: A Server Pointing to its Mementos in a Snapshot Archive

5. IANA Considerations

This memo requires IANA to register the Accept-Datetime and Memento-Datetime HTTP headers defined in Section 2.1.1 in the appropriate IANA registry.

This memo requires IANA to register the Relation Types "original", "timegate", "timemap", and "memento" defined in Section 2.2.1 in the appropriate IANA registry.

This memo requires IANA to register the "datetime" and "license" attributes for the "memento" Relation Type, as defined in Section 2.2.1.4, in the appropriate IANA registry.

This memo requires IANA to register the "from" and "until" attributes for the "timemap" Relation Type, as defined in Section 2.2.1.4, in the appropriate IANA registry.

6. Security Considerations

Provision of a "timegate" HTTP "Link" header in responses to requests for an Original Resource that is protected (e.g., 401 or 403 HTTP response codes) is OPTIONAL. The inclusion of this Link when requesting authentication is at the server's discretion; cases may exist in which a server protects the current state of a resource, but supports open access to prior states and thus chooses to supply a "timegate" HTTP "Link" header. Conversely, the server may choose to not advertise the TimeGate URIs (e.g., they exist in an intranet archive) for unauthenticated requests.

The veracity of archives and the relationships between Original Resources and Mementos is beyond the scope of this document. Even in the absence of malice, it is possible for separate archives to have different Mementos for the same Original Resource at the same datetime if the state of the Original Resource was dependent on the requesting archive's user agent IP address, specific HTTP request headers, and possibly other factors.

Further authentication, encryption and other security related issues are otherwise orthogonal to Memento.

7. Changelog

v05 2012-09-01 HVDS MLN RS draft-vandesompel-memento-05

  • Clarified the section on Memento Relation Types.
  • Re-introduced "license" attribute for "memento" Relation Type as it will become essential for IIPC.
  • Introduced from and until attributes for "timemap" links to accomodate paged TimeMap cases.
  • Introduced the notion of Redirecting Resource and inserted related information in various sections.
  • Added discovery of Mementos via host-meta.
  • Corrected ambiguous uses of the term "representation".

v04 2012-05-18 HVDS MLN RS draft-vandesompel-memento-04

  • Removed the possibility to use an interval indicator in an Accept-Datetime header as no one is implementing it.
  • Corrected typo in Other Relation Types table.
  • Added TimeMap examples to illustrate index of TimeMaps and TimeMap paging.
  • Changed Discovery component from using robots.txt with Memento-specific add-ons to well-known URI and host-meta.
  • Removed "embargo" and "license" attributes for links with a "memento" Relation Type because no one is using them.

v04 2011-12-20 HVDS MLN RS draft-vandesompel-memento-03

  • Added description of Mementos of HTTP responses with 3XX, 4XX and 5XX status code.
  • Clarified that a TimeGate must not use the "Memento-Datetime" header.
  • Added wording to warn for possible cache problems with Memento implementations that choose to have an Original Resource and and its TimeGate coincide.

v03 2011-05-11 HVDS MLN RS draft-vandesompel-memento-02

  • Added scenario in which a TimeGate redirects to another TimeGate.
  • Reorganized TimeGate section to better reflect the difference between requests with and without interval indicator.
  • Added recommendation to provide "memento" links to Mementos in the vicinity of the preferred interval provided by the client, in case of a 406 response.
  • Removed TimeMap Feed material from the Discovery section as a result of discussions regarding (lack of) scalability of the approach with representatives of the International Internet Preservation Consortium. An alternative approach to support batch discovery of Mementos will be specified.

v02 2011-04-28 HVDS MLN RS draft-vandesompel-memento-01

  • Introduced wording and reference to indicate a Memento is a FixedResource.
  • Introduced "Sticky Memento-Datetime" notion and clarified wording about retaining "Memento-Datetime" headers and values when a Memento is mirrored at different URI.
  • Introduced section about handling both datetime and regular negotiation.
  • Introduced section about Mementos Without TimeGate.
  • Made various changes in the section Relation Type "memento", including addition of "license" and "embargo" attributes, and clarification of rules regarding the use of "memento" links.
  • Moved section about TimeMaps inside the Datetime Negotiation section, and updated it.
  • Restarted the Discovery section from scratch.

v01 2010-11-11 HVDS MLN RS First public version draft-vandesompel-memento-00

v00 2010-10-19 HVDS MLN RS Limited circulation version

2010-07-22 HVDS MLN First internal version

8. Acknowledgements

The Memento effort is funded by the Library of Congress. Many thanks to Kris Carpenter Negulescu, Michael Hausenblas, Erik Hetzner, Larry Masinter, Gordon Mohr, Mark Nottingham, David Rosenthal, Ed Summers for early feedback. Many thanks to Samuel Adams, Scott Ainsworth, Lyudmilla Balakireva, Frank McCown, Harihar Shankar, Brad Tofel for early implementations.

9. References

9.1. Normative References

[RFC5988] Nottingham, M., "Web Linking", RFC 5988, October 2010.
[RFC6690] Shelby, Z., "Constrained RESTful Environments (CoRE) Link Format", RFC 6690, August 2012.
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC5829] Brown, A., Clemm, G. and J. Reschke, "Link Relation Types for Simple Version Navigation between Web Resources", RFC 5829, April 2010.
[RFC4287] Nottingham, M. and R. Sayre, "The Atom Syndication Format", RFC 4287, December 2005.
[RFC4151] Kindberg, T. and S. Hawke, "The 'tag' URI Scheme", RFC 4151, October 2005.
[RFC5785] Nottingham, M. and E. Hammer-Lahav, "Defining Well-Known Uniform Resource Identifiers (URIs)", RFC 5785, April 2010.
[RFC6415] Hammer-Lahav, E. and B. Cook, "Web Host Metadata", RFC 6415, October 2011.

9.2. Informative References

[RFC1123] Braden, R., "Requirements for Internet Hosts - Application and Support", STD 3, RFC 1123, October 1989.
[I-D.masinter-dated-uri] Masinter, L, "The 'tdb' and 'duri' URI schemes, based on dated URIs", Internet-Draft draft-masinter-dated-uri-10, January 2012.
[W3C.gen-ont-20090420] Berners-Lee, , "Architecture of the World Wide Web", April 2009.
[W3C.REC-aww-20041215] Jacobs, and Walsh, "Architecture of the World Wide Web", December 2004.
[Fitch] Fitch, , "Web site archiving - an approach to recording every materially different response produced by a website", July 2003.

Appendix A. Appendix: A Sample, Successful Memento Request/Response cycle

Step 1 : UA --- HTTP HEAD/GET; Accept-Datetime: Tj ---------> URI-R
      
HEAD / HTTP/1.1
Host: a.example.org
Accept-Datetime: Tue, 11 Sep 2001 20:35:00 GMT
Connection: close

Step 2 : UA <-- HTTP 200; Link: URI-G ----------------------- URI-R

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Link: <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate"
Content-Length: 255
Connection: close
Content-Type: text/html; charset=iso-8859-1

Step 3 : UA --- HTTP HEAD/GET; Accept-Datetime: Tj ---------> URI-G

GET /timegate/http://a.example.org 
 HTTP/1.1
Host: arxiv.example.net
Accept-Datetime: Tue, 11 Sep 2001 20:35:00 GMT
Connection: close

Step 4 : UA <-- HTTP 302; Location: URI-Mj; Vary; Link:
    URI-R, URI-T, URI-M0, URI-Mn, URI-Mi, URI-Mj, URI-Mk ---- URI-G
    
HTTP/1.1 302 Found
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
Vary: negotiate, accept-datetime
Location: 
 http://arxiv.example.net/web/20010911203610/http://a.example.org
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/web/20000915112826/http://a.example.org>
   ; rel="first memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
 <http://arxiv.example.net/web/20080708093433/http://a.example.org>
   ; rel="last memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="timemap"; type="application/link-format",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="memento"; datetime="Tue, 11 Sep 2001 20:36:10 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="prev memento"; datetime="Tue, 11 Sep 2001 20:30:51 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org> 
   ; rel="next memento"; datetime="Tue, 11 Sep 2001 20:47:33 GMT"
Content-Length: 0
Content-Type: text/plain; charset=UTF-8
Connection: close

Step 5 : UA --- HTTP GET URI-Mj; Accept-Datetime: Tj -------> URI-Mj

GET /web/20010911203610/http://a.example.org 
 HTTP/1.1
Host: arxiv.example.net
Accept-Datetime: Tue, 11 Sep 2001 20:35:00 GMT
Connection: close

Step 6 : UA <-- HTTP 200; Memento-Datetime: Tj; Link: URI-R, 
    URI-T, URI-G, URI-M0, URI-Mn, URI-Mi, URI-Mj, URI-Mk ---- URI-Mj

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:09:40 GMT
Server: Apache-Coyote/1.1
Memento-Datetime: Tue, 11 Sep 2001 20:36:10 GMT
Link: <http://a.example.org>; rel="original",
 <http://arxiv.example.net/web/20000915112826/http://a.example.org>
   ; rel="first memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
 <http://arxiv.example.net/web/20080708093433/http://a.example.org>
   ; rel="last memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
 <http://arxiv.example.net/timemap/http://a.example.org>
   ; rel="timemap"; type="application/link-format", 
 <http://arxiv.example.net/timegate/http://a.example.org>
   ; rel="timegate",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="memento"; datetime="Tue, 11 Sep 2001 20:36:10 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org>
   ; rel="prev memento"; datetime="Tue, 11 Sep 2001 20:30:51 GMT",
 <http://arxiv.example.net/web/20010911203610/http://a.example.org> 
   ; rel="next memento"; datetime="Tue, 11 Sep 2001 20:47:33 GMT"
Content-Length: 23364
Content-Type: text/html;charset=utf-8
Connection: close

Authors' Addresses

Herbert VandeSompel Los Alamos National Laboratory PO Box 1663 Los Alamos, New Mexico 87545 USA Phone: +1 505 667 1267 EMail: hvdsomp@gmail.com URI: http://public.lanl.gov/herbertv/
Michael Nelson Old Dominion University Norfolk, Virginia 23529 USA Phone: +1 757 683 6393 EMail: mln@cs.odu.edu URI: http://www.cs.odu.edu/~mln/
Robert Sanderson Los Alamos National Laboratory PO Box 1663 Los Alamos, New Mexico 87545 USA Phone: +1 505 665 5804 EMail: azaroth42@gmail.com URI: http://public.lanl.gov/rsanderson/