Internet DRAFT - draft-kundrat-incthread
draft-kundrat-incthread
Internet Engineering Task Force J. Kundrat
Internet-Draft August 27, 2013
Intended status: Standards Track
Expires: February 26, 2014
IMAP Extension for Incremental Threading (INCTHREAD)
draft-kundrat-incthread-02
Abstract
This document describes the INCTHREAD IMAP extension which enables
clients to retrieve incremental updates of the mailbox threading.
The extension repurposes the ESEARCH response for passing along the
threading information and builds on top of Arnt Gulbrandsen's work on
the INTHREAD search key. The UID THREAD command is also extended to
allow activating this extension. Together, these changes make it
possible for clients not to fetch the complete mailbox threading each
time a new message arrives.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 26, 2014.
Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
Kundrat Expires February 26, 2014 [Page 1]
Internet-Draft IMAP INCTHREAD Extension August 2013
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Drawbacks and Alternatives . . . . . . . . . . . . . . . . 3
1.2. Requirements Language . . . . . . . . . . . . . . . . . . 4
2. IMAP Protocol Changes . . . . . . . . . . . . . . . . . . . . 4
2.1. New SEARCH Keys . . . . . . . . . . . . . . . . . . . . . 4
2.2. Modified IMAP Commands . . . . . . . . . . . . . . . . . . 4
2.2.1. Modified UID THREAD Command . . . . . . . . . . . . . 4
2.3. ESEARCH Extension . . . . . . . . . . . . . . . . . . . . 5
2.3.1. The THREAD ESEARCH Return Value . . . . . . . . . . . 5
2.3.2. The INCTHREAD ESEARCH Return Value . . . . . . . . . . 5
2.3.2.1. Processing Incremental Threading Updates . . . . . 5
2.4. New Capabilities . . . . . . . . . . . . . . . . . . . . . 6
3. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. General Mode of Operation . . . . . . . . . . . . . . . . 6
3.2. Inserting a Single Message . . . . . . . . . . . . . . . . 7
3.3. Joining Threads . . . . . . . . . . . . . . . . . . . . . 8
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 8
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8
6. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 9
7. Security Considerations . . . . . . . . . . . . . . . . . . . 9
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9
8.1. Normative References . . . . . . . . . . . . . . . . . . . 10
8.2. Informative References . . . . . . . . . . . . . . . . . . 10
Appendix A. Changelog . . . . . . . . . . . . . . . . . . . . . . 10
Appendix A.1. Changes in -02 since -01 . . . . . . . . . . . . 10
Appendix A.2. Changes in -01 since -00 . . . . . . . . . . . . 10
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 10
1. Introduction
Online IMAP clients which want to conserve the required bandwidth and
also show messages in threads to their users have an option to
delegate the message threading to the IMAP server through mechanisms
outlines in [RFC5256]. Using the UID SEARCH command, clients do not
have to download the message headers (like the Message-Id, References
and In-Reply-To), and can instead fetch the resulting thread mapping
of all messages in a mailbox.
Kundrat Expires February 26, 2014 [Page 2]
Internet-Draft IMAP INCTHREAD Extension August 2013
Unfortunately, the savings in transferred data are significantly
reduced when clients have to fetch the thread mapping over and over
again, for example when a new message arrives. Even if the client
downloads the relevant headers of new arrivals, these data alone are
not sufficient to determine a proper place where to insert the newly
arriving message. Furthermore, a single newly arriving message could
potentially affect placement of many messages (or even all of them in
a pathological case due to joining of adjacent threads). This issue
prevents using approach similar to the CONTEXT=SEARCH and
CONTEXT=SORT extensions [RFC5267] where only a position of the new
arrival is communicated in an incremental manner.
This extension builds on Arnt Gulbrandsen's work [I-D.ietf-morg-
inthread] and reuses the INTHREAD search key defined in said draft.
This search key is used to inform the server that the search
conditions are to refer to all threads containing any messages which
match the original search criteria. However, as the untagged THREAD
response does not contain any data about the position of the affected
thread among other threads in the mailbox, support for INTHREAD alone
does not relieve the clients from performing additional operations
due to missing information. A naive workaround where the affected
threads are always placed at the logical end of the mailbox would
yield results different from the complete THREAD command when copying
older messages. Similarly, attempting to reuse the original thread
position would significantly limit the usefulness of the REFS
algorithm [I-D.ietf-morg-inthread] which sorts threads with "fresh
messages" at the end of the view.
Because the THREAD response cannot transmit the position of the
resulting thread relative to other threads in the mailbox, the
ESEARCH response [RFC4731] is used and the UID THREAD command is
extended to allow for specifying the return options in manner
consistent with how SEARCH and UID SEARCH were modified by ESEARCH.
Finally, two new ESEARCH return options, the THREAD and the
INCTHREAD, are defined.
These modifications together allow clients to delegate the threading
operation completely to the server side without significantly
increasing network traffic even on busy mailboxes.
1.1. Drawbacks and Alternatives
This extension can still transfer excessive amounts of data because
the commands return complete threads instead of incremental
difference updates. However, this approach allows for reusing the
clients' and servers' existing facilities, both for parsing and
response processing. In addition, unless the protocol mandated
history tracking of the threading tree, a much intrusive and
resource-demanding feature, the incremental updates would be only
Kundrat Expires February 26, 2014 [Page 3]
Internet-Draft IMAP INCTHREAD Extension August 2013
possible in case where the mailbox is currently selected. This
extension affirms the decision about threading requests to remain on
the client side, letting it use its policies about when to request
full threading information and when to use the incremental updates.
No support for automated updates of the threading data in the sense
of the CONTEXT extension [RFC5267] are defined at this point. This
might change based on feedback from other server and client vendors.
1.2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2. IMAP Protocol Changes
2.1. New SEARCH Keys
This document reuses the INTHREAD search key from [I-D.ietf-morg-
inthread] with no changes in its meaning or semantics.
Please note that the "INTHREAD" search key and the "INCTHREAD"
ESEARCH return option are two distinct features.
2.2. Modified IMAP Commands
2.2.1. Modified UID THREAD Command
The UID THREAD command is extended with the following two return
options:
THREAD Return the threading information through the ESEARCH response
using syntax similar to the untagged THREAD response.
INCTHREAD Return a dedicated INCTHREAD section for each thread found
in the result set.
The THREAD and INCTHREAD return options are mutually exclusive.
Servers MUST return a tagged BAD response if a client specifies both
return options in a single UID THREAD command.
Servers MUST use the ESEARCH response instead of the untagged THREAD
response when responding to the extended form of the UID THREAD
command.
The THREAD and UID THREAD are two distinct commands. Because clients
are not expected to rely on transient identifiers like the message
sequence numbers for threading retrieval and storage and because of
the requirement of using UIDs in the INCTHREAD response, no
modification to the THREAD command is defined. This might change in
future iterations of this draft if client authors express sufficient
interest.
Kundrat Expires February 26, 2014 [Page 4]
Internet-Draft IMAP INCTHREAD Extension August 2013
2.3. ESEARCH Extension
Servers announcing the INCTHREAD capability support two new search
return options:
THREAD Method for conveying the threading information in a form
similar to the THREAD untagged response.
INCTHREAD Response contains the threading information along with the
specification of a UID of the previous thread root.
2.3.1. The THREAD ESEARCH Return Value
The THREAD return value uses the same format as the threading data in
the untagged THREAD responses [RFC5256]. This return value is
defined to allow clients to easily match data received over network
with the tag of the command which caused them, as per the usual
ESEARCH rules.
Due to the rules of [RFC4466], the [RFC5256]-styled response is
enclosed in one extra pair of parentheses when returned as ESEARCH
data. As an example, threading data (1)(2 3) are encoded as ((1)(2
3)) when sent in the THREAD ESEARCH response.
2.3.2. The INCTHREAD ESEARCH Return Value
The INCTHREAD return value consists of "previous thread root UID"
followed by "threading response containing a single thread". The UID
of the previous thread root MUST be zero if the thread sorts first in
the resulting list of threads. The tuple is enclosed in parentheses
to conform to [RFC4466].
A dedicated INCTHREAD record MUST be present for each thread
contained in the result set.
2.3.2.1. Processing Incremental Threading Updates
At first, the client removes any messages referenced in the received
INCTHREAD response from its thread mapping. This step is crucial to
allow new arrivals joining previously independent threads together.
In the second step, the client extracts the threading information
from the received INCTHREAD response. The threading data is parsed
as in [RFC5256] and the newly formed thread is inserted just behind a
thread whose root message has UID specified in the "uid" argument of
the INCTHREAD response.
If non-zero, the UID of the previous thread root message MUST refer
to the previous thread in a mapping which contains all messages from
a mailbox. In particular, no matter what additional searching
criteria the client has used, the previous thread MUST always be
Kundrat Expires February 26, 2014 [Page 5]
Internet-Draft IMAP INCTHREAD Extension August 2013
identified without any search criteria being applied.
If the client supports incremental threading, the INCTHREAD blocks of
an ESEARCH response MUST be processed in order. In particular,
inserting the newly formed threads to a proper location shall happen
immediately when the new thread is created because the subsequent
responses COULD refer to the root message of the just-inserted
thread.
If the UID is said to be zero (0), it is interpreted specially to
mean that the newly formed thread is the very first one among the
list of threads in the mailbox. Servers MUST NOT send a number which
does not refer to any thread root UID unless the number is 0 to
indicate the very first thread.
Clients MUST deal with servers sending a UID which does not refer to
any thread root or any message in the mailbox. Is is implementation-
defined at which position such thread shall be inserted, but the
thread MUST appear in the list of threads.
2.4. New Capabilities
This document adds two new IMAP capabilities, the ETHREAD and
INCTHREAD.
Servers announcing the ETHREAD capability support the extended UID
THREAD command syntax and the THREAD return option.
Servers supporting the INCTHREAD capability MUST support and announce
the ETHREAD capability as well.
3. Examples
This section contains a few examples to illustrate how the INCTHREAD
extension operates.
3.1. General Mode of Operation
Using the proposed extension, a typical communication between two
compliant IMAP protocol speakers might look like the following:
S: * 666 EXISTS
C: x1 UID FETCH 665:* (FLAGS)
S: * 666 FETCH (UID 1666 FLAGS ())
S: x1 OK fetched
C: x2 UID THREAD RETURN (INCTHREAD) REFS utf-8
INTHREAD REFS 666
S: * ESEARCH (TAG "x2") UID INCTHREAD (400
Kundrat Expires February 26, 2014 [Page 6]
Internet-Draft IMAP INCTHREAD Extension August 2013
(600 601 (640 666)(602 603)))
S: x2 OK sent
The actual resulting message threading looks like the following:
...
|
| (All preceding threads are simply skipped.)
|
+-- 400
| +-- ...
| (No data for the previous thread besides its root node is sent.)
|
+-- 600
| +-- 601
| +-- 640
| | +-- 666 <-- the new arrival
| +-- 602
| +-- 603
|
... (No data about any subsequent threads is included in the response.)
3.2. Inserting a Single Message
Consider the following threading for a mailbox:
C: x1 UID THREAD (RETURN THREAD) REFS utf-8 ALL
S: * ESEARCH (TAG "x2") UID THREAD ((1)(2)(3)(4))
S: x1 OK Threading sent
Such a response corresponds to the following threading:
+-- 1
+-- 2
+-- 3
+-- 4
A new message arrives and the client asks for the reading
information:
S: * 5 EXISTS
S: * 5 FETCH (UID 5)
C: x2 UID THREAD (RETURN INCTHREAD) REFS utf-8 INTHREAD REFS UID 5
S: * ESEARCH (TAG "x2") UID INCTHREAD (2 (3 5))
S: x2 OK Threading sent
The updated threading information should look like the following:
+-- 1
+-- 2
+-- 3
| +-- 5
+-- 4
Kundrat Expires February 26, 2014 [Page 7]
Internet-Draft IMAP INCTHREAD Extension August 2013
In this case, the thread with thread root with UID 4 is still
considered "fresher" by the selected thread algorithm.
3.3. Joining Threads
The following example shows a more complicated scenario where
independent threads are joined together. This illustrates the need
for clients to remove the referenced messages from their thread
mapping:
C: x1 UID THREAD (RETURN THREAD) REFS utf-8 ALL
S: * ESEARCH (TAG "x2") UID THREAD ((1 2)(3 4)(5))
S: x1 OK Threading sent
Such a response corresponds to the following threading:
+-- 1
| +-- 2
+-- 3
| +-- 4
+-- 5
A new response arrives, joining the first two threads in the mailbox
together:
S: * 6 EXISTS
S: * 6 FETCH (UID 6)
C: x2 UID THREAD (RETURN INCTHREAD) REFS utf-8 INTHREAD REFS UID 6
S: * ESEARCH (TAG "x2") UID INCTHREAD (0 (6 (1 2)(3 4)))
S: x2 OK Threading sent
The newly formed thread remains at the beginning of a mailbox:
+-- 6 <-- new arrival
| +-- 1 <-- previous thread #1
| | +-- 2
| +-- 3 <-- previous thread #2
| +-- 4
+-- 5 <-- previous thread #3 remains as a standalone thread
4. Acknowledgements
This extension builds upon the SEARCH=INTHREAD extension [I-D.ietf-
morg-inthread] and the THREAD extension [RFC5256].
5. IANA Considerations
IMAP4 capabilities are registered by publishing a standards track or
IESG approved experimental RFC. The registry is currently located
at:
http://www.iana.org/assignments/imap4-capabilities
Kundrat Expires February 26, 2014 [Page 8]
Internet-Draft IMAP INCTHREAD Extension August 2013
This document defines the ETHREAD and INCTHREAD IMAP capabilities.
IANA will be asked to add these capability to the registry.
6. Formal Syntax
The following syntax specification uses the Augmented Backus-Naur
Form (ABNF) notation as specified in [RFC5234].
Non-terminals referenced but not defined below are as defined by
[RFC3501], [RFC4466], [RFC4731], or [RFC5256].
capability =/ "ETHREAD" / "INCTHREAD"
;; <capability> from [RFC3501]
modifier-ethread = "THREAD"
modifier-incthread = "INCTHREAD"
thread-return-opt = modifier-thread / modifier-incthread
;; Similar to <search-return-opt> from [RFC4466]
ret-data-thread = "THREAD" [SP "(" 1*thread-list ")" ]
;; <thread-list> from [RFC5256]
ret-data-incthread = "INCTHREAD" SP "(" uid SP thread-list ")"
;; <uid> from [RFC3501]
;; <thread-list> from [RFC5256]
search-return-data =/ ret-data-thread / ret-data-incthread
;; <search-return-data> from [RFC4466]
thread =/ "UID" SP "THREAD" [thread-return-opts]
SP thread-alg SP search-criteria
;; <thread> and <thread-alg> from [RFC5256]
;; <search-criteria> from [RFC3501] as amended by
;; [I-D.ietf-morg-inthread]
;;
;; The thread-return-opts MUST contain exactly one of
;; modifier-thread or modifier-incthread
thread-return-opts = SP "RETURN" SP "(" [thread-return-opt
*(SP thread-return-opt)] ")"
;; similar to the <search-return-opts> from [RFC4466]
7. Security Considerations
This document is believed to not have any security implications
besides those already implied by [RFC5256] and [I-D.ietf-morg-
inthread].
8. References
Kundrat Expires February 26, 2014 [Page 9]
Internet-Draft IMAP INCTHREAD Extension August 2013
8.1. Normative References
[I-D.ietf-morg-inthread]
Gulbrandsen, A. and a. them, "The IMAP SEARCH=INTHREAD
and THREAD=REFS Extensions", Internet-Draft draft-ietf-
morg-inthread-01, July 2010.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
4rev1", RFC 3501, March 2003.
[RFC4466] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4
ABNF", RFC 4466, April 2006.
[RFC4731] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH
Command for Controlling What Kind of Information Is
Returned", RFC 4731, November 2006.
[RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5256] Crispin, M. and K. Murchison, "Internet Message Access
Protocol - SORT and THREAD Extensions", RFC 5256, June
2008.
8.2. Informative References
[RFC5267] Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267,
July 2008.
Appendix A. Changelog
Appendix A.1. Changes in -02 since -01
o Enclose the ESEARCH response data in parentheses to conform with
RFC4466. Thanks to Alexey Melnikov for his review.
Appendix A.2. Changes in -01 since -00
o Added some clarifications that each INCTHREAD block MUST be
processed immediately because the subsequent block might refer to
its results
Author's Address
Jan Kundrat
Eledrova 558
Prague 181 00
CZ
Email: jkt@flaska.net
Kundrat Expires February 26, 2014 [Page 10]
Internet-Draft IMAP INCTHREAD Extension August 2013
Kundrat Expires February 26, 2014 [Page 11]
