Presentation of Text Conversation in realtime and en-bloc form

This specification defines methods for presentation of a text conversation with focus on the real-time features. The aim is to give the participants in a conversation a good opportunity to perceive the real-time flow of the conversation and also provide a display of the history of the conversation that makes it easy to read. Both two-party and multi-party situations are defined.

Requirements Language

Status of this Memo

This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License.

1. Introduction
2. Terminology
3. Scope
4. Real-time text presentation methods
    4.1. Common Aspects
        4.1.1. Paragraph dividing
        4.1.2. Completion of local entry
        4.1.3. Moving between different states
        4.1.4. Erasure
        4.1.5. Presentation of detected errors
        4.1.6. Display formatting
        4.1.7. En bloc transmission
        4.1.8. Multi-party sessions
    4.2. Real-time Text Preview
        4.2.1. Entries in creation
        4.2.2. Completion of local entry
        4.2.3. Completion of preview entry
        4.2.4. Order of entries
        4.2.5. Display formatting
        4.2.6. Scrolling and buffering
    4.3. Column view
5. PSTN Aspects
    5.1. Auto-real-time for Emergency calls and Textphone calls
    5.2. Reasons to finish an entry
    5.3. Interoperability considerations with PSTN
6. Transport and presentation considerations
    6.1. Time sampling and smooth display
    6.2. RTP based transport
    6.3. MSRP and message chunk based transport
    6.4. Identification of entries
7. Presence indication
8. Alerting
9. IANA Considerations
10. Security Considerations
11. Normative References
§ Authors' Addresses

1. Introduction

This is a specification of methods for presentation of a real-time text conversation. The aim is to give the participants in a conversation a good opportunity to perceive the real-time flow of the conversation and also provide a display of the history of the conversation that makes it easy to follow the flow. One reason to specify the presentation method is to be able to give participants a synchronized view of the conversation even if they use different presentation characteristics. The methods are intended for use in a protocol environment where text can be transmitted in real-time or in fragments of messages. Both two-party situations and multi-party session presentations are specified. The specification is mainly held on the presentation level, relatively independent of the transport layer. It has though some requirements on the lower layers, as well as some characteristics of the lower layers cause slightly different user experience of the presentation.

2. Terminology

3. Scope

This specification describes methods for presenting a text conversation so that the gradual entry of text is made visible to the users. One method has text flowing in real-time in a way that is similar to common text chat, but with the possibility to follow entries while they are created in a real-time preview area. The method may be applied on any text conversation transport method that transmits text character by character, time-sampled, word by word, or any other method based on small chunks. The methods cover both two-party and multi-party situations.

4. Real-time text presentation methods

The presentation method described here are intended to give a convenient view of a text conversation between two or more participants in a session. It is intended to fulfill the requirements of ITU-T T.140 [1], and be feasible to implement in terminals with small displays. The basic concept however could be implemented in other text technologies as well and displayed in different ways. ITU-T T.140 [1], Appendix I, describes a traditional chat view and a two-column view. The display formats shall be implemented so that terminals in a session can implement different display views meeting the requirements of T.140 [1] and giving the users a synchronized view of the flow of the conversation.

_________________________________________________
|                                             |^|
|                                             | |
|                                             | |
|                                             | |
| [Anne] Hi, Anne here.                       | |
|                                             | |
| [Eve] Hi, this is Eve, calling from Paris.  | |
|       I thought you should be here.         | |
|                                             | |
| [Anne] I am coming on Thursday,             | |
|       my performance is not until Friday    | |
|       morning.                              | |
| [Anne] Can we meet on Thursday evening?     | |
|                                             | |
| [Eve] Yes, definitely. How about 7pm.       | |
|       at the entrance of the restaurant     | |
|       Le Lion Blanc?                        | |
| [Eve] we can have dinner and then take a    | |
|       walk                                  |_|
|  <Eve-typing> But I need to be back at the  |_|
|       hotel by 11 because I have t          |v|
|_____________________________________________|_|
| OK, no prob                                   |
|                                               |
|_______________________________________________|

Figure 1 (Two-party call with real-time preview.): The text is here displayed in a traditional chat view, with labelled entries from each participant ordered in a list with newest entry last. Older entries are scrolled up, out of the screen area when there is no room for them.

Real-time text arriving from other participants is displayed in 'preview areas' within the scrolling 'history' window. They are formatted to look different and are presented at the bottom of the history window until they are completed. When completed they move up in the history window and added to the history record. The text being entered by the local participant appears in a separate entry field that is preferably placed below the history field (to minimize eye movements when reading and typing).

____________________________________________________________________
|       Bob          |       Eve          |       Alice            |
|____________________|____________________|________________________|
|                    |                    |I will arrive by TGV.   |
|My flight is to Orly|                    |Convenient to the main  |
|                    |Hi all, can we plan |station.                |
|                    |for the seminar?    |                        |
|Eve, will you do    |                    |                        |
|your presentation on|                    |                        |
|Friday?             |Yes, Friday at 10.  |                        |
|Fine, wo            |                    |We need to meet befo    |
|__________________________________________________________________|

An alternative view is presented in Figure 2 (Two-party call with column view.), where text from each participant occupies one panel, and text is placed in its vertical position to show its time relation.

4.1. Common Aspects

4.1.1. Paragraph dividing

Text entries are divided into paragraphs by hard or soft return. Hard return finalizes a text entry. Once a text entry has been terminated by hard return it can no longer be erased or modified. The control sequences used for producing hard return are CRLF or paragraph separator U+2029.

When soft return is used for creating a new paragraph, previous paragraph can still be erased or modified. Soft return is produced by line separator U+2028.

4.1.2. Completion of local entry

The end-of-entry event may be triggered by a send button, a RETURN, or when another condition selectable by the local user to "post what I have so far" is met ( such as a pause in typing, a delimiting character such as a period, or a turn-taking token).

When an end-of-entry event occurs, if the entry does not end with end of paragraph as defined by the device, the sending system SHALL append one.

4.1.3. Moving between different states

Entries can be either "real-time (preview)" or "historical" and they can be either "displayed" or "hidden". When real-time text is received it SHALL BE classified as a real-time entry until an end-of-entry indicator is received. Real-time entries SHOULD be displayed in the real-time preview field. Once an end-of-entry indicator is received, the entry SHALL become historical and SHOULD be move into the history display field. Its position within the history SHALL be determined by the time that its end-of-entry indicator was received.

The local user may select to hide either the entries while they are real-time (previews) or when they are historical. Hiding entries when they are in real-time state may be done to avoid distraction for the local participant. The feature to hide the entries while in real-time state SHOULD provide some alert when an end-of-entry indicator is received as well as when real-time text stops coming in for a period of time. (The alert due to pause in incoming text is important because some real-time text users are not accustomed to sending end-of-entry indications(e.g. RETURNs) or may use a text based end-of-entry indication (such as GA).

An entry (or category of entries) can be placed in a hidden state by user command to hide it (them). (e.g. Hide all but "Mary" to make it easier to see her thread)

4.1.4. Erasure

Transmitted characters that take no position on the display (e.g. Bell or Alert in Session) SHALL not take any specific erasing action, but be regarded to be erased simultaneously with the succeeding character.

Characters that are composed by multiple keystrokes SHALL be erased by one erasing action.

New lines inserted by automatic line break and word wrap actions purely for display formatting purposes by the local system SHALL not require any specific erasing action.

New lines inserted by the user shall be erased in one erasing action even if they are represented by multiple characters.

The scope of erasure shall only be possible back to the latest new paragraph (hard return) sent from the erasing party. This may be valid for message borders in certain message based transmission systems. When using T.140, this condition should be signalled with an "unsupported request" protocol element from the entity that cannot perform the requested erasure.

4.1.5. Presentation of detected errors

If a transmission error is detected and it is likely that it has resulted in loss of text, a character SHALL be inserted in the text for display at that point. The character should be the "Replacement character", a question mark within a rhombus. For cases when this character cannot be represented on the display, the replacement character SHOULD be presented as an apostrophe (" ' ") .

The same applies for incoming text that cannot be presented on the local terminal because of limitations in available fonts.

4.1.6. Display formatting

4.1.7. En bloc transmission

It SHOULD be possible for the participants to hold their text and not have it sent to the other participants until after the end-of-message event occurs. This enables users who do not want their message to be viewed by other participants until they have verified it. This also facilitates editing since random editing can be done on the text block before it is sent. This also allows a block of text to be pasted into the text entry area and then edited before it is sent. This could be new text or a previous text entry that the user would like to resend with edits. The en bloc method SHALL NOT be the only method for sending. A 'real-time / block send' switch SHOULD be located near the local user's text entry field Real-time SHALL be the default method for sending but a user preference setting MAY change the default to en bloc.

4.1.8. Multi-party sessions

A multi-party session can be presented in a similar manner as the two-party session. The chat-view with real-time entry at the bottom of the window is one possible view.

_________________________________________________
|                                              |^|
|                                              | |
|                                              | |
|                                              | |
|[Alice] Hi, Alice here.                       | |
|                                              | |
|[Bob] Bob as well.                            | |
|                                              | |
|[Eve] Hi, this is Eve, calling from Paris.    | |
|      I thought you should be here.           | |
|                                              | |
|[Alice] I am coming on Thursday, my           | |
|      performance is not until Friday morning.| |
|                                              | |
|[Bob] And I on Wednesday evening.             | |
|                                              | |
|[Alice] Can we meet on Thursday evening?      | |
|                                              | |
|[Eve] Yes, definitely. How about 7pm.         | |
|     at the entrance of the restaurant        | |
|     Le Lion Blanc?                           | |
|[Eve] we can have dinner and then take a walk | |
|                                              | |
| <Eve-typing> But I need to be back to        | |
|    the hotel by 11 because I need            | |
|                                              |-|
| <Bob-typing> I wou                           |-|
|______________________________________________|v|
| of course, I underst                           |
|________________________________________________|

______________________________________________________________________
|       Bob          |       Eve            |       Alice            |
|____________________|______________________|________________________|
|                    |                      |I will arrive by TGV.   |
|My flight is to Orly|                      |Convenient to the main  |
|                    |Hi all, can we plan   |station.                |
|                    |for the seminar?      |                        |
|Eve, will you do    |                      |                        |
|your presentation on|                      |                        |
|Friday?             |Yes, Friday at 10.    |                        |
|Fine, wo            |                      |We need to meet befo    |
|____________________________________________________________________|

In the column view, the column showing text transmitted from the device where the presentation is made, SHOULD be placed to the right of all other columns, so that users recognize the operating environment between different devices.

In an environment with less space in the display it MAY be necessary to give up on displaying the relative time order in the column view in order to display more of the conversation contents in available space.

Yet other situations may call for display in separate windows for example underneath video images from each participant.

_______________________   _____________________   ___________________
|          Bob        |   |         Eve       |   |     Alice        |
|_____________________|   |___________________|   |__________________|
|         ooooo       |   |        888        |   |        ...       |
|        / o  o\      |   |      8/- -\8      |   |       |||||      |
|       (   _   |     |   |     0|  |  |0     |   |     || o o ||    |
|        \_____/      |   |      |  _  |      |   |     ||  v  ||    |
|                     |   |       \___/       |   |     ||\___/||    |
|_____________________|   |___________________|   |__________________|
|Help me to spell     |   |necessary          |   |ne.... OK you take|
|nessessarry,I always |   |                   |   |it                |
|get it wrong         |   |                   |   |                  |
|_____________________|   |___________________|   |__________________|

When implemented in an environment that supports multi-party calls, it may be felt less important to maintain a real-time preview view of text from all participants. It may be very important for some participants to have rapid real-time preview presentation of selected participants, e.g. for live captioning of the call by a third party.

Thus it may be desirable to be able to turn on or off the preview presentation per user. When turning off real-time preview from one participant, its presentation SHALL disappear from the preview window, and text SHALL be entered en-bloc to the history display as they are finished for that participant.

4.2. Real-time Text Preview

4.2.1. Entries in creation

Entries in creation SHALL be displayed in a real-time preview area, one for each participant who has entries in creation. The real-time preview areas MAY be placed under the list of completed entries as shown in Figure 1 (Two-party call with real-time preview.) and Figure 3 (Three-party call with real-time preview.), or at any other suitable place in the user interface. If video from the participants is also displayed, then it MAY be suitable to display the real-time preview areas under the video image of the participant. The real-time text MAY also be displayed in a manner more closely associated with earlier exchanged text entries by the same participant (e.g. text from each participant goes in its own column).

If real-time previews from other participants are placed under the list of completed entries as in Figure 1 (Two-party call with real-time preview.) and Figure 3 (Three-party call with real-time preview.), the text being entered by the local participant SHOULD be placed at the bottom in its own text entry field. This is recommended for a number of reasons. First, this is the only "editable" text on the screen. It also facilitates an optional input behavior where the local user may sometimes be holding their text back until it is completed while normally transmission occurs in real-time. Having the user input area be in a separate field MAY also be useful when scrolling the output field so that the input field always stays in view even as the history and text previews are scrolled out of view to read older text.

For ease of reading different entries, it is RECOMMENDED that all entries be placed close together in the display area.

For text input technologies requiring a number of keystrokes before the character or characters are finally decided, no characters shall be submitted to communication until they are decided from this input preparation process. This is for example valid for input of some Asian languages, and for some text entry methods from number keypads, and word prediction systems.

4.2.2. Completion of local entry

Text from the local participant SHALL be entered in the local user input field, until an end-of-entry event occurs. The completed entry SHALL be moved to the history display area. If the protocol used defines an end-of-message indicator, it SHALL also be issued.

4.2.3. Completion of preview entry

Text from the remote participants SHALL be entered in the preview area until an end-of entry event occurs. The end-of-entry is identified by any variant of a NEW LINE coded in the character set used, or an end of message indicator if there is a specific coding of that event. When an end-of-entry event occurs, the completed entry SHALL be moved to the history display area.

4.2.4. Order of entries

The order of displayed entries in the display area SHALL be the timing order when the entries were "posted" to the display from the preview area. That is, when the new line or end-of-message indicator is received.

4.2.5. Display formatting

The labels on the entries SHOULD display the user name of the participant. If this information is not available, labels indicating "Received" and "Transmitted" or other suitable names for the participants SHOULD be used.

The real-time preview display area MAY follow the same display formatting regarding font size, colours etc as the display area or MAY be different.

Each real-time preview area MAY have a fixed or adjustable size. It MAY also have no specific scrolling features or its own scrolling mechanism.

4.2.6. Scrolling and buffering

When there are entries in the history area that have been pushed (scrolled) out of the view by new text coming in, it SHOULD be possible to scroll back to them within a practical limit. When the display area is scrolled, it SHALL stay in that scrolled position until scrolling is changed again or (at the user's option) when a new entry is received. When scrolled to the bottom, the display area SHALL auto-scroll as needed to show new entries. When the display arrangement is made with the preview field placed just under the history field as in Figure 1 (Two-party call with real-time preview.) and Figure 3 (Three-party call with real-time preview.), the preview field and the history field SHOULD scroll together as one display area.

The input field of the local participant SHOULD always be visible regardless of the scroll position of the history field.

4.3. Column view

5. PSTN Aspects

5.1. Auto-real-time for Emergency calls and Textphone calls

When it is detected that the session is used for emergency purposes, the text transmission SHALL be switched to real-time regardless of its previous setting.

The user SHALL still be provided with a possibility to switch to en bloc sending after the session is established.

5.2. Reasons to finish an entry

Users with dominating experience from real-time text communication in PSTN may have a habit of not ending entries with a new line. There will be a risk that entries are left in real-time mode unintentionally not displayed and not read if the receiving end has hidden real-time display. Some actions are needed in order to avoid this risk.

If an entry is left in real-time mode without any input activity for a long period (e.g. 10 seconds), the local participant should be given an indication that there is an unfinished entry in the input field, and given a hint to complete it. Optionally, e.g. when using a voice-to-text application for generating text, the application SHOULD create the end-of-entry.

A period (".") followed by a short inactivity MAY also be configured to be used as an end-of-entry indication.

5.3. Interoperability considerations with PSTN

For PSTN text gateways having user input from PSTN text telephones, the following sequences SHOULD be included among those causing an entry to be finished. These terminations would usually be done by the PSTN gateway in its transmission towards the IP side:

White spaces (space bar, New line, CR, and LF) after those characters SHALL be accepted and included in the finished entry. (Some users do type a space character after the turn-taking indicator and some textphones will send return after the turn-taking symbol).

6. Transport and presentation considerations

6.1. Time sampling and smooth display

It is RECOMMENDED that characters be buffered and transmitted in 300 ms intervals on the transport level. It is permissible to buffer characters for transmission in up to 500 ms intervals. Display of received chunks of text SHOULD be done one character at a time spread over the transmission interval so that adding a chunk of text to the real-time preview window approximately covers the transmission interval to give a smoother flow.

The presentation method MAY be used with transport methods for real-time text and for all text message methods where it is possible to use timer based transmission to transmit fragments of message entries.

6.2. RTP based transport

6.3. MSRP and message chunk based transport

6.4. Identification of entries

The transport method SHALL allow identification of the source of text, so that text from different sources can be arranged for convenient and readable presentation at the receiving end. [e.g. to attach labels to the incoming text).

7. Presence indication

Appropriate SIP based presence features SHOULD be used to indicate status in the user interface, e.g. that the user is typing when in ‘en bloc’ mode.

8. Alerting

In order to be useful for hearing impaired, deaf and deaf-blind users as well as all situations with all users, it is important to provide audible, visual and, where possible, tactile alerting from events in the text conversation application.
It should be possible for a user to get external alerting signals with a method preferred by the user. It may for example be vibration, light flashes or sound as selected by the user. It should also be possible to get alerting on the screen at certain events.
The signals to external alerting systems should be issued when an incoming request for session initiation is received. When the method is used in connection with T.140 (ITU-T, “Recommendation T.140, Protocol for Multimedia Application Text Conversation (including Addendum),” February 2000.) [T140.refs] presentation, it should also be issued when the alert-in-session T.140 control event is received.
For minor events, for example when an entry from a user is completed and displayed in the conversation display area, an indication MAY be given e.g. by an on-screen flashing or any other suitable alerting signal.
It may be useful to provide external alerting also for these minor events in specific situations. If the user has not touched the application for a number of minutes when the minor event occurs it may be of interest to get an external alert. Details of such arrangements are outside the scope of this document.

9. IANA Considerations

10. Security Considerations

This specification does not introduce any procedures that change security issues from what is already specified for the session and transport environment where the presentation method is applied.

[I-D.hellstrom-text-conference]	Hellstrom, G. and A. Wijk, “Text media handling in RTP based real-time conferences,” draft-hellstrom-text-conference-02 (work in progress), March 2010 (TXT).
[RFC2119]	Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” BCP 14, RFC 2119, March 1997 (TXT, HTML, XML).
[RFC3550]	Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, “RTP: A Transport Protocol for Real-Time Applications,” STD 64, RFC 3550, July 2003 (TXT, PS, PDF).
[RFC4103]	Hellstrom, G. and P. Jones, “RTP Payload for Text Conversation,” RFC 4103, June 2005 (TXT).
[RFC4975]	Campbell, B., Mahy, R., and C. Jennings, “The Message Session Relay Protocol (MSRP),” RFC 4975, September 2007 (TXT).
[T140.refs]	ITU-T, “Recommendation T.140, Protocol for Multimedia Application Text Conversation (including Addendum),” February 2000.

	Gunnar Hellstrom
	Omnitor
	PO Box 92054
	12006 Stockholm
	Sweden
Phone:	+46-8-58900050
Email:	gunnar.hellstrom@omnitor.se

	Norman Williams
	Gallaudet University
	800 Florida Ave
	SLCC-1120
	Washington, DC 20002
	USA
Email:	norman.williams@gallaudet.edu

	Arnoud A. T. van Wijk
	Real-Time Text Taskforce (R3TF)
	NL
Email:	arnoud@realtimetext.org
URI:	http://www.realtimetext.org

	Gregg C. Vanderheiden
	Trace R&D Center, University of Wisconsin-Madison
	1550 Engineering Drive
	Madison, WI 53706
	USA
Email:	gv@trace.wisc.edu
URI:	http://www.engr.wisc.edu/ie/faculty/vanderheiden_gregg.html

Presentation of Text Conversation in realtime and en-bloc formdraft-hellstrom-textpreview-07

Abstract