| Network Working Group | P. Hoffman |
| Internet-Draft | ICANN |
| Intended status: Informational | J. Hildebrand |
| Expires: January 1, 2017 | Cisco |
| June 30, 2016 |
RFC v3 Prep Tool Description
draft-iab-rfcv3-preptool-02
This document describes some aspects of the “prep tool” that is expected to be created when the new RFC v3 specification is deployed.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 1, 2017.
Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
For the future of the RFC format, the RFC Editor has decided that XML (using the XML2RFCv3 vocabulary [I-D.iab-xml2rfc]) is the canonical format, in the sense that it is the data that is the authorized, recognized, accepted, and archived version of the document. See [RFC6949] for more detail on this.
Most people will read other formats, such as HTML, PDF, ASCII text, or other formats of the future, however. In order to ensure each of these formats is as similar as possible to one another as well as the canonical XML, there is a desire that the translation from XML into the other formats will be straightforward syntactic translation. To make that happen, a good amount of data will need to be in the XML format that is not there today. That data will be added by a program called the “prep tool”, which will often run as a part of the xml2rfc process.
This draft specifies the steps that the prep tool will have to take. As changes to [I-D.iab-xml2rfc] are made, this document will be updated.
The details (particularly any vocabularies) described in this document are expected to change based on experience gained in implementing the RFC Production Center’s (RPC) toolset. Revised documents will be published capturing those changes as the toolset is completed. Other implementers must not expect those changes to remain backwards-compatible with the details described in this document.
The prep tool will have several settings:
There are only a few differences between the two settings. For example, the boilerplate output will be different, as will the date output on the front page.
Note that this only describes what the IETF-sponsored prep tool does. Others might create their own work-alike prep tools for their own formatting needs. However, an output format developer does not need to change the prep tool in order to create their own formatter: they only need to be able to consume prepared text. The IETF-sponsored prep tool runs in two different modes: “I-D” mode when the tool is run during Internet-Draft submission and processing, and “RFC production mode” when the tool is run by the RFC Production Center while producing an RFC.
This tool is described as if it is a separate tool so that we can reason about its architectural properties. In actual implementation, it might be a part of a larger suite of functionality.
When the IETF draft submission tool accepts v3 XML as an input format, the submission tool runs the submitted file through the prep tool. This is called “I-D mode” in this document. If the tool finds no errors, it keeps two XML files: the submitted file and the prepped file.
The prepped file provides a record of what a submitter was attesting to at the time of submission. It represents a self-contained record of what any external references resolved to at the time of submission.
The prepped file is used by the IETF formatters to create outputs such as HTML, PDF, and text (or the tools act in a way indistinguishable from this). The message sent out by the draft submission tool includes a link to the original XML as well as the other outputs, including the prepped XML.
The prepped XML can be used by tools not yet developed to output new formats that have as similar output as possible to the current IETF formatters. For example, if the IETF creates a .mobi output renderer later, it can run that renderer on all of the prepped XML that has been saved, ensuring that the content of included external references and all of the part numbers and boilerplate will be the same as what was produced by the previous IETF formatters at the time the document was first uploaded.
During AUTH48, the RPC will run the prep tool in canonical RFC production mode and make the results available to the authors so they can see what the final output might look like. When the document has passed AUTH48 review, the RPC runs the prep tool in canonical RFC production mode one last time, locks down the canonicalized XML, runs the formatters for the publication formats, and publishes all of those.
This document assumes that the prep tool will be used in the following manner by the RPC; they may use something different, or with different configuration.
Similarly to I-D’s, the prepped XML can be used later to re-render the output formats, or to generate new formats.
The steps listed here are in order of processing. In all cases where the prep tool would “add” an attribute or element, if that attribute or element already exists, the prep tool will check that the attribute or element has valid values. If the value is incorrect, the prep tool will warn with the old and new values, then replace the incorrect value with the new value.
Currently, the IETF uses a tool called “idnits” to check text input to the Internet-Drafts publishing process. idnits indicates if it encountered any errors, and also provide text with all of the warnings and errors in a human-readable form. The prep tool should probably check for as many of these errors and warnings as possible when it is processing the XML input. For the moment, tooling might run idnts on the text output from the prepared XML. The list below contains some of those errors and warnings, but the deployed version of the prep tool may contain additional steps to include more or the checks from idnits.
These steps will ensure that the input document is properly formatted, and that all XML processing has been performed.
Process all <x:include> elements. Note: <x:include>d XML may include more <x:include>s (with relative references resolved against the base URI potentially modified by a previously inserted xml:base attribute). The tool may be configurable with a limit on the depth of recursion.
Fully process any Document Type Definitions (DTDs) in the input document, then remove the DTD. At a minimum, this entails processing the entity references and includes for external files.
Remove processing instructions.
Check the input against the RNG in [I-D.iab-xml2rfc]. If the input is not valid, give an error.
Check all elements for “anchor” attributes. If any “anchor” attribute begins with “s-“, “f-“, “t-“, or “i-“, give an error.
These steps will ensure that all default values have been filled in to the XML, in case the defaults change at a later date. Steps in this section will not overwrite existing values in the input file.
If the <rfc> element has a “version” attribute with a value other than “3”, give an error. If the <rfc> element has no “version” attribute, add one with the value “3”.
If the <front> element of the <rfc> element does not already have a <seriesInfo> element, add a <seriesInfo> element with the name attribute based on the mode in which the preptool is running (“Internet-Draft” for Draft mode and “RFC” for RFC production mode) and a value that is the input filename minus any extension for Internet-Drafts, and is a number specified by the RFC Editor for RFCs.
If the <front> element in the <rfc> element does not contain a <date> element, add it and fill in the “day”, “month”, and “year” attibutes from the current date. If the <front> element in the <rfc> element has a <date> element with “day”, “month”, and “year” attibutes, but the date indicated is more than three days in the past or is in the future, give a warning. If the <front> element in the <rfc> element has a <date> element with some but not all of the “day”, “month”, and “year” attibutes, give an error.
If the input document includes a “prepTime” attribute of <rfc>, exit with an error.
Fill in the “prepTime” attribute of <rfc> with the current datetime.
Add a “start” attribute to every <ol> element containing a group that does not already have a start.
Fill in any default values for attributes on elements, except “keepWithNext” and “keepWithPrevious” of <t>, and “toc” of <section>. Some default values can be found in the Relax NG schema, while others can be found in the prose describing the elements in [I-D.iab-xml2rfc]).
For each <section>, modify the “toc” attribute to be either “include” or “exclude”:
If in I-D mode, if there is a <note> or <section> element with a “removeInRFC” attribute that has the value “true”, add a paragraph to the top of the element with the text “This note is to be removed before publishing as an RFC.” or “This section…”, unless a paragraph consisting of that exact text already exists.
These steps will ensure that ideas that can be expressed in multiple different ways in the input document are only found in one way in the prepared document.
Normalize the values of “month” attributes in all <date> elements in <front> elements in <rfc> elements to numeric values.
In every <email>, <organization>, <street>, <city>, <region>, <country>, and <code> element, if there is an “ascii” attribute and the value of that attribute is the same as the content of the element, remove the “ascii” element and issue a warning about the removal.
In every <author> element, if there is an “asciiFullname”, “asciiInitials”, or “asciiSurname” attribute, check the content of that element against its matching “fullname”, “initials”, or “surname” element (respectively). If the two are the same, remove the “ascii*” elelement and issue a warning about the removal.
For every <section>, <note>, <figure>, <references>, and <texttable> element that has a (deprecated) “title” attribute, remove the “title” attribute and insert a <name> element with the title from the attribute.
These steps will generate new content, overriding existing similar content in the input document. Some of these steps are important enough that they specify a warning to be generated when the content being overwritten does not match the new content.
If in I-D mode, fill in “expiresDate” attribute of <rfc> based on the <date> element of the document’s <front> element.
Create a <boilerplate> element if it does not exist. If there are any children of the <boilerplate> element, produce a warning that says “Existing boilerplate being removed. Other tools, specifically the draft submission tool, will treat this condition as an error” and remove the existing children.
Verify that <rfc> “submissionType” and <seriesInfo> “stream” are the same if they are both present. If either is missing, add it. Note that both have a default value of “IETF”.
Add the “Status of this Memo” section to the <boilerplate> element with current values. The application will use the “submissionType”, and “consensus” attributes of the <rfc> element, the <workgroup> element, and the “status” and “stream” attributes of the <seriesInfo> element, to determine which [I-D.iab-rfc5741bis] boilerplate to include, as described in Appendix A of [I-D.iab-xml2rfc].
Add the “Copyright Notice” section to the <boilerplate> element. The application will use the “ipr” and “submissionType” attributes of the <rfc> element and the <date> element to determine which portions and which version of the TLP to use, as described in A.1 of [I-D.iab-xml2rfc].
For any <reference> element that does not already have a “target” attribute, fill the target attribute in if the element has one or more <seriesinfo> child element(s) and the “name” attribute of the <seriesinfo> element is “RFC”, “Internet-Draft”, or “DOI” or other value for which it is clear what the “target” should be. The particular URLs for RFCs, Internet-Drafts, and DOIs for this step will be specified later by the RFC Editor and the IESG. These URLs might also be different before and after the v3 format is adopted.
Add a “slugifiedName” attribute to each <name> element that does not contain one; replace the attribute if it contains a value that begins with “n-“.
If the “sortRefs” attribute of the <rfc> element is true, sort the <reference>s and <referencegroup>s lexically by the value of the “anchor” attribute, as modified by the “to” attribute of any <displayreference> element. The RFC Editor needs to determine what the rules for lexical sorting are. The authors of this document acknowledge that getting consensus on this will be a difficult task.
Add “pn” attributes for all parts. Parts are:
In every <iref> element, create a document-unique “pn” attribute. The value of the “pn” attribute will start with ‘i-‘, and use the item attribute, the subitem attribute (if it exists), and a counter to ensure uniqueness. For example, the first instance of “<iref item=’foo’ subitem=’bar’>” will get the irefid ‘i-foo-bar-1’.
For each <xref> element that has content, fill the “derivedContent” with the element content, having first trimmed the whitespace from ends of content text. Issue a warning if the “derivedContent” attribute already exists and has a different value from what was being filled in.
For each <xref> element that does not have content, fill the “derivedContent” based on the “format” attribute. * For format=’counter’, the “derivedContent” is the section, figure, table, or ordered list number of the element with anchor equal to the xref target. * For format=’default’ and the “target” attribute points to a <reference> or <referencegroup> element, the “derivedContent” is the value of the “target” attribute (or the “to” attribute of a <displayreference> element for the targeted <reference>). * For format=’default’ and the “target” attribute points to a <section>, <figure>, or <table>, the “derivedContent” is the name of the thing pointed to, such as “Section 2.3”, “Figure 12”, or “Table 4”. * For format=’title’, if the target is a <reference> element, the “derivedContent” attribute is the name of the reference, extracted from the <title> child of the <front> child of the reference. * For format=’title’, if the target element has a <name> child element, the “derivedContent” attribute is the text content of that <name> element concatenated with the text content of each descendant node of <name> (that is, stripping out all of the XML markup, leaving only the text). * For format=’title’, if the target element does not contain a <name> child element, the “derivedContent” attribute is the value of the “target” attribute with no other adornment. Issue a warning if the “derivedContent” attribute already exists and has a different value from what was being filled in.
If any <relref> element’s “target” attribute refers to anything but a <reference> element, give an error.
For each <relref> element, fill in the “derivedLink” attribute.
These steps will include external files into the output document.
These steps provide extra cleanup of the output document in RFC production mode.
If in RFC production mode, if there is a <note> or <section> element with a “removeInRFC” attribute that has the value “true”, remove the element.
If in RFC production mode, remove all <cref> elements.
If in RFC production mode, remove XML comments.
If in RFC production mode, remove all “xml:base” or “originalSrc” attributes from all elements.
If in RFC production mode, ensure that the result is in full compliance to v3 schema, without any deprecated elements or attributes, and give an error if any issues are found.
These steps provide the finishing touches on the output document.
Determine all the characters used in the document, and fill in the “scripts” attribute for <rfc>.
Pretty-format the XML output. (Note: there are many tools that do an adequate job.)
There will be a need for Internet-Draft authors who include files from their local disk (such as for <artwork src=”mydrawing.svg”/>) to have the contents of those files inlined to their drafts before submitting them to the Internet-Draft processor. (There is a possibility that the Internet-Draft processor will allow XML files and accompanying files to be submitted at the same time, but this seems troublesome from a security, portability, and complexity standpoint.) For these users, having a local copy of the prep tool that has an option to just inline all local files would be terribly useful. That option would be a proper subset of the steps given in Section 5.
A feature that might be useful in a local prep tool would be the inverse of the “just inline” option would be “extract all”. This would allow a user who has a v3 RFC or Internet-Draft to dump all of the <artwork> and <sourcecode> elements into local files instead of having to find each one in the XML. This option might even do as much validation as possible on the extracted <sourcecode> elements. This feature might also remove some of the features added by the prep tool (such as part numbers and slugifiedName’s starting with “n-“) in order to make the resulting file easier to edit.
None.
Steps in this document attempt to prevent the <artwork> and <sourcecode> entities from exposing the contents of files outside the directory in which the document being processed resides. For example, values starting with “/”, “./”, or “../” should generate errors.
The security considerations in [RFC3470] apply here. In specific, processing XML external references can expose a prep tool implementation to various threats by causing the implementation to access external resources automatically. It is important to disallow arbitrary access to such external references within XML data from untrusted sources.
Many people contributed valuable ideas to this document. Special thanks go to Robert Sparks for his in-depth review and contributions early in the development of this document, and to Julian Reschke for his help getting the document structured more clearly.
| [I-D.iab-rfc5741bis] | Halpern, J., Daigle, L. and O. Kolkman, "On RFC Streams, Headers, and Boilerplates", Internet-Draft draft-iab-rfc5741bis-02, February 2016. |
| [I-D.iab-xml2rfc] | Hoffman, P., "The "xml2rfc" version 3 Vocabulary", Internet-Draft draft-iab-xml2rfc-04, June 2016. |
| [RFC3470] | Hollenbeck, S., Rose, M. and L. Masinter, "Guidelines for the Use of Extensible Markup Language (XML) within IETF Protocols", BCP 70, RFC 3470, DOI 10.17487/RFC3470, January 2003. |
| [RFC6949] | Flanagan, H. and N. Brownlee, "RFC Series Format Requirements and Future Development", RFC 6949, DOI 10.17487/RFC6949, May 2013. |
| [RFC7669] | Levine, J., "Assigning Digital Object Identifiers to RFCs", RFC 7669, DOI 10.17487/RFC7669, October 2015. |