Internet DRAFT - draft-nottingham-iesg-review-workload
draft-nottingham-iesg-review-workload
Network Working Group M. Nottingham
Internet-Draft 30 March 2023
Intended status: Standards Track
Expires: 1 October 2023
IESG Document Review Expectations: Impact on AD Workload
draft-nottingham-iesg-review-workload-00
Abstract
Arguably, IETF Area Directors are overloaded with document review
duties. This document surveys the relevant background, discusses the
implications, and makes a proposal for improvements.
About This Document
This note is to be removed before publishing as an RFC.
Status information for this document may be found at
https://datatracker.ietf.org/doc/draft-nottingham-iesg-review-
workload/.
information can be found at https://mnot.github.io/I-D/.
Source for this draft and an issue tracker can be found at
https://github.com/mnot/I-D/labels/ad-workload.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 1 October 2023.
Nottingham Expires 1 October 2023 [Page 1]
Internet-Draft Document Review and AD Workload March 2023
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.1. Policy . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2. Policy Discussion . . . . . . . . . . . . . . . . . . . . 6
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 7
6. Security Considerations . . . . . . . . . . . . . . . . . . . 7
7. Informative References . . . . . . . . . . . . . . . . . . . 7
Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 7
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 7
1. Introduction
The job of an IETF Area Director is notoriously difficult. Beyond
the impact on the individual, this reputation is widely thought to
reduce the pool of potential candidates for the position, resulting
in a lack of diversity, vulnerability to failure to find a candidate,
or the possibility of having to accept a poor candidate. See
Section 2.6.2 of [RFC3774] for further discussion.
One of the key responsibilities of an Area Director is reviewing each
document that comes to the IESG for publication. Although Area
Directors do much more than simply review documents, the sheer volume
of pages that the IETF publishes makes this a significant component
of the job, in terms of both their time and attention.
Section 2 surveys relevant background materials. Section 3 discusses
the requirements for document review in the IETF process, and
Section 4 makes a proposal to modify the Area Director's role in it,
in order to make it possible for more people to consider putting
their hands up to fill the position.
Nottingham Expires 1 October 2023 [Page 2]
Internet-Draft Document Review and AD Workload March 2023
2. Background
The responsibility of an Area Director to review all documents being
considered for publication originates in Section 6.1.2 of [RFC2026],
which describes the responsibilities of the IESG when reviewing a
specification for approval:
The IESG shall determine whether or not a specification submitted
to it according to section 6.1.1 satisfies the applicable criteria
for the recommended action (see sections 4.1 and 4.2), and shall
in addition determine whether or not the technical quality and
clarity of the specification is consistent with that expected for
the maturity level to which the specification is recommended.
The criteria referred to regard the maturity of the document as
indicated by its stability, its resolution of design choices, level
of community review, implementation and operational experience.
Section 5 of [RFC3710] states that:
The IESG is expected to ensure that the documents are of a
sufficient quality for release as RFCs, that they describe their
subject matter well, and that there are no outstanding engineering
issues that should be addressed before publication. The degree of
review will vary with the intended status and perceived importance
of the documents.
However, that document notes that "[i]t does not claim to represent
consensus of the IETF" but rather "was written as a 'documentation of
existing practice'".
The IESG has created internal policies to ensure that this goal of
sufficient review is achieved. The IESG Ballot Procedures
(https://www.ietf.org/standards/process/iesg-ballots/) document
describes the system used. Notably, the description of the "No
Objection" position includes this statement:
This ballot position may be interpreted as "This is outside my
area of expertise or have no cycles", in that you exercise the
ability to move a document forward on the basis of trust towards
the other ADs.
This language implies that Area Directors need not read every page of
every document; it is an "escape clause" for an overloaded AD.
Nottingham Expires 1 October 2023 [Page 3]
Internet-Draft Document Review and AD Workload March 2023
Expectations for document review are also set by the job descriptions
used by the NOMCOM. The General IESG Member Expertise
(https://datatracker.ietf.org/nomcom/2022/expertise/) desired by the
2023 NOMCOM includes:
An AD should be able to personally review every Internet-Draft
that they sponsor. For other Internet-Drafts an AD needs to be
satisfied that adequate review has taken place, though many ADs
personally review these documents as well.
However, it later sets a more onerous expectation:
Basic IESG activities can consume significant time during a
typical non-meeting week. Enough time must be allocated to [...]
read on the order of 500 pages of internet-drafts every two weeks
[...]
... which implies that they should be reading every draft.
3. Discussion
IESG document review undeniably serves an important function:
maintaining the output quality of IETF specifications, by making Area
Directors both responsible for document review and accountable to the
community (through transparency mechanisms like the Open Mic session
at plenaries, and ultimately through the risk of appeal and recall).
This accountability and quality enhances the legitimacy of the IETF's
work, and ultimately, its success.
However, the expectations for review are not clearly stated: while
there are clear affordances in the IESG-internal policies and in the
NOMCOM job description for an AD to skip a detailed review of the
document, many IETF participants and Area Directors alike seem to
believe that ADs have a fundamental responsibility to review every
document published for any concerns they may have -- to "read on the
order of 500 pages [...] every two weeks."
These conflicting and ill-stated expectations arguably have the
effect of discouraging many qualified candidates from applying for
Area Director positions.
A solution should:
* align RFC-specified requirements, IESG policy, and community
expectations
* maintain (or improve) the output quality of the IETF stream
Nottingham Expires 1 October 2023 [Page 4]
Internet-Draft Document Review and AD Workload March 2023
* reduce the required workload for Area Directors in a meaningful
way
* maintain the accountability relationships that enhance our
output's legitimacy
These requirements rule out many potential solutions. For example,
allowing Area Directors to delegate their reviews to Directorates
would be seen to harm the accountability relationship, since the
person reviewing the document would no longer be directly accountable
to the community.
On the other hand, it is clear that Area Directors are not expected
to be expert in every technology that crosses their doorstep; per the
2023 NOMCOM guidelines:
An AD need not be the ultimate expert and authority in any
technical area. The abilities to manage, to guide and judge
consensus, to know who the subject-matter experts are and to seek
their advice, and to mentor other IETF participants to take the
technical lead is at least as important as their own technical
abilities.
This implies that an Area Director might rely on others in their
review responsibilities, but cannot avoid responsibility for them.
But what _is_ that responsibility, exactly? Clearly, they are not
exercising only their own judgement; an Area Director who refused
documents based only upon their personal beliefs would effectively be
a tyrant.
This is supported by the materials. At their core, the desired
criteria listed in Sections 4.1 and 4.2 of [RFC2026] are all fairly
objective; for example:
A Proposed Standard specification is generally stable, has
resolved known design choices, is believed to be well-understood,
has received significant community review, and appears to enjoy
enough community interest to be considered valuable.
Any proposal, then, should also be focused on assuring these
properties.
This document makes one proposal below, as a starting point for
discussion; others might also meet these requirements.
Nottingham Expires 1 October 2023 [Page 5]
Internet-Draft Document Review and AD Workload March 2023
4. Proposal
To clarify the nature and role of Area Director reviews and thereby
partially address workload issues as described in [RFC3774], this
document recommends that the policy described below be adopted,
either as an IESG Statement or through IETF Consensus. Once that
takes place, supporting material (such as the NOMCOM job
descriptions) should be clarified and aligned with it.
4.1. Policy
This policy sets the expectation that Area Directors are responsible
for assuring that, from the perspective of their Area, documents
being considered for publication on the IETF stream meet the
requirements in Section 4 of [RFC2026]. This implies that an Area
Director can ballot "No Objection" for a document that they judge to
have no implications for their Area without further review.
Furthermore, they need only review those portions of other documents
that do have implications for their Area.
When reviewing a document, Area Directors may rely on expertise of
others to judge the desired properties; they need not be expert in
every technology in their Area. However, in doing so, they do not
avoid responsibility for meeting the requirements stated in
[RFC2026], and they may be held accountable if those requirements are
not met, from the perspective of their Area.
This policy does not prevent an Area Director from exceeding these
expectations. However, Area Director reviews should be based in the
requirements of [RFC2026], as elaborated upon by the DISCUSS Criteria
(https://www.ietf.org/about/groups/iesg/statements/iesg-discuss-
criteria/).
4.2. Policy Discussion
This policy is not a very large change from current practice of at
least some ADs, based upon discussions I've had. As such, its most
important function is to level-set expectations between the community
and the IESG.
Practically, this allows an AD to delegate their review (or partially
do so) to someone that they judge as appropriate, based upon their
expertise. However, they cannot avoid responsibility -- which means
that delegating to a review directorate of volunteers may be unwise.
Overall, the expectation is that ADs can (and perhaps should) rely on
other experts in their area more than they do the reviewing
themselves. They are responsible for understanding and managing any
Nottingham Expires 1 October 2023 [Page 6]
Internet-Draft Document Review and AD Workload March 2023
objections that come from the experts they rely on, and are expected
to decide the relative importance of actually requiring that any
issue raised by such an expert be addressed.
5. IANA Considerations
This document has no actions for IANA.
6. Security Considerations
Undoubtedly changing how the IESG reviews documents has potential
security implications. Caveat emptor.
7. Informative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, October 1996,
<https://www.rfc-editor.org/rfc/rfc2026>.
[RFC3710] Alvestrand, H., "An IESG charter", RFC 3710,
DOI 10.17487/RFC3710, February 2004,
<https://www.rfc-editor.org/rfc/rfc3710>.
[RFC3774] Davies, E., Ed., "IETF Problem Statement", RFC 3774,
DOI 10.17487/RFC3774, May 2004,
<https://www.rfc-editor.org/rfc/rfc3774>.
Appendix A. Acknowledgements
Thanks to Pete Resnick for his thoughts and a snippet of text.
Author's Address
Mark Nottingham
Prahran
Australia
Email: mnot@mnot.net
URI: https://www.mnot.net/
Nottingham Expires 1 October 2023 [Page 7]
