RFC Annotations Tool

The RFC annotations tool allows a user to view a set of RFCs with relevant annotations. Some annotations are automatically generated (such as which RFCs update the one being viewed), but it is also easy to create your own annotations and share them with others, and to read RFCs annotated by many other people. The annotated RFCs are static HTML with active links between the documents and to other documents outside those generated by the tool.

IMPORTANT NOTE: This project is pre-release. We are encouraging new issues and pull requests during August 2022, and will incorporate changes as appropriate. Around October 2022, the project will have its first official release (and will still be open to issues and pull requests, of course). For now, consider this an alpha release where significant features and interfaces may change before the final release.

This release of the RFC annotations tool is aimed at the RFCs that relate to the DNS. All errata are shown in-line, and preliminary custom annotations include descriptions of how each RFC is updated by others. The primary intended audiences of this tool and the associated annotations are security researchers, DNS software developers, and DNS protocol developers.

You can see the output of the tool for the DNS RFCs [here] (https://rfc-annotations.research.icann.org/). You can also clone this repo and build the tool on a local system.

Although the automatically generated annotations will be useful to the intended audiences, the expected value of the tool will be incorporating annotations from individuals. Some examples of valuable annotations include:

Errata placed in-line, possibly with commentary on the errata
Notes of how an RFC is updated by other RFCs
Instructions on how to configure open-source software to turn on protocol features
Pointers to articles about security and/or operational issues with protocol features
Links to mailing list discussions of interpretation of parts of RFCs
Commentary from developers about issues they had with particular text
Historical notes about how some features appeared in the RFCs

The generated RFCs are in HTML format with the annotations shown on the right side of the screen. The files use standard CSS which can be modified locally to highlight particular types of annotations or simply to suit the individual viewer's preferences.

(Special thanks go to the authors of rfc2html which is the basis of the HTMLization of RFCs in this tool.)

Prerequisites

Running the tool to generate the annotated RFCs requires either Docker or the combination of python3 (version 3.7 or greater) and rsync. make is mandatory if you want to use the targets defined in the Makefile.

Process Overview

The tool takes a list of target RFCs, reads the text version of the RFCs, and converts those RFCs to HTML. The status of RFCs is fetched and turned into annotations that appear at the top of the generated RFCs. The entire corpus of current Internet Drafts is also fetched or updated, and scanned for indications that particular drafts intend to update or obsolete RFCs when they are eventually published as RFCs. The complete list of errata is fetched or updated; this is used for manual creation of annotations of the relevant errata.

The tool goes through all the annotations and incorporates them in the RFCs. The last step in running the tool is to create the index.html file that has links to all the generated RFCs.

Viewing the Annotated RFCs

The generated HTML output, including the index file, are found in the generated-html/ directory. The RFCs show annotations near the top that list the RFCs by which they are updated or obsoleted, as well as an annotation with links to all the relevant errata.

Within annotations, links to RFCs that are in the set of generated RFCs lead to the generated RFCs; links to other RFCs lead to the RFC Editor's site.

To view just the annotations in an RFC, click the "Hide RFC" button at the top right of the window. When the RFC is hidden, to show the RFC again and jump to the location of the annotation, click the location link at the left of the annotations's caption.

Annotations can be collapsed and expanded by clicking in the header line of the annotation.

Annotations are displayed with standard CSS. You can customize the CSS to change the way that annotations are displayed as described below.

See the annotations/README.md file for instructions on how to create annotations.

Running the Tool

The tool can be configured in many ways:

List of RFCs that are annotated
Javascript used to sort columns in the list of RFCs
Sources for annotations
CSS used to display the RFCs and annotations
Javascript used to show and hide displayed content

All configuration is stored in the default-config/ directory. If you want to make local changes to the default configuration, copy the file from the default-config/ directory to the local-config/ directory and edit it there. This prevents your changes from being lost if you later do a git pull. When you edit files in local-config/, you need to re-run the tool to download and generate the RFCs again.

List of RFCs

The list of RFCs is stored in a file whose name follows the pattern*-rfcs.txt. The tool comes with a dns-rfcs.txt file pre-populated with over 100 RFCs related to the DNS. The index file that is generated has the same prefix as the input name, such as dns-index.html.

ICANN supports the list of DNS-related RFCs in this repository. A similar repository for HTTP-related RFCs can be found here.

Annotations

The annotations are stored in the annotations/ directory. Annotations that are automatically generated by the tool are put in annotations/_generated; it is unlikely that you want to touch that folder. You can add local annotations by adding subdirectories to the annotations/ directory.

The tool will collect annotations from the sources listed in the annotation-sources.txt configuration file. The default file comes with sources of DNS-related annotations created by ICANN (such as errata and updates discussion) as well as outside sources that have told ICANN they want to be included.

See the README in the annotations/ folder for a description of the format of the annotation files.

The tool goes through the annotations/ directory and all subdirectories looking for files whose name is "rfcnnnn.*" where "nnnn" matches one of the listed RFCs. The file extension can be anything. Any subdirectory of the annotations/ directory that has a file called .ignore will be skipped by the tool when adding annotations to the RFCs, as will subdirectories named .git.

Creating the HTML Locally

The easiest way to run the tool is with make.

make clean will remove all collected files and generated annotations (but not your own annotations).
make and make all collect the text RFCs, Internet Drafts, status information, and errata, then generate the HTMLized RFCs.
make annotations only refreshes the generated HTML by scanning the annotations directories (make annotations does not create the index.html file).

The programs called by make (which are in program/) allow user configuration through environment variables.

RFC_TXT_DIR, RFC_OUTPUT and RFC_ANNOTATIONS change the defaults of raw-originals/, generated-html/, and annotations/, respectively,
RFC_LIST can be used to act on a subset of RFCs. The value is a space-separated list of numbers, such as 1035 2181. This can be used to speed processing, but processing all RFCs on a typical laptop is likely fast enough.
RFC_INDEX tells the tool whether to create the index file. Generating this file can sometimes be slow, so setting RFC_INDEX=NO can speed up the overall building process.
RFC_HTML_WARNINGS set to YES tells the annotation collector to display warnings about HTML fix-ups that it is automatically applying.
RFC_VERBOSE set to YES produces more output to the console which may be helpful if issues occur.

Personal CSS and Javascript

The css.html configuration file controls the display of the RFCs and annotations. Its contents are added to each generated RFC. You can copy this to the local-config/ directory and change the values, or add your own.

For example:

.area .annotation .reported {display: block; background-color: #F88;}
.area .annotation .rejected {display: block; background-color: #F88;}
.area .annotation .updated {background-color: #88F;}

This would display rejected and reported errata on red background (whereas they are normally not shown), and the status of "updated" show the annotations that have on a blue background.

You can also create CSS for individual document authors, such as:

.area .annotation .chris_smith {font-size: 125%; background-color: #000;}

Similarly, the scripts.html configuration file contains Javascript that is added to each generated RFC.

Using Docker

If Docker is installed locally, you can run the tool without any further prerequisites. make encapsulates the commands for the build process for Docker using targets clean, docker-remove, docker-annotations and docker.

reschke / rfc-annotations