BSD Software Development

Amsterdam NL job@bsd.nl https://www.bsd.nl/

Deutsche Telekom

Münster DE Theodor-Fedor.Vompe@telekom.de

This document describes a format and data storage approach for materialization of RPKI data in order to support a range of use cases such as auditing Certification Authorities and analytical research. The rpkispool format can be used for high-latency replication of raw RPKI data and associated validation outcomes as efficiently compressed durable objects. The method uses widely available standardized tooling and is designed to support long-term preservation of RPKI data in a cost-effective way.

Introduction The ability to economically archive multiple years worth of RPKI data produced by Certification Authorities (CAs) worldwide is essential for ongoing protocol maintenance, development of best practises, and incident research. This document describes a format and data storage approach for efficiently materializing RPKI data for long-term preservation in compact archives using standardized tooling. The project adheres to the Sushi Principle (): "raw" data is better than "cooked" data. The guiding principle is that for maximum flexibility, to allow for future and unforeseen use cases, the data should be accessible in its original form, rather than some aggregated or processed form. In order to collect RPKI material the RPKIViews project employs multiple topologically and geographically diverse vantages points and synchronizes using both Rsync and RRDP. In February 2026, using the method described in this document, the project discovered and stored 4,961,325 RPKI objects (, ) and produced 53,826 CCRs (). Together this data would consume 1.2 TB in uncompressed form, however after compression only 16.3 GB remained, a 98.6% reduction. The daily checkpoints together consumed 307 GB in uncompressed form and 14 GB in compressed form, a 95.4% reduction in size. In other words, a full month's worth of RPKI data only manducated 30 GB of disk space. Storing all the world's RPKI data at a rate of roughly 1 GB per day makes research fairly accessible and affordable.

Storage Concept and File Formats To capture the global RPKI's endless stream of data, batch processors divide the data stream into chunks of fixed duration, processing a day's worth of data at the end of every day. Each day starts with a set of initial and internally consistent snapshots, which together form the initstate, and throughout the day all change data, i.e., all newly discovered RPKI objects and associated validation outcome states (CCRs) are appended to a log: the rpkispool. Bundling the RPKI objects together with CCRs and sorting the archive members in a specific way optimally allows compression algorithms to find redundant data and significantly improves compression ratios. Archive files are formatted following the specification and compressed in Zstandard form with windowLog 27 (--long) at compression level 19.

Filename Scheme This section describes the filenames used for the archive members. The filenaming scheme was designed to allow researchers to extract multiple rpkispool archives in a single directory without naming conflicts.

Initstate filenaming scheme The filenames of the members of an initstate archive are constructed as follows: ${RPKIVIEWS_NODE_ID}/${PUBLICATION_POINT_FQDN}/path/to/object.${EXTENSION}. An example is as follows: $ zstdcat 20260301-initstate.tar.zst | tar -tf- | head -n 20 ams1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa blr1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa blr2/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa dus1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa miso/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa nyc1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa sng1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa syd1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa syd2/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa yyz1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa zur1/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa zur2/akane.maru.co.jp/repo/1073c6/1/3134302e3233352e3139392e302f32342d3234203d3e20323134363735.roa ams1/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl blr1/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl blr2/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl dus1/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl miso/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl nyc1/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl sng1/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl syd1/akane.maru.co.jp/repo/1073c6/1/F03205B3993400CC3FC657EFE68E0696DA332458.crl

Rpkispool filenaming scheme The filenames of the members of an rpkispool archive are constructed in one of two ways. Members whose filename starts with static/ are DER-encoded RPKI objects where the filename is the SHA-256 () of the object encoded using Base64 with the filename safe alphabet (). For performance reasons, the directory hierarchy in static/is constructed using the last few bytes of the SHA-256. Any other members are auxiliary data and grouped according to production date, using the form ${YEAR}/${MONTH}/${DAY}/${ISO8601}-${NODEID}.${EXTENSION}. The filename extension signifies the type of file. An example is as follows: $ zstdcat 20260301-rpkispool.tar.zst | tar -tf- | grep -m 1 -B 10 -A 10 2026/ static/xs/Ec/-fNEEKsF1NLwAtLAxQtqolj7UXWj9I9nJjBWGwlxsEc static/yI/iA/Ix5u73xRGkXAwjg2-93ULtv-yHXqV9_ucDIXZ-5yIiA static/ya/Z8/vfnWEX976G9V1uRL_-i6G-G4DICffwpq7drYOnJyaZ8 static/zM/Q4/rleR5X9N8s1vj_zuwV7JyleKmfcAfglrd1CCuHEzMQ4 static/-j/lc/lJ7XMmaXnUt0OstDsW4rKBVi8XKE51r6iC4sxq8-jlc static/0K/LU/HvnvCbGHIE42NpG6Fpx8NK_94LLNHHbZsuh1Q7l0KLU static/0s/8Y/eQmMs0N8T2FObu-k7HorubQqUrVQd3lkM7Mm_kZ0s8Y static/1A/HE/QWFu2t5XTsuMGhaVowKVMrKyRmLlHJlmqL7uf0M1AHE static/1D/XQ/sKeIyf9Fj71hW0R0SPcFViBchkZvcNhi6VL45V-1DXQ static/1Z/jU/2SMhqOUgK5NYtq3L06ZqFHvZdKbhDi9HYAbCPlc1ZjU 2026/03/01/20260301T000035Z-miso.ccr 2026/03/01/20260301T000035Z-miso.log 2026/03/01/20260301T000035Z-miso.metrics static/2C/M8/gRAUD18G1a4Sr5A0jrH7kTpX4Yj1Zfrywjt5yeD2CM8 static/2N/j4/TVkptGmN3prJANdeWxQVS1Bt-UpnjtDX6RzIG-B2Nj4 static/2k/7U/vKNQKfG--QClvWrie6lq_LZSPiXhkcxHK_Thfkf2k7U static/2p/YM/jRRbQXFhjlhk-Xd6jAgShu3v1eyFxfIekY_BUSc2pYM static/3I/rM/P-9fPiA2KjBGj9WTOzv2ESP9JDpbUzjPeIaSjfb3IrM static/3K/0s/j_TEWaLDXKqqHPxUC-im0MxTfwwJCpI5tkMHoG93K0s static/3Q/Lk/gM_ST0n22aL3Di66t90ntLACau2tQHvl2g8rb7J3QLk static/4Q/0I/3LP1SRW2KXKgA8SpsyF-F4LHyBAoCh9_tJxkNir4Q0I

Rpkispool Construction The vantage points are relying party instances which periodically perform synchronization and validation and produce a CCR for each iteration. After each iteration the previous and current CCR are compared to deduce which objects are newly discovered. Newly discovered objects are appended to an hourly tar archive together with the CCR and any other auxiliary data files. These hourly intermediate states are collected and materialized to a durable distributed filesystem serve as input to the daily compactization process. The daily compactization proces deduplicates objects, normalizes timestamps using the method specified in , and performs Zstandard compression.

RPKIViews Mirrors The project produces a daily rpkispool combined from 10+ vantage points worldwide. Starting January 1st, 2026, every day a new rpkispool file is made available containing all yesterday's RPKI data. These rpkispool files are mirrored at the following publicly accessible locations:

Netherlands

rsync://josephine.sobornost.net/rpki/rpkispools/

Japan

rsync://dango.attn.jp/rpki/rpkispools/

United States

rsync://rpkiviews.kerfuffle.net/rpki/rpkispools/

Security Considerations The storage format provides no authenticity and may appear to be zip bombs.

IANA Considerations This document has no IANA actions.

References Normative References This document describes the commonly used base 64, base 32, and base 16 encoding schemes. It also discusses the use of line-feeds in encoded data, use of padding in encoded data, use of non-alphabet characters in encoded data, use of different encoding alphabets, and canonical encodings. [STANDARDS-TRACK] This document defines a standard profile for X.509 certificates for the purpose of supporting validation of assertions of "right-of-use" of Internet Number Resources (INRs). The certificates issued under this profile are used to convey the issuer's authorization of the subject to be regarded as the current holder of a "right-of-use" of the INRs that are described in the certificate. This document contains the normative specification of Certificate and Certificate Revocation List (CRL) syntax in the Resource Public Key Infrastructure (RPKI). This document also specifies profiles for the format of certificate requests and specifies the Relying Party RPKI certificate path validation procedure. [STANDARDS-TRACK] This document defines a generic profile for signed objects used in the Resource Public Key Infrastructure (RPKI). These RPKI signed objects make use of Cryptographic Message Syntax (CMS) as a standard encapsulation format. [STANDARDS-TRACK] Zstandard, or "zstd" (pronounced "zee standard"), is a lossless data compression mechanism. This document describes the mechanism and registers a media type, content encoding, and a structured syntax suffix to be used when transporting zstd-compressed content via MIME. Despite use of the word "standard" as part of Zstandard, readers are advised that this document is not an Internet Standards Track specification; it is being published for informational purposes only. This document replaces and obsoletes RFC 8478. In the Resource Public Key Infrastructure (RPKI), Signed Objects are defined as Cryptographic Message Syntax (CMS) protected content types. A Signed Object contains a signing-time attribute, representing the purported time at which the object was signed by its issuer. RPKI repositories are accessible using the rsync and RPKI Repository Delta protocols, allowing Relying Parties (RPs) to synchronize a local copy of the RPKI repository used for validation with the remote repositories. This document describes how the CMS signing-time attribute can be used to avoid needless retransfers of data when switching between different synchronization protocols. This document updates RFC 6488 by mandating the presence of the CMS signing-time attribute and disallowing the use of the binary-signing-time attribute. BSD Software Development RIPE NCC RIPE NCC OpenBSD This document specifies a Canonical Cache Representation (CCR) content type for use with the Resource Public Key Infrastructure (RPKI). CCR is a DER-encoded data interchange format which can be used to represent various aspects of the state of a validated cache at a particular point in time. The CCR profile is a compact and versatile format well-suited for a diverse set of applications such as audit trail keeping, validated payload dissemination, and analytics pipelines. National Institute of Standards and Technology IEEE/Open Group Informative References