| Network Working Group | W. Kumari |
| Internet-Draft | |
| Intended status: Informational | C. Doyle |
| Expires: November 12, 2020 | Juniper Networks |
| May 11, 2020 |
Secure Device Install
draft-ietf-opsawg-sdi-10
Deploying a new network device in a location where the operator has no staff of its own often requires that an employee physically travel to the location to perform the initial install and configuration, even in shared datacenters with "smart-hands" type support. In many cases, this could be avoided if there were a secure way to initially provision the device.
This document extends existing auto-install / Zero-Touch Provisioning mechanisms to make the process more secure.
[ Ed note: Text inside square brackets ([]) is additional background information, answers to frequently asked questions, general musings, etc. They will be removed before publication. This document is being collaborated on in Github at: https://github.com/wkumari/draft-wkumari-opsawg-sdi. The most recent version of the document, open issues, etc should all be available here. The authors (gratefully) accept pull requests. ]
[ Ed note: This document introduces concepts and serves as the basic for discussion - because of this, it is conversational, and would need to be firmed up before being published ]
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 12, 2020.
Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
In a growing, global network, significant amounts of time and money are spent deploying new devices and "forklift" upgrading existing devices. In many cases, these devices are in shared datacenters (for example, Internet Exchange Points (IXP) or "carrier neutral datacenters"), which have staff on hand that can be contracted to perform tasks including physical installs, device reboots, loading initial configurations, etc. There are also a number of (often vendor proprietary) protocols to perform initial device installs and configurations - for example, many network devices will attempt to use DHCP [RFC2131]to get an IP address and configuration server, and then fetch and install a configuration when they are first powered on.
The configurations of network devices contain a significant amount of security related and/or proprietary information (for example, RADIUS [RFC2865] or TACACS+ [I-D.ietf-opsawg-tacacs] secrets). Exposing these to a third party to load onto a new device (or using an auto-install techniques which fetch an unencrypted config file via TFTP [RFC1350]) or something similar, is an unacceptable security risk for many operators, and so they send employees to remote locations to perform the initial configuration work; this costs, time and money.
There are some workarounds to this, such as asking the vendor to pre- configure the devices before shipping it; asking the smart-hands to install a terminal server; providing a minimal, unsecured configuration and using that to bootstrap to a complete configuration, etc; but these are often clumsy and have security issues - for example, in the terminal server case, the console port connection could be easily snooped.
This document layers security onto existing auto-install solutions to provide a secure method to initially configure new devices. It is optimized for simplicity, both for the implementor and the operator; it is explicitly not intended to be an "all singing, all dancing" fully featured system for managing installed / deployed devices, nor is it intended to solve all use-cases - rather it is a simple targeted solution to solve a common operational issue where the network device has been delivered, fibre laid (as appropriate) but there is no trusted member of the operator's staff to perform the initial configuration.
Solutions such as Secure Zero Touch Provisioning (SZTP)", [I-D.ietf-anima-bootstrapping-keyinfra] and similar are much more fully featured, but also more complex to implement and/or are not widely deployed yet.
This solution is specifically designed to be a simple method on top of exiting device functionality. If devices do not support this new method, they can continue to use the existing functionality. In addition, operators can choose to use this to protect their configuration information, or can continue to use the existing functionality.
The issue of securely installing devices is in no way a new issue, nor is it limited to network devices; it occurs when deploying servers, PCs, IoT devices, and in many other situations. While the solution described in this document is obvious (encrypt the config, then decrypt it with a device key), this document only discusses the use for network devices, such as routers and switches.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
Most network devices already include some sort of initial bootstrapping logic (sometimes called 'autoboot', or 'autoinstall'). This generally works by having a newly installed / unconfigured device obtain an IP address and address of a config server (often called 'next-server', 'siaddr' or 'tftp-server-name') using DHCP (see [RFC2131]). The device then contacts this configuration server to download its initial configuration, which is often identified using the devices serial number, MAC address or similar. This document extends this (vendor specific) paradigm by allowing the configuration file to be encrypted.
This document describes a concept, and some example ways of implementing this concept. As devices have different capabilities, and use different configuration paradigms, one method will not suit all, and so it is expected that vendors will differ in exactly how they implement this.
This document uses the serial number of the device as a unique device identifier for simplicity; some vendors may not want to implement the system using the serial number as the identifier for business reasons (a competitor or similar could enumerate the serial numbers and determine how many devices have been manufactured). Implementors are free to choose some other way of generating identifiers (e.g., UUID [RFC4122]), but this will likely make it somewhat harder for operators to use (the serial number is usually easy to find on a device, a more complex system is likely harder to track).
[ Ed note: This example also uses TFTP because that is what many vendors use in their auto-install / ZTP feature. It could easily instead be HTTP, FTP, etc. ]
Operator_A needs another peering router, and so they order another router from Vendor_B, to be drop-shipped to the Point of Presence (POP) / datacenter. Vendor_B begins assembling the new device, and tells Operator_A what the new device's serial number will be (SN:17894321). When Vendor_B first installs the firmware on the device and boots it, the device generates a public-private keypair, and Vendor_B publishes the public key on their keyserver (in a public key certificate, for ease of use).
While the device is being shipped, Operator_A generates the initial device configuration, fetches the certificate from Vendor_B keyservers by providing the serial number of the new device. Operator_A then encrypts the device configuration and puts this encrypted config on a (local) TFTP server.
When the device arrives at the POP, it gets installed in Operator_A' rack, and cabled as instructed. The new device powers up and discovers that it has not yet been configured. It enters its autoboot state, and begins the DHCP process. Operator_A' DHCP server provides it with an IP address and the address of the configuration server. The router uses TFTP to fetch its config file (note that all this is existing functionality). The device attempts to load the config file - if the config file is unparsable, (new functionality) the device tries to use its private key to decrypt the file, and, assuming it validates, installs the new configuration.
Only the "correct" device will have the required private key and be able to decrypt and use the config file (See Security Considerations). An attacker would be able to connect to the network and get an IP address. They would also be able to retrieve (encrypted) config files by guessing serial numbers (or perhaps the server would allow directory listing), but without the private keys an attacker will not be able to decrypt the files.
This section describes the vendors roles and responsibilities and provides an overview of what the device needs to do.
Each devices requires a public-private key keypair, and for the public part to be published and retrievable by the operator. The cryptograthic algorithm and key lengths to be used are out of the scope of this document. This section illustrates one method, but, as with much of this document the exact mechanism may vary by vendor. EST [RFC7030]and [I-D.gutmann-scep] are methods which vendors may want to consider.
During the manufacturing stage, when the device is initially powered on, it will generate a public-private keypair. It will send its unique device identifier and the public key to the vendor's Certificate Publication Server to be published. The vendor's Certificate Publication Server should only accept certificates from the manufacturing facility, and which match vendor defined policies (for example, extended key usage, extensions, etc.) Note that some devices may be constrained, and so may send the raw public key and unique device identifier to the certificate publication server, while more capable devices may generate and send self-signed certificates.
The certificate publication server contains a database of certificates. If newly manufactured devices upload certificates the certificate publication server can simply publish these; if the devices provide the raw public keys and unique device identifier, the certificate publication server will need to wrap these in a certificate.
The customers (e.g., Operator_A) query this server with the serial number (or other provided unique identifier) of a device, and retrieve the associated certificate. It is expected that operators will receive the unique device identifier (serial number) of devices when they purchase them, and will download and store / cache the certificate. This means that there is not a hard requirement on the uptime / reachability of the certificate publication server.
+------------+
+------+ |Certificate |
|Device| |Publication |
+------+ | Server |
+------------+
+----------------+ +--------------+
| +---------+ | | |
| | Initial | | | |
| | boot? | | | |
| +----+----+ | | |
| | | | |
| +------v-----+ | | |
| | Generate | | | |
| |Self-signed | | | |
| |Certificate | | | |
| +------------+ | | |
| | | | +-------+ |
| +-------|---|-->|Receive| |
| | | +---+---+ |
| | | | |
| | | +---v---+ |
| | | |Publish| |
| | | +-------+ |
| | | |
+----------------+ +--------------+
Initial certificate generation and publication.
When purchasing a new device, the accounting department will need to get the unique device identifier (likely serial number) of the new device and communicate it to the operations group.
The operator will contact the vendor's publication server, and download the certificate (by providing the unique device identifier of the device). The operator SHOULD fetch the certificate using a secure transport (e.g., HTTPS). The operator will then encrypt the initial configuration (for example, using SMIME [RFC5751]) using the key in the certificate, and place it on their TFTP server. See Appendix B for examples.
+------------+
+--------+ |Certificate |
|Operator| |Publication |
+--------+ | Server |
+------------+
+----------------+ +----------------+
| +-----------+ | | +-----------+ |
| | Fetch | | | | | |
| | Device |<------>|Certificate| |
| |Certificate| | | | | |
| +-----+-----+ | | +-----------+ |
| | | | |
| +-----v------+ | | |
| | Encrypt | | | |
| | Device | | | |
| | Config | | | |
| +-----+------+ | | |
| | | | |
| +-----v------+ | | |
| | Publish | | | |
| | TFTP | | | |
| | Server | | | |
| +------------+ | | |
| | | |
+----------------+ +----------------+
Fetching the certificate, encrypting the configuration, publishing the encrypted configuration.
When the device is first booted by the customer (and on subsequent boots), if the device does not have a valid configuration, it will use existing auto-install functionality. As an example, it performs DHCP Discovery until it gets a DHCP offer including DHCP option 66 (Server-Name) or 150 (TFTP server address), contacts the server listed in these DHCP options and downloads its config file. Note that this is existing functionality (for example, Cisco devices fetch the config file named by the Bootfile-Name DHCP option (67)).
After retrieving the config file, the device needs to determine if it is encrypted or not. If it is not encrypted, the existing behavior is used. If the configuration is encrypted, the process continues as described in this document. The method used to determine if the config is encrypted or not is implementation dependant; there are a number of (obvious) options, including having a magic string in the file header, using a file name extension (e.g., config.enc), or using specific DHCP options.
If the file is encrypted, the device will attempt to decrypt and parse the file. If able, it will install the configuration, and start using it. If it cannot decrypt the file, or if parsing the configurations fails, the device will either abort the auto-install process, or will repeat this process until it succeeds. When retrying, care should be taken to not overwhelm the server hosting the encrypted configuration files. It is suggested that the device retry every 5 minutes for the first hour, and then every hour after that. As it is expected that devices may be installed well before the configuration file is ready, a maximum number of retrys is not specified.
Note that the device only needs to be able to download the config file; after the initial power-on in the factory it never needs to access the Internet or vendor or certificate publication server - it (and only it) has the private key and so has the ability to decrypt the config file.
+--------+ +--------------+
| Device | |Config server |
+--------+ | (e.g. TFTP) |
+--------------+
+---------------------------+ +------------------+
| +-----------+ | | |
| | | | | |
| | DHCP | | | |
| | | | | |
| +-----+-----+ | | |
| | | | |
| +-----v------+ | | +-----------+ |
| | | | | | Encrypted | |
| |Fetch config|<------------------>| config | |
| | | | | | file | |
| +-----+------+ | | +-----------+ |
| | | | |
| X | | |
| / \ | | |
| / \ N +--------+ | | |
| | Enc?|---->|Install,| | | |
| \ / | Boot | | | |
| \ / +--------+ | | |
| V | | |
| |Y | | |
| | | | |
| +-----v------+ | | |
| |Decrypt with| | | |
| |private key | | | |
| +-----+------+ | | |
| | | | |
| v | | |
| / \ | | |
| / \ Y +--------+ | | |
| |Sane?|---->|Install,| | | |
| \ / | Boot | | | |
| \ / +--------+ | | |
| V | | |
| |N | | |
| | | | |
| +----v---+ | | |
| |Give up,| | | |
| |go home | | | |
| +--------+ | | |
| | | |
+---------------------------+ +------------------+
Device boot, fetch and install config file
Ideally, the keypair would be stored in a Trusted Platform Module (TPM) on something which is identified as the “router” - for example, the chassis / backplane. This is so that a keypair is bound to what humans think of as the “device”, and not, for example (redundant) routing engines. Devices which implement IEEE 802.1AR [IEEE802-1AR] could choose to use the IDevID for this purpose.
It is anticipated that some operator may want to replace the (vendor provided) keys after installing the device. There are two options when implementing this - a vendor could allow the operator's key to completely replace the initial device generated key (which means that, if the device is ever sold, the new owner couldn't use this technique to install the device), or the device could prefer the operators installed key. This is an implementation decision left to the vendor.
Increasingly, operations is moving towards an automated model of device management, whereby portions (or the entire) configuration is programmatically generated. This means that operators may want to generate an entire configuration after the device has been initially installed and ask the device to load and use this new configuration. It is expected (but not defined in this document, as it is vendor specific) that vendors will allow the operator to copy a new, encrypted config (or part of a config) onto a device and then request that the device decrypt and install it (e.g.: ‘load replace <filename> encrypted)). The operator could also choose to reset the device to factory defaults, and allow the device to act as though it were the initial boot (see Section 4.3).
This document makes no requests of the IANA.
This mechanism is intended to replace either expensive (traveling employees) or insecure mechanisms of installing newly deployed devices such as: unencrypted config files which can be downloaded by connecting to unprotected ports in datacenters, mailing initial config files on flash drives, or emailing config files and asking a third-party to copy and paste it over a serial terminal. It does not protect against devices with malicious firmware, nor theft and reuse of devices.
An attacker (e.g., a malicious datacenter employee) who has physical access to the device before it is connected to the network the attacker may be able to extract the device private key (especially if it is not stored in a TPM), pretend to be the device when connecting to the network, and download and extract the (encrypted) config file.
This mechanism does not protect against a malicious vendor - while the keypair should be generated on the device, and the private key should be securely stored, the mechanism cannot detect or protect against a vendor who claims to do this, but instead generates the keypair off device and keeps a copy of the private key. It is largely understood in the operator community that a malicious vendor or attacker with physical access to the device is largely a "Game Over" situation.
Even when using a secure bootstrapping mechanism, security conscious operators may wish to bootstrapping devices with a minimal / less sensitive config, and then replace this with a more complete one after install.
The authors wish to thank everyone who contributed, including Benoit Claise, Francis Dupont, Mirja Kuehlewind, Sam Ribeiro, Michael Richardson, Sean Turner and Kent Watsen. Joe Clarke also provided significant comments and review, and Tom Petch provided significant editorial contributions to better describe the use cases, and clarify the scope.
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997. |
| [RFC8174] | Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017. |
[RFC Editor: Please remove this section before publication ]
From -09 to -10
From -08 to -09
From -04 to -08
From -03 to -04
From individual WG-01 to -03:
From individual -04 to WG-01:
From -00 to -01
From -01 to -03:
From -03 to -04:
Addressed a number of comments received before / at IETF104 (Prague). These include:
This section contains a rough demo / proof of concept of the system. It is only intended for illustration, and is not intended to be used in production.
It uses OpenSSL from the command line, in production something more automated would be used. In this example, the unique device identifier is the serial number of the router, SN19842256.
This step is performed by the router. It generates a key, then a csr, and then a self signed certificate.
$ openssl genrsa -out key.pem 2048 Generating RSA private key, 2048 bit long modulus ................................................. ................................................. ..........................+++ ...................+++ e is 65537 (0x10001)
$ openssl req -new -key key.pem -out SN19842256.csr Country Name (2 letter code) [AU]:. State or Province Name (full name) [Some-State]:. Locality Name (eg, city) []:. Organization Name (eg, company) [Internet Widgits Pty Ltd]:. Organizational Unit Name (eg, section) []:. Common Name (e.g. server FQDN or YOUR name) []:SN19842256 Email Address []:. Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:.
$ openssl req -x509 -days 36500 -key key.pem -in SN19842256.csr -out SN19842256.crt
The router then sends the key to the vendor’s keyserver for publication (not shown).
The operator now wants to deploy the new router.
They generate the initial config (using whatever magic tool generates router configs!), fetch the router’s certificate and encrypt the config file to that key. This is done by the operator.
$ wget http://keyserv.example.net/certificates/SN19842256.crt
I'm using S/MIME because it is simple to demonstrate. This is almost definitely not the best way to do this.
$ openssl smime -encrypt -aes-256-cbc -in SN19842256.cfg\ -out SN19842256.enc -outform PEM SN19842256.crt $ more SN19842256.enc -----BEGIN PKCS7----- MIICigYJKoZIhvcNAQcDoIICezCCAncCAQAxggE+MIIBOgIBADAiMBUxEzARBgNV BAMMClNOMTk4NDIyNTYCCQDJVuBlaTOb1DANBgkqhkiG9w0BAQEFAASCAQBABvM3 ... LZoq08jqlWhZZWhTKs4XPGHUdmnZRYIP8KXyEtHt -----END PKCS7-----
$ scp SN19842256.enc config.example.com:/tftpboot
When the router connects to the operator's network it will detect that does not have a valid configuration file, and will start the “autoboot” process. This is a well documented process, but the high level overview is that it will use DHCP to obtain an IP address and config server. It will then use TFTP to download a configuration file, based upon its serial number (this document modifies the solution to fetch an encrypted config file (ending in .enc)). It will then decrypt the config file, and install it.
$ tftp 2001:0db8::23 -c get SN19842256.enc
$ openssl smime -decrypt -in SN19842256.enc -inform pkcs7\ -out config.cfg -inkey key.pem
If an attacker does not have the correct key, they will not be able to decrypt the config:
$ openssl smime -decrypt -in SN19842256.enc -inform pkcs7\ -out config.cfg -inkey wrongkey.pem Error decrypting PKCS#7 structure 140352450692760:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:evp_enc.c:592: $ echo $? 4