1ACVP Client 2=========== 3 4[ACVP](https://github.com/usnistgov/ACVP) is the next version of NIST's [CAVP](https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program)—a program for running cryptographic implementations against a set of test vectors. CAVP involved emailing around zip files of somewhat-INI-like test vectors where no two files had quite the same format. ACVP is supposed to replace that with a) TLS connections rather than email (but not yet) and b) JSON rather than bespoke formats. 5 6The tool in this directory can speak to ACVP servers, fetch test vectors, and lower the operations to a simpler protocol. A wrapper around the FIPS module in question implements that protocol for testing. Test vectors can also come from files on disk. 7 8The tool also provides an interface for querying and manipulating the ACVP database, which contains lists of modules, vendors, contacts, operating environments, etc. 9 10For an alternative, see [libacvp](https://github.com/cisco/libacvp). 11 12## Building 13 14This tool builds independently of the rest of the BoringSSL: just run `go build` within the `acvptool` directory. 15 16## File-based operation 17 18While ACVP hopes to replace emailing files around at some point, emailing is still common practice with most NVLAP labs. To process a file of test vectors, use the `-json` flag: 19 20``` 21% ./acvptool -json input.json > output.json 22``` 23 24The top-level structure of these JSON files is not specified by NIST. This tool consumes the form that appears to be most commonly used. 25 26The lab will need to know the configuration of the module to generate tests. Obtain that with the `-regcap` option and redirect the output to a file. 27 28### Testing other FIPS modules 29 30Lowering ACVP to a simpler form might be useful for other modules so the protocol is described here. The tool has far from complete coverage of ACVP's mountain of options, but common cases are handled. If you have additional needs then it's hopefully straightforward to extend the tool yourself. 31 32The FIPS module being tested needs to be wrapped such that the tool can fork and exec a binary that speaks this protocol over stdin/stdout. For BoringSSL that binary is in the `modulewrapper` directory and serves as a reference implementation if you have questions about the protocol that aren't answered below. BoringSSL's modulewrapper contains the FIPS module itself, but your binary could forward the communication over, e.g., a serial link to a hardware module. Specify the path to the binary with the `-wrapper` option. 33 34The protocol is request–response: the subprocess only speaks in response to a request and there is exactly one response for every request. Requests consist of one or more byte strings and responses consist of zero or more byte strings. 35 36A request contains: the number of byte strings, the length of each byte string, and the contents of each byte string. All numbers are 32-bit little-endian and values are concatenated in the order specified. The first byte string is mandatory and is the name of the command to perform. A response has the same format except that there may be zero byte strings and the first byte string has no special meaning. 37 38All implementations must support the `getConfig` command which takes no arguments and returns a single byte string which is a JSON blob of ACVP algorithm configuration. This blob describes all the algorithms and capabilities that the module supports and is an array of JSON objects suitable for including as the `algorithms` value when [creating an ACVP vector set](http://usnistgov.github.io/ACVP/artifacts/draft-fussell-acvp-spec-00.html#rfc.section.11.15.2.1). 39 40Crafting this JSON is an art. You can get some information from reading the [NIST documentation](https://github.com/usnistgov/ACVP#supported-algorithms) but you might also want to crib from the BoringSSL wrapper in `modulewrapper`. 41 42The other commands are as follows. (Note that you only need to implement the commands required by the ACVP configuration returned.) 43 44| Command | Arguments | Outputs | 45|----------------------|---------------------------|---------| 46| 3DES-CBC/decrypt | Key, ciphertext, IV, num iterations¹ | Result, Previous result | 47| 3DES-CBC/encrypt | Key, plaintext, IV, num iterations¹ | Result, Previous result | 48| 3DES/decrypt | Key, input block, num iterations¹ | Result, Previous result | 49| 3DES/encrypt | Key, input block, num iterations¹ | Result, Previous result | 50| AES-CBC/decrypt | Key, ciphertext, IV, num iterations¹ | Result, Previous result | 51| AES-CBC/encrypt | Key, plaintext, IV, num iterations¹ | Result, Previous result | 52| AES-CBC-CS3/decrypt | Key, ciphertext, IV, num iterations² | Result | 53| AES-CBC-CS3/encrypt | Key, plaintext, IV, num iterations² | Result | 54| AES-CCM/open | Tag length, key, ciphertext, nonce, ad | One-byte success flag, plaintext or empty | 55| AES-CCM/seal | Tag length, key, plaintext, nonce, ad | Ciphertext | 56| AES-CTR/decrypt | Key, ciphertext, initial counter, constant 1 | Plaintext | 57| AES-CTR/encrypt | Key, plaintexttext, initial counter, constant 1 | Ciphertext | 58| AES-GCM/open | Tag length, key, ciphertext, nonce, ad | One-byte success flag, plaintext or empty | 59| AES-GCM/seal | Tag length, key, plaintext, nonce, ad | Ciphertext | 60| AES-GCM-randnonce/open | Tag length, key, ciphertext (with rand nonce appended), nonce (empty), ad | One-byte success flag, plaintext or empty | 61| AES-GCM-randnonce/seal | Tag length, key, plaintext, nonce (empty), ad | Ciphertext (with rand nonce appended) | 62| AES-KW/open | (dummy), key, ciphertext, (dummy), (dummy) | One-byte success flag, plaintext or empty | 63| AES-KW/seal | (dummy), key, plaintext, (dummy), (dummy) | Ciphertext | 64| AES-KWP/open | (dummy), key, ciphertext, (dummy), (dummy) | One-byte success flag, plaintext or empty | 65| AES-KWP/seal | (dummy), key, plaintext, (dummy), (dummy) | Ciphertext | 66| AES-XTS/decrypt | Key, ciphertext, tweak | Plaintext | 67| AES-XTS/encrypt | Key, plaintext, tweak | Ciphertext | 68| AES/decrypt | Key, input block, num iterations¹ | Result, Previous result | 69| AES/encrypt | Key, input block, num iterations¹ | Result, Previous result | 70| CMAC-AES | Number output bytes, key, message | MAC | 71| CMAC-AES/verify | Key, message, claimed MAC | One-byte success flag | 72| ctrDRBG/AES-256 | Output length, entropy, personalisation, ad1, ad2, nonce | Output | 73| ctrDRBG-reseed/AES-256| Output length, entropy, personalisation, reseedAD, reseedEntropy, ad1, ad2, nonce | Output | 74| ctrDRBG-pr/AES-256 | Output length, entropy, personalisation, ad1, entropy1, ad2, entropy2, nonce | Output | 75| ECDH/<CURVE> | X, Y, private key | X, Y, shared key | 76| ECDSA/keyGen | Curve name | Private key, X, Y | 77| ECDSA/keyVer | Curve name, X, Y | Single-byte valid flag | 78| ECDSA/sigGen | Curve name, private key, hash name, message | R, S | 79| ECDSA/sigVer | Curve name, hash name, message, X, Y, R, S | Single-byte validity flag | 80| DetECDSA/sigGen | Curve name, private key, hash name, message | R, S | 81| EDDSA/keyGen | Curve name | private key seed (D), public key (Q) | 82| EDDSA/keyVer | Curve name, public key (Q) | Single-byte valid flag | 83| EDDSA/sigGen | Curve name, private key seed (D), message, single-byte prehash flag, prehash context | Signature | 84| EDDSA/sigVer | Curve name, message, public key (Q), signature, single-byte prehash flag | Single-byte validity flag | 85| FFDH | p, q, g, peer public key, local private key (or empty), local public key (or empty) | Local public key, shared key | 86| HKDF/<HASH> | key, salt, info, num output bytes | Key | 87| HKDFExtract/<HASH> | secret, salt | Key | 88| HKDFExpandLabel/<HASH> | Output length, secret, label, transcript hash | Key | 89| HMAC-SHA-1 | Value to hash, key | Digest | 90| HMAC-SHA2-224 | Value to hash, key | Digest | 91| HMAC-SHA2-256 | Value to hash, key | Digest | 92| HMAC-SHA2-384 | Value to hash, key | Digest | 93| HMAC-SHA2-512 | Value to hash, key | Digest | 94| HMAC-SHA2-512/224 | Value to hash, key | Digest | 95| HMAC-SHA2-512/256 | Value to hash, key | Digest | 96| hmacDRBG/<HASH>| Output length, entropy, personalisation, ad1, ad2, nonce | Output | 97| hmacDRBG-reseed/<HASH>| Output length, entropy, personalisation, reseedAD, reseedEntropy, ad1, ad2, nonce | Output | 98| hmacDRBG-pr/<HASH>| Output length, entropy, personalisation, ad1, entropy1, ad2, entropy2, nonce | Output | 99| KDF-counter | Number output bytes, PRF name, counter location string, key (or empty), number of counter bits | key, counter, derived key | 100| KDF-feedback | Number output bytes, PRF name, counter location string, key (or empty), number of counter bits | key, counter, derived key | 101| RSA/keyGen | Modulus bit-size | e, p, q, n, d | 102| RSA/sigGen/<HASH>/pkcs1v1.5 | Modulus bit-size, message | n, e, signature | 103| RSA/sigGen/<HASH>/pss | Modulus bit-size, message | n, e, signature | 104| RSA/sigVer/<HASH>/pkcs1v1.5 | n, e, message, signature | Single-byte validity flag | 105| RSA/sigVer/<HASH>/pss | n, e, message, signature | Single-byte validity flag | 106| SHA-1 | Value to hash | Digest | 107| SHA2-224 | Value to hash | Digest | 108| SHA2-256 | Value to hash | Digest | 109| SHA2-384 | Value to hash | Digest | 110| SHA2-512 | Value to hash | Digest | 111| SHA2-512/224 | Value to hash | Digest | 112| SHA2-512/256 | Value to hash | Digest | 113| SHA3-224 | Value to hash | Digest | 114| SHA3-256 | Value to hash | Digest | 115| SHA3-384 | Value to hash | Digest | 116| SHA3-512 | Value to hash | Digest | 117| SHAKE-128 | Value to hash, output length bytes | Digest | 118| SHAKE-128/VOT | Value to hash, output length bytes | Digest | 119| SHAKE-128/MCT | Initial seed¹, min output bytes, max output bytes, output length bytes | Digest, output length bytes | 120| SHAKE-256 | Value to hash, output length bytes | Digest | 121| SHAKE-256/VOT | Value to hash, output length bytes | Digest | 122| SHAKE-256/MCT | Initial seed¹, min output bytes, max output bytes, output length bytes | Digest, output length bytes | 123| cSHAKE-128 | Value to hash, output length bytes, function name bytes, customization bytes | Digest | 124| cSHAKE-128/MCT | Initial seed¹, min output bytes, max output bytes, output length bytes, customization bytes | Digest, output length bytes, customization bytes | 125| SHA-1/MCT | Initial seed¹ | Digest | 126| SHA2-224/MCT | Initial seed¹ | Digest | 127| SHA2-256/MCT | Initial seed¹ | Digest | 128| SHA2-384/MCT | Initial seed¹ | Digest | 129| SHA2-512/MCT | Initial seed¹ | Digest | 130| SHA2-512/224/MCT | Initial seed¹ | Digest | 131| SHA2-512/256/MCT | Initial seed¹ | Digest | 132| SHA3-224/MCT | Initial seed¹ | Digest | 133| SHA3-256/MCT | Initial seed¹ | Digest | 134| SHA3-384/MCT | Initial seed¹ | Digest | 135| SHA3-512/MCT | Initial seed¹ | Digest | 136| TLSKDF/1.2/<HASH> | Number output bytes, secret, label, seed1, seed2 | Output | 137| PBKDF | HMAC name, key length (bits), salt, password, iteration count | Derived key | 138| ML-DSA-XX/keyGen | Seed | Public key, private key | 139| ML-DSA-XX/sigGen | Private key, message, randomizer | Signature | 140| ML-DSA-XX/sigVer | Public key, message, signature | Single-byte validity flag | 141| ML-KEM-XX/keyGen | Seed | Public key, private key | 142| ML-KEM-XX/encap | Public key, entropy | Ciphertext, shared secret | 143| ML-KEM-XX/decap | Private key, ciphertext | Shared secret | 144| SLH-DSA-XX/keyGen | Seed | Private key, public key | 145| SLH-DSA-XX/sigGen | Private key, message, entropy or empty | Signature | 146| SLH-DSA-XX/sigVer | Public key, message, signature | Single-byte validity flag | 147| SSHKDF/<HASH>/client | K, H, SessionID, cipher algorithm | client IV key, client encryption key, client integrity key | 148| SSHKDF/<HASH>/server | K, H, SessionID, cipher algorithm | server IV key, server encryption key, server integrity key | 149| KTS-IFC/<HASH>/initiator | output length bytes, serverN bytes, serverE bytes | generated ciphertext (iutC), derived keying material (dkm) | 150| KTS-IFC/<HASH>/responder | iutN bytes, iutE bytes, iutP bytes, iutQ bytes, iutD bytes, ciphertext (serverC) bytes | derived keying material (dkm) | 151| OneStepNoCounter/<HASH> | key, info, salt, output length bytes | derived key | 152 153¹ The iterated tests would result in excessive numbers of round trips if the module wrapper handled only basic operations. Thus some ACVP logic is pushed down for these tests so that the inner loop can be handled locally. Either read the NIST documentation ([block-ciphers](https://pages.nist.gov/ACVP/draft-celi-acvp-symmetric.html#name-monte-carlo-tests-for-block) [hashes](https://pages.nist.gov/ACVP/draft-celi-acvp-sha.html#name-monte-carlo-tests-for-sha-1)) to understand the iteration count and return values or, probably more fruitfully, see how these functions are handled in the `modulewrapper` directory. 154 155² Will always be one because MCT tests are not supported for CS3. 156 157### Batching 158 159Requests are written without waiting for responses. Implementations can run a read-execute-reply loop without worrying about this. However, if batching is useful then implementations may gather up multiple requests before executing them. But this risks deadlock because some requests depend on the result of the previous one. If the `getConfig` result contains a dummy entry for the algorithm `acvptool` it will be filtered out when running with `-regcap`. However, a list of strings called `features` in that block may include the string `batch` to indicate that the implementation would like to receive a `flush` command whenever previous results must be received in order to progress. Implementations that batch can observe this to avoid deadlock. 160 161The `flush` command must not produce a response itself; it only indicates that all previous responses must be received to progress. The `getConfig` command must always be serviced immediately because a flush command will not be sent prior to processing the `getConfig` response. 162 163## Online operation 164 165If you have credentials to speak to either of the NIST ACVP servers then you can run the tool in online mode. 166 167Configuration is done via a `config.json` file in the current working directory. Here's a template: 168 169``` 170{ 171 "ACVPServer": "https://demo.acvts.nist.gov/", 172 "CertPEMFile": "certificate_from_nist.pem", 173 "PrivateKeyFile": "your_private_key.key", 174 "TOTPSecret": "<base64 from NIST goes here>", 175 "SessionTokensCache": "~/.cache/acvp-session-tokens", 176 "LogFile": "log" 177} 178``` 179 180NIST's ACVP servers use both TLS client certificates and TOTP for authentication. When registering with NIST they'll sign a CSR and return a certificate in PEM format, which is pointed to by `CertPEMFile`. The corresponding private key is expected in `PrivateKeyFile`. Lastly, NIST will provide a file that contains the base64-encoded TOTP seed, which must be pasted in as the value of `TOTPSecret`. 181 182NIST's ACVP server provides special access tokens for each test session and test sessions can _only_ be accessed via those tokens. The reasoning behind this is unclear but this client can, optionally, keep records of these access tokens in the directory named by `SessionTokensCache`. If that directory name begins with `~/` then that prefix will be replaced with the value of `$HOME`. 183 184Lastly, a log of all HTTP traffic will be written to the file named by `LogFile`, if provided. This is useful for debugging. 185 186### Interactive Use 187 188ACVP provides a fairly complex interface to a database of several types of objects. A rough UI is provided for this which is triggered when the client is invoked with no command-line arguments. 189 190The simplest objects in ACVP are request objects. These record the status of requested changes to the database and, in practice, changes to the NIST demo database never succeed. The set of pending requests for the current user can be enumerated just by evaluating the `requests` object: 191 192``` 193> requests 194[ 195 { 196 "url": "/acvp/v1/requests/374", 197 "status": "processing" 198 }, 199 { 200 "url": "/acvp/v1/requests/218", 201 "status": "processing" 202 } 203] 204``` 205 206A specific request can be evaluated by using indexing syntax: 207 208``` 209> requests[374] 210{ 211 "url": "/acvp/v1/requests/374", 212 "status": "processing" 213} 214``` 215 216The list of vendors provides a more complex example. Since there are large number of duplicates in NIST's database, there are more than 10 000 vendor objects and enumerating them all takes a long time. Thus evaluating the `vendors` object doesn't do that: 217 218``` 219> vendors 220[object set vendors] 221``` 222 223It is still possible to use indexing syntax to read a specific vendor object if you know the ID: 224 225``` 226> vendors[1234] 227{ 228 "url": "/acvp/v1/vendors/1234", 229 "name": "Apple Inc.", 230 "website": "www.apple.com", 231 "contactsUrl": "/acvp/v1/vendors/1234/contacts", 232 "addresses": [ 233 { 234 "url": "/acvp/v1/vendors/1234/addresses/1234", 235 "street1": "1 Infinite Loop", 236 "locality": "Cupertino", 237 "region": "CA", 238 "country": "USA", 239 "postalCode": "95014" 240 } 241 ] 242} 243``` 244 245Finding a vendor when the ID is not known requires searching and the ACVP spec [documents](http://usnistgov.github.io/ACVP/artifacts/draft-fussell-acvp-spec-00.html#rfc.section.11.8.1), for each object type, what values and what relations can be searched on. This is reflected in a variant of the indexing syntax: 246 247``` 248> vendors[where name contains "Google LLC"] 249[ 250 { 251 "url": "/acvp/v1/vendors/11136", 252 "name": "Google LLC", 253 "website": "www.google.com", 254 "contactsUrl": "/acvp/v1/vendors/11136/contacts", 255 "addresses": [ 256 { 257 "url": "/acvp/v1/vendors/11136/addresses/11136", 258 "street1": "1600 Amphitheatre Parkway", 259 "locality": "Mountain View", 260 "region": "CA", 261 "country": "USA", 262 "postalCode": "94043" 263 } 264 ] 265 }, 266 { 267 "url": "/acvp/v1/vendors/11137", 268 "name": "Google LLC", 269 "website": "www.google.com", 270 "contactsUrl": "/acvp/v1/vendors/11137/contacts", 271 "addresses": [ 272 { 273 "url": "/acvp/v1/vendors/11137/addresses/11137", 274 "street1": "1600 Amphitheatre Parkway", 275 "locality": "Mountain View", 276 "region": "CA", 277 "country": "USA", 278 "postalCode": "94043" 279 } 280 ] 281 } 282] 283``` 284 285In general, `&&` and `||` can be used as in C and the relationships are `==`, `!=`, `contains`, `startsWith`, and `endsWith`. Only values and relations listed in the ACVP spec for a given object can be used. 286 287More complex interaction remains to be fleshed out. However, it is generally possible to create new objects by evaluating, for example, `vendors.new()`. That will invoke `$EDITOR` to edit the JSON to be submitted. (For now, however, no helpful templates are provided.) 288 289The current list of objects is: 290 291* `requests` 292* `vendors` 293* `persons` 294* `modules` 295* `oes` (operating environments) 296* `deps` 297* `algos` 298* `sessions` 299 300### Running test sessions 301 302In online mode, a given algorithm can be run by using the `-run` option. For example, `-run SHA2-256`. This will fetch a vector set, have the module-under-test answer it, and upload the answer. If you want to just fetch the vector set for later use with the `-json` option (documented above) then you can use `-fetch` instead of `-run`. The `-fetch` option also supports passing `-expected-out <filename>` to fetch and write the expected results, if the server supports that. 303 304After results have been produced with `-json`, they can be uploaded with `-upload`. So `-run` is effectively these three steps combined: 305 306``` 307./acvptool -fetch SHA2-256 > request 308./acvptool -json request > result 309./acvptool -upload result 310``` 311