1.. SPDX-License-Identifier: GPL-2.0 2 3.. _fsverity: 4 5======================================================= 6fs-verity: read-only file-based authenticity protection 7======================================================= 8 9Introduction 10============ 11 12fs-verity (``fs/verity/``) is a support layer that filesystems can 13hook into to support transparent integrity and authenticity protection 14of read-only files. Currently, it is supported by the ext4, f2fs, and 15btrfs filesystems. Like fscrypt, not too much filesystem-specific 16code is needed to support fs-verity. 17 18fs-verity is similar to `dm-verity 19<https://www.kernel.org/doc/Documentation/device-mapper/verity.txt>`_ 20but works on files rather than block devices. On regular files on 21filesystems supporting fs-verity, userspace can execute an ioctl that 22causes the filesystem to build a Merkle tree for the file and persist 23it to a filesystem-specific location associated with the file. 24 25After this, the file is made readonly, and all reads from the file are 26automatically verified against the file's Merkle tree. Reads of any 27corrupted data, including mmap reads, will fail. 28 29Userspace can use another ioctl to retrieve the root hash (actually 30the "fs-verity file digest", which is a hash that includes the Merkle 31tree root hash) that fs-verity is enforcing for the file. This ioctl 32executes in constant time, regardless of the file size. 33 34fs-verity is essentially a way to hash a file in constant time, 35subject to the caveat that reads which would violate the hash will 36fail at runtime. 37 38Use cases 39========= 40 41By itself, the base fs-verity feature only provides integrity 42protection, i.e. detection of accidental (non-malicious) corruption. 43 44However, because fs-verity makes retrieving the file hash extremely 45efficient, it's primarily meant to be used as a tool to support 46authentication (detection of malicious modifications) or auditing 47(logging file hashes before use). 48 49Trusted userspace code (e.g. operating system code running on a 50read-only partition that is itself authenticated by dm-verity) can 51authenticate the contents of an fs-verity file by using the 52`FS_IOC_MEASURE_VERITY`_ ioctl to retrieve its hash, then verifying a 53digital signature of it. 54 55A standard file hash could be used instead of fs-verity. However, 56this is inefficient if the file is large and only a small portion may 57be accessed. This is often the case for Android application package 58(APK) files, for example. These typically contain many translations, 59classes, and other resources that are infrequently or even never 60accessed on a particular device. It would be slow and wasteful to 61read and hash the entire file before starting the application. 62 63Unlike an ahead-of-time hash, fs-verity also re-verifies data each 64time it's paged in. This ensures that malicious disk firmware can't 65undetectably change the contents of the file at runtime. 66 67fs-verity does not replace or obsolete dm-verity. dm-verity should 68still be used on read-only filesystems. fs-verity is for files that 69must live on a read-write filesystem because they are independently 70updated and potentially user-installed, so dm-verity cannot be used. 71 72The base fs-verity feature is a hashing mechanism only; actually 73authenticating the files may be done by: 74 75* Userspace-only 76 77* Builtin signature verification + userspace policy 78 79 fs-verity optionally supports a simple signature verification 80 mechanism where users can configure the kernel to require that 81 all fs-verity files be signed by a key loaded into a keyring; 82 see `Built-in signature verification`_. 83 84* Integrity Measurement Architecture (IMA) 85 86 IMA supports including fs-verity file digests and signatures in the 87 IMA measurement list and verifying fs-verity based file signatures 88 stored as security.ima xattrs, based on policy. 89 90 91User API 92======== 93 94FS_IOC_ENABLE_VERITY 95-------------------- 96 97The FS_IOC_ENABLE_VERITY ioctl enables fs-verity on a file. It takes 98in a pointer to a struct fsverity_enable_arg, defined as 99follows:: 100 101 struct fsverity_enable_arg { 102 __u32 version; 103 __u32 hash_algorithm; 104 __u32 block_size; 105 __u32 salt_size; 106 __u64 salt_ptr; 107 __u32 sig_size; 108 __u32 __reserved1; 109 __u64 sig_ptr; 110 __u64 __reserved2[11]; 111 }; 112 113This structure contains the parameters of the Merkle tree to build for 114the file, and optionally contains a signature. It must be initialized 115as follows: 116 117- ``version`` must be 1. 118- ``hash_algorithm`` must be the identifier for the hash algorithm to 119 use for the Merkle tree, such as FS_VERITY_HASH_ALG_SHA256. See 120 ``include/uapi/linux/fsverity.h`` for the list of possible values. 121- ``block_size`` is the Merkle tree block size, in bytes. In Linux 122 v6.3 and later, this can be any power of 2 between (inclusively) 123 1024 and the minimum of the system page size and the filesystem 124 block size. In earlier versions, the page size was the only allowed 125 value. 126- ``salt_size`` is the size of the salt in bytes, or 0 if no salt is 127 provided. The salt is a value that is prepended to every hashed 128 block; it can be used to personalize the hashing for a particular 129 file or device. Currently the maximum salt size is 32 bytes. 130- ``salt_ptr`` is the pointer to the salt, or NULL if no salt is 131 provided. 132- ``sig_size`` is the size of the signature in bytes, or 0 if no 133 signature is provided. Currently the signature is (somewhat 134 arbitrarily) limited to 16128 bytes. See `Built-in signature 135 verification`_ for more information. 136- ``sig_ptr`` is the pointer to the signature, or NULL if no 137 signature is provided. 138- All reserved fields must be zeroed. 139 140FS_IOC_ENABLE_VERITY causes the filesystem to build a Merkle tree for 141the file and persist it to a filesystem-specific location associated 142with the file, then mark the file as a verity file. This ioctl may 143take a long time to execute on large files, and it is interruptible by 144fatal signals. 145 146FS_IOC_ENABLE_VERITY checks for write access to the inode. However, 147it must be executed on an O_RDONLY file descriptor and no processes 148can have the file open for writing. Attempts to open the file for 149writing while this ioctl is executing will fail with ETXTBSY. (This 150is necessary to guarantee that no writable file descriptors will exist 151after verity is enabled, and to guarantee that the file's contents are 152stable while the Merkle tree is being built over it.) 153 154On success, FS_IOC_ENABLE_VERITY returns 0, and the file becomes a 155verity file. On failure (including the case of interruption by a 156fatal signal), no changes are made to the file. 157 158FS_IOC_ENABLE_VERITY can fail with the following errors: 159 160- ``EACCES``: the process does not have write access to the file 161- ``EBADMSG``: the signature is malformed 162- ``EBUSY``: this ioctl is already running on the file 163- ``EEXIST``: the file already has verity enabled 164- ``EFAULT``: the caller provided inaccessible memory 165- ``EFBIG``: the file is too large to enable verity on 166- ``EINTR``: the operation was interrupted by a fatal signal 167- ``EINVAL``: unsupported version, hash algorithm, or block size; or 168 reserved bits are set; or the file descriptor refers to neither a 169 regular file nor a directory. 170- ``EISDIR``: the file descriptor refers to a directory 171- ``EKEYREJECTED``: the signature doesn't match the file 172- ``EMSGSIZE``: the salt or signature is too long 173- ``ENOKEY``: the fs-verity keyring doesn't contain the certificate 174 needed to verify the signature 175- ``ENOPKG``: fs-verity recognizes the hash algorithm, but it's not 176 available in the kernel's crypto API as currently configured (e.g. 177 for SHA-512, missing CONFIG_CRYPTO_SHA512). 178- ``ENOTTY``: this type of filesystem does not implement fs-verity 179- ``EOPNOTSUPP``: the kernel was not configured with fs-verity 180 support; or the filesystem superblock has not had the 'verity' 181 feature enabled on it; or the filesystem does not support fs-verity 182 on this file. (See `Filesystem support`_.) 183- ``EPERM``: the file is append-only; or, a signature is required and 184 one was not provided. 185- ``EROFS``: the filesystem is read-only 186- ``ETXTBSY``: someone has the file open for writing. This can be the 187 caller's file descriptor, another open file descriptor, or the file 188 reference held by a writable memory map. 189 190FS_IOC_MEASURE_VERITY 191--------------------- 192 193The FS_IOC_MEASURE_VERITY ioctl retrieves the digest of a verity file. 194The fs-verity file digest is a cryptographic digest that identifies 195the file contents that are being enforced on reads; it is computed via 196a Merkle tree and is different from a traditional full-file digest. 197 198This ioctl takes in a pointer to a variable-length structure:: 199 200 struct fsverity_digest { 201 __u16 digest_algorithm; 202 __u16 digest_size; /* input/output */ 203 __u8 digest[]; 204 }; 205 206``digest_size`` is an input/output field. On input, it must be 207initialized to the number of bytes allocated for the variable-length 208``digest`` field. 209 210On success, 0 is returned and the kernel fills in the structure as 211follows: 212 213- ``digest_algorithm`` will be the hash algorithm used for the file 214 digest. It will match ``fsverity_enable_arg::hash_algorithm``. 215- ``digest_size`` will be the size of the digest in bytes, e.g. 32 216 for SHA-256. (This can be redundant with ``digest_algorithm``.) 217- ``digest`` will be the actual bytes of the digest. 218 219FS_IOC_MEASURE_VERITY is guaranteed to execute in constant time, 220regardless of the size of the file. 221 222FS_IOC_MEASURE_VERITY can fail with the following errors: 223 224- ``EFAULT``: the caller provided inaccessible memory 225- ``ENODATA``: the file is not a verity file 226- ``ENOTTY``: this type of filesystem does not implement fs-verity 227- ``EOPNOTSUPP``: the kernel was not configured with fs-verity 228 support, or the filesystem superblock has not had the 'verity' 229 feature enabled on it. (See `Filesystem support`_.) 230- ``EOVERFLOW``: the digest is longer than the specified 231 ``digest_size`` bytes. Try providing a larger buffer. 232 233FS_IOC_READ_VERITY_METADATA 234--------------------------- 235 236The FS_IOC_READ_VERITY_METADATA ioctl reads verity metadata from a 237verity file. This ioctl is available since Linux v5.12. 238 239This ioctl allows writing a server program that takes a verity file 240and serves it to a client program, such that the client can do its own 241fs-verity compatible verification of the file. This only makes sense 242if the client doesn't trust the server and if the server needs to 243provide the storage for the client. 244 245This is a fairly specialized use case, and most fs-verity users won't 246need this ioctl. 247 248This ioctl takes in a pointer to the following structure:: 249 250 #define FS_VERITY_METADATA_TYPE_MERKLE_TREE 1 251 #define FS_VERITY_METADATA_TYPE_DESCRIPTOR 2 252 #define FS_VERITY_METADATA_TYPE_SIGNATURE 3 253 254 struct fsverity_read_metadata_arg { 255 __u64 metadata_type; 256 __u64 offset; 257 __u64 length; 258 __u64 buf_ptr; 259 __u64 __reserved; 260 }; 261 262``metadata_type`` specifies the type of metadata to read: 263 264- ``FS_VERITY_METADATA_TYPE_MERKLE_TREE`` reads the blocks of the 265 Merkle tree. The blocks are returned in order from the root level 266 to the leaf level. Within each level, the blocks are returned in 267 the same order that their hashes are themselves hashed. 268 See `Merkle tree`_ for more information. 269 270- ``FS_VERITY_METADATA_TYPE_DESCRIPTOR`` reads the fs-verity 271 descriptor. See `fs-verity descriptor`_. 272 273- ``FS_VERITY_METADATA_TYPE_SIGNATURE`` reads the signature which was 274 passed to FS_IOC_ENABLE_VERITY, if any. See `Built-in signature 275 verification`_. 276 277The semantics are similar to those of ``pread()``. ``offset`` 278specifies the offset in bytes into the metadata item to read from, and 279``length`` specifies the maximum number of bytes to read from the 280metadata item. ``buf_ptr`` is the pointer to the buffer to read into, 281cast to a 64-bit integer. ``__reserved`` must be 0. On success, the 282number of bytes read is returned. 0 is returned at the end of the 283metadata item. The returned length may be less than ``length``, for 284example if the ioctl is interrupted. 285 286The metadata returned by FS_IOC_READ_VERITY_METADATA isn't guaranteed 287to be authenticated against the file digest that would be returned by 288`FS_IOC_MEASURE_VERITY`_, as the metadata is expected to be used to 289implement fs-verity compatible verification anyway (though absent a 290malicious disk, the metadata will indeed match). E.g. to implement 291this ioctl, the filesystem is allowed to just read the Merkle tree 292blocks from disk without actually verifying the path to the root node. 293 294FS_IOC_READ_VERITY_METADATA can fail with the following errors: 295 296- ``EFAULT``: the caller provided inaccessible memory 297- ``EINTR``: the ioctl was interrupted before any data was read 298- ``EINVAL``: reserved fields were set, or ``offset + length`` 299 overflowed 300- ``ENODATA``: the file is not a verity file, or 301 FS_VERITY_METADATA_TYPE_SIGNATURE was requested but the file doesn't 302 have a built-in signature 303- ``ENOTTY``: this type of filesystem does not implement fs-verity, or 304 this ioctl is not yet implemented on it 305- ``EOPNOTSUPP``: the kernel was not configured with fs-verity 306 support, or the filesystem superblock has not had the 'verity' 307 feature enabled on it. (See `Filesystem support`_.) 308 309FS_IOC_GETFLAGS 310--------------- 311 312The existing ioctl FS_IOC_GETFLAGS (which isn't specific to fs-verity) 313can also be used to check whether a file has fs-verity enabled or not. 314To do so, check for FS_VERITY_FL (0x00100000) in the returned flags. 315 316The verity flag is not settable via FS_IOC_SETFLAGS. You must use 317FS_IOC_ENABLE_VERITY instead, since parameters must be provided. 318 319statx 320----- 321 322Since Linux v5.5, the statx() system call sets STATX_ATTR_VERITY if 323the file has fs-verity enabled. This can perform better than 324FS_IOC_GETFLAGS and FS_IOC_MEASURE_VERITY because it doesn't require 325opening the file, and opening verity files can be expensive. 326 327Accessing verity files 328====================== 329 330Applications can transparently access a verity file just like a 331non-verity one, with the following exceptions: 332 333- Verity files are readonly. They cannot be opened for writing or 334 truncate()d, even if the file mode bits allow it. Attempts to do 335 one of these things will fail with EPERM. However, changes to 336 metadata such as owner, mode, timestamps, and xattrs are still 337 allowed, since these are not measured by fs-verity. Verity files 338 can also still be renamed, deleted, and linked to. 339 340- Direct I/O is not supported on verity files. Attempts to use direct 341 I/O on such files will fall back to buffered I/O. 342 343- DAX (Direct Access) is not supported on verity files, because this 344 would circumvent the data verification. 345 346- Reads of data that doesn't match the verity Merkle tree will fail 347 with EIO (for read()) or SIGBUS (for mmap() reads). 348 349- If the sysctl "fs.verity.require_signatures" is set to 1 and the 350 file is not signed by a key in the fs-verity keyring, then opening 351 the file will fail. See `Built-in signature verification`_. 352 353Direct access to the Merkle tree is not supported. Therefore, if a 354verity file is copied, or is backed up and restored, then it will lose 355its "verity"-ness. fs-verity is primarily meant for files like 356executables that are managed by a package manager. 357 358File digest computation 359======================= 360 361This section describes how fs-verity hashes the file contents using a 362Merkle tree to produce the digest which cryptographically identifies 363the file contents. This algorithm is the same for all filesystems 364that support fs-verity. 365 366Userspace only needs to be aware of this algorithm if it needs to 367compute fs-verity file digests itself, e.g. in order to sign files. 368 369.. _fsverity_merkle_tree: 370 371Merkle tree 372----------- 373 374The file contents is divided into blocks, where the block size is 375configurable but is usually 4096 bytes. The end of the last block is 376zero-padded if needed. Each block is then hashed, producing the first 377level of hashes. Then, the hashes in this first level are grouped 378into 'blocksize'-byte blocks (zero-padding the ends as needed) and 379these blocks are hashed, producing the second level of hashes. This 380proceeds up the tree until only a single block remains. The hash of 381this block is the "Merkle tree root hash". 382 383If the file fits in one block and is nonempty, then the "Merkle tree 384root hash" is simply the hash of the single data block. If the file 385is empty, then the "Merkle tree root hash" is all zeroes. 386 387The "blocks" here are not necessarily the same as "filesystem blocks". 388 389If a salt was specified, then it's zero-padded to the closest multiple 390of the input size of the hash algorithm's compression function, e.g. 39164 bytes for SHA-256 or 128 bytes for SHA-512. The padded salt is 392prepended to every data or Merkle tree block that is hashed. 393 394The purpose of the block padding is to cause every hash to be taken 395over the same amount of data, which simplifies the implementation and 396keeps open more possibilities for hardware acceleration. The purpose 397of the salt padding is to make the salting "free" when the salted hash 398state is precomputed, then imported for each hash. 399 400Example: in the recommended configuration of SHA-256 and 4K blocks, 401128 hash values fit in each block. Thus, each level of the Merkle 402tree is approximately 128 times smaller than the previous, and for 403large files the Merkle tree's size converges to approximately 1/127 of 404the original file size. However, for small files, the padding is 405significant, making the space overhead proportionally more. 406 407.. _fsverity_descriptor: 408 409fs-verity descriptor 410-------------------- 411 412By itself, the Merkle tree root hash is ambiguous. For example, it 413can't a distinguish a large file from a small second file whose data 414is exactly the top-level hash block of the first file. Ambiguities 415also arise from the convention of padding to the next block boundary. 416 417To solve this problem, the fs-verity file digest is actually computed 418as a hash of the following structure, which contains the Merkle tree 419root hash as well as other fields such as the file size:: 420 421 struct fsverity_descriptor { 422 __u8 version; /* must be 1 */ 423 __u8 hash_algorithm; /* Merkle tree hash algorithm */ 424 __u8 log_blocksize; /* log2 of size of data and tree blocks */ 425 __u8 salt_size; /* size of salt in bytes; 0 if none */ 426 __le32 __reserved_0x04; /* must be 0 */ 427 __le64 data_size; /* size of file the Merkle tree is built over */ 428 __u8 root_hash[64]; /* Merkle tree root hash */ 429 __u8 salt[32]; /* salt prepended to each hashed block */ 430 __u8 __reserved[144]; /* must be 0's */ 431 }; 432 433Built-in signature verification 434=============================== 435 436With CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y, fs-verity supports putting 437a portion of an authentication policy (see `Use cases`_) in the 438kernel. Specifically, it adds support for: 439 4401. At fs-verity module initialization time, a keyring ".fs-verity" is 441 created. The root user can add trusted X.509 certificates to this 442 keyring using the add_key() system call, then (when done) 443 optionally use keyctl_restrict_keyring() to prevent additional 444 certificates from being added. 445 4462. `FS_IOC_ENABLE_VERITY`_ accepts a pointer to a PKCS#7 formatted 447 detached signature in DER format of the file's fs-verity digest. 448 On success, this signature is persisted alongside the Merkle tree. 449 Then, any time the file is opened, the kernel will verify the 450 file's actual digest against this signature, using the certificates 451 in the ".fs-verity" keyring. 452 4533. A new sysctl "fs.verity.require_signatures" is made available. 454 When set to 1, the kernel requires that all verity files have a 455 correctly signed digest as described in (2). 456 457fs-verity file digests must be signed in the following format, which 458is similar to the structure used by `FS_IOC_MEASURE_VERITY`_:: 459 460 struct fsverity_formatted_digest { 461 char magic[8]; /* must be "FSVerity" */ 462 __le16 digest_algorithm; 463 __le16 digest_size; 464 __u8 digest[]; 465 }; 466 467fs-verity's built-in signature verification support is meant as a 468relatively simple mechanism that can be used to provide some level of 469authenticity protection for verity files, as an alternative to doing 470the signature verification in userspace or using IMA-appraisal. 471However, with this mechanism, userspace programs still need to check 472that the verity bit is set, and there is no protection against verity 473files being swapped around. 474 475Filesystem support 476================== 477 478fs-verity is supported by several filesystems, described below. The 479CONFIG_FS_VERITY kconfig option must be enabled to use fs-verity on 480any of these filesystems. 481 482``include/linux/fsverity.h`` declares the interface between the 483``fs/verity/`` support layer and filesystems. Briefly, filesystems 484must provide an ``fsverity_operations`` structure that provides 485methods to read and write the verity metadata to a filesystem-specific 486location, including the Merkle tree blocks and 487``fsverity_descriptor``. Filesystems must also call functions in 488``fs/verity/`` at certain times, such as when a file is opened or when 489pages have been read into the pagecache. (See `Verifying data`_.) 490 491ext4 492---- 493 494ext4 supports fs-verity since Linux v5.4 and e2fsprogs v1.45.2. 495 496To create verity files on an ext4 filesystem, the filesystem must have 497been formatted with ``-O verity`` or had ``tune2fs -O verity`` run on 498it. "verity" is an RO_COMPAT filesystem feature, so once set, old 499kernels will only be able to mount the filesystem readonly, and old 500versions of e2fsck will be unable to check the filesystem. 501 502Originally, an ext4 filesystem with the "verity" feature could only be 503mounted when its block size was equal to the system page size 504(typically 4096 bytes). In Linux v6.3, this limitation was removed. 505 506ext4 sets the EXT4_VERITY_FL on-disk inode flag on verity files. It 507can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be cleared. 508 509ext4 also supports encryption, which can be used simultaneously with 510fs-verity. In this case, the plaintext data is verified rather than 511the ciphertext. This is necessary in order to make the fs-verity file 512digest meaningful, since every file is encrypted differently. 513 514ext4 stores the verity metadata (Merkle tree and fsverity_descriptor) 515past the end of the file, starting at the first 64K boundary beyond 516i_size. This approach works because (a) verity files are readonly, 517and (b) pages fully beyond i_size aren't visible to userspace but can 518be read/written internally by ext4 with only some relatively small 519changes to ext4. This approach avoids having to depend on the 520EA_INODE feature and on rearchitecturing ext4's xattr support to 521support paging multi-gigabyte xattrs into memory, and to support 522encrypting xattrs. Note that the verity metadata *must* be encrypted 523when the file is, since it contains hashes of the plaintext data. 524 525ext4 only allows verity on extent-based files. 526 527f2fs 528---- 529 530f2fs supports fs-verity since Linux v5.4 and f2fs-tools v1.11.0. 531 532To create verity files on an f2fs filesystem, the filesystem must have 533been formatted with ``-O verity``. 534 535f2fs sets the FADVISE_VERITY_BIT on-disk inode flag on verity files. 536It can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be 537cleared. 538 539Like ext4, f2fs stores the verity metadata (Merkle tree and 540fsverity_descriptor) past the end of the file, starting at the first 54164K boundary beyond i_size. See explanation for ext4 above. 542Moreover, f2fs supports at most 4096 bytes of xattr entries per inode 543which usually wouldn't be enough for even a single Merkle tree block. 544 545f2fs doesn't support enabling verity on files that currently have 546atomic or volatile writes pending. 547 548btrfs 549----- 550 551btrfs supports fs-verity since Linux v5.15. Verity-enabled inodes are 552marked with a RO_COMPAT inode flag, and the verity metadata is stored 553in separate btree items. 554 555Implementation details 556====================== 557 558Verifying data 559-------------- 560 561fs-verity ensures that all reads of a verity file's data are verified, 562regardless of which syscall is used to do the read (e.g. mmap(), 563read(), pread()) and regardless of whether it's the first read or a 564later read (unless the later read can return cached data that was 565already verified). Below, we describe how filesystems implement this. 566 567Pagecache 568~~~~~~~~~ 569 570For filesystems using Linux's pagecache, the ``->read_folio()`` and 571``->readahead()`` methods must be modified to verify folios before 572they are marked Uptodate. Merely hooking ``->read_iter()`` would be 573insufficient, since ``->read_iter()`` is not used for memory maps. 574 575Therefore, fs/verity/ provides the function fsverity_verify_blocks() 576which verifies data that has been read into the pagecache of a verity 577inode. The containing folio must still be locked and not Uptodate, so 578it's not yet readable by userspace. As needed to do the verification, 579fsverity_verify_blocks() will call back into the filesystem to read 580hash blocks via fsverity_operations::read_merkle_tree_page(). 581 582fsverity_verify_blocks() returns false if verification failed; in this 583case, the filesystem must not set the folio Uptodate. Following this, 584as per the usual Linux pagecache behavior, attempts by userspace to 585read() from the part of the file containing the folio will fail with 586EIO, and accesses to the folio within a memory map will raise SIGBUS. 587 588In principle, verifying a data block requires verifying the entire 589path in the Merkle tree from the data block to the root hash. 590However, for efficiency the filesystem may cache the hash blocks. 591Therefore, fsverity_verify_blocks() only ascends the tree reading hash 592blocks until an already-verified hash block is seen. It then verifies 593the path to that block. 594 595This optimization, which is also used by dm-verity, results in 596excellent sequential read performance. This is because usually (e.g. 597127 in 128 times for 4K blocks and SHA-256) the hash block from the 598bottom level of the tree will already be cached and checked from 599reading a previous data block. However, random reads perform worse. 600 601Block device based filesystems 602~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 603 604Block device based filesystems (e.g. ext4 and f2fs) in Linux also use 605the pagecache, so the above subsection applies too. However, they 606also usually read many data blocks from a file at once, grouped into a 607structure called a "bio". To make it easier for these types of 608filesystems to support fs-verity, fs/verity/ also provides a function 609fsverity_verify_bio() which verifies all data blocks in a bio. 610 611ext4 and f2fs also support encryption. If a verity file is also 612encrypted, the data must be decrypted before being verified. To 613support this, these filesystems allocate a "post-read context" for 614each bio and store it in ``->bi_private``:: 615 616 struct bio_post_read_ctx { 617 struct bio *bio; 618 struct work_struct work; 619 unsigned int cur_step; 620 unsigned int enabled_steps; 621 }; 622 623``enabled_steps`` is a bitmask that specifies whether decryption, 624verity, or both is enabled. After the bio completes, for each needed 625postprocessing step the filesystem enqueues the bio_post_read_ctx on a 626workqueue, and then the workqueue work does the decryption or 627verification. Finally, folios where no decryption or verity error 628occurred are marked Uptodate, and the folios are unlocked. 629 630On many filesystems, files can contain holes. Normally, 631``->readahead()`` simply zeroes hole blocks and considers the 632corresponding data to be up-to-date; no bios are issued. To prevent 633this case from bypassing fs-verity, filesystems use 634fsverity_verify_blocks() to verify hole blocks. 635 636Filesystems also disable direct I/O on verity files, since otherwise 637direct I/O would bypass fs-verity. 638 639Userspace utility 640================= 641 642This document focuses on the kernel, but a userspace utility for 643fs-verity can be found at: 644 645 https://git.kernel.org/pub/scm/fs/fsverity/fsverity-utils.git 646 647See the README.md file in the fsverity-utils source tree for details, 648including examples of setting up fs-verity protected files. 649 650Tests 651===== 652 653To test fs-verity, use xfstests. For example, using `kvm-xfstests 654<https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md>`_:: 655 656 kvm-xfstests -c ext4,f2fs,btrfs -g verity 657 658FAQ 659=== 660 661This section answers frequently asked questions about fs-verity that 662weren't already directly answered in other parts of this document. 663 664:Q: Why isn't fs-verity part of IMA? 665:A: fs-verity and IMA (Integrity Measurement Architecture) have 666 different focuses. fs-verity is a filesystem-level mechanism for 667 hashing individual files using a Merkle tree. In contrast, IMA 668 specifies a system-wide policy that specifies which files are 669 hashed and what to do with those hashes, such as log them, 670 authenticate them, or add them to a measurement list. 671 672 IMA supports the fs-verity hashing mechanism as an alternative 673 to full file hashes, for those who want the performance and 674 security benefits of the Merkle tree based hash. However, it 675 doesn't make sense to force all uses of fs-verity to be through 676 IMA. fs-verity already meets many users' needs even as a 677 standalone filesystem feature, and it's testable like other 678 filesystem features e.g. with xfstests. 679 680:Q: Isn't fs-verity useless because the attacker can just modify the 681 hashes in the Merkle tree, which is stored on-disk? 682:A: To verify the authenticity of an fs-verity file you must verify 683 the authenticity of the "fs-verity file digest", which 684 incorporates the root hash of the Merkle tree. See `Use cases`_. 685 686:Q: Isn't fs-verity useless because the attacker can just replace a 687 verity file with a non-verity one? 688:A: See `Use cases`_. In the initial use case, it's really trusted 689 userspace code that authenticates the files; fs-verity is just a 690 tool to do this job efficiently and securely. The trusted 691 userspace code will consider non-verity files to be inauthentic. 692 693:Q: Why does the Merkle tree need to be stored on-disk? Couldn't you 694 store just the root hash? 695:A: If the Merkle tree wasn't stored on-disk, then you'd have to 696 compute the entire tree when the file is first accessed, even if 697 just one byte is being read. This is a fundamental consequence of 698 how Merkle tree hashing works. To verify a leaf node, you need to 699 verify the whole path to the root hash, including the root node 700 (the thing which the root hash is a hash of). But if the root 701 node isn't stored on-disk, you have to compute it by hashing its 702 children, and so on until you've actually hashed the entire file. 703 704 That defeats most of the point of doing a Merkle tree-based hash, 705 since if you have to hash the whole file ahead of time anyway, 706 then you could simply do sha256(file) instead. That would be much 707 simpler, and a bit faster too. 708 709 It's true that an in-memory Merkle tree could still provide the 710 advantage of verification on every read rather than just on the 711 first read. However, it would be inefficient because every time a 712 hash page gets evicted (you can't pin the entire Merkle tree into 713 memory, since it may be very large), in order to restore it you 714 again need to hash everything below it in the tree. This again 715 defeats most of the point of doing a Merkle tree-based hash, since 716 a single block read could trigger re-hashing gigabytes of data. 717 718:Q: But couldn't you store just the leaf nodes and compute the rest? 719:A: See previous answer; this really just moves up one level, since 720 one could alternatively interpret the data blocks as being the 721 leaf nodes of the Merkle tree. It's true that the tree can be 722 computed much faster if the leaf level is stored rather than just 723 the data, but that's only because each level is less than 1% the 724 size of the level below (assuming the recommended settings of 725 SHA-256 and 4K blocks). For the exact same reason, by storing 726 "just the leaf nodes" you'd already be storing over 99% of the 727 tree, so you might as well simply store the whole tree. 728 729:Q: Can the Merkle tree be built ahead of time, e.g. distributed as 730 part of a package that is installed to many computers? 731:A: This isn't currently supported. It was part of the original 732 design, but was removed to simplify the kernel UAPI and because it 733 wasn't a critical use case. Files are usually installed once and 734 used many times, and cryptographic hashing is somewhat fast on 735 most modern processors. 736 737:Q: Why doesn't fs-verity support writes? 738:A: Write support would be very difficult and would require a 739 completely different design, so it's well outside the scope of 740 fs-verity. Write support would require: 741 742 - A way to maintain consistency between the data and hashes, 743 including all levels of hashes, since corruption after a crash 744 (especially of potentially the entire file!) is unacceptable. 745 The main options for solving this are data journalling, 746 copy-on-write, and log-structured volume. But it's very hard to 747 retrofit existing filesystems with new consistency mechanisms. 748 Data journalling is available on ext4, but is very slow. 749 750 - Rebuilding the Merkle tree after every write, which would be 751 extremely inefficient. Alternatively, a different authenticated 752 dictionary structure such as an "authenticated skiplist" could 753 be used. However, this would be far more complex. 754 755 Compare it to dm-verity vs. dm-integrity. dm-verity is very 756 simple: the kernel just verifies read-only data against a 757 read-only Merkle tree. In contrast, dm-integrity supports writes 758 but is slow, is much more complex, and doesn't actually support 759 full-device authentication since it authenticates each sector 760 independently, i.e. there is no "root hash". It doesn't really 761 make sense for the same device-mapper target to support these two 762 very different cases; the same applies to fs-verity. 763 764:Q: Since verity files are immutable, why isn't the immutable bit set? 765:A: The existing "immutable" bit (FS_IMMUTABLE_FL) already has a 766 specific set of semantics which not only make the file contents 767 read-only, but also prevent the file from being deleted, renamed, 768 linked to, or having its owner or mode changed. These extra 769 properties are unwanted for fs-verity, so reusing the immutable 770 bit isn't appropriate. 771 772:Q: Why does the API use ioctls instead of setxattr() and getxattr()? 773:A: Abusing the xattr interface for basically arbitrary syscalls is 774 heavily frowned upon by most of the Linux filesystem developers. 775 An xattr should really just be an xattr on-disk, not an API to 776 e.g. magically trigger construction of a Merkle tree. 777 778:Q: Does fs-verity support remote filesystems? 779:A: So far all filesystems that have implemented fs-verity support are 780 local filesystems, but in principle any filesystem that can store 781 per-file verity metadata can support fs-verity, regardless of 782 whether it's local or remote. Some filesystems may have fewer 783 options of where to store the verity metadata; one possibility is 784 to store it past the end of the file and "hide" it from userspace 785 by manipulating i_size. The data verification functions provided 786 by ``fs/verity/`` also assume that the filesystem uses the Linux 787 pagecache, but both local and remote filesystems normally do so. 788 789:Q: Why is anything filesystem-specific at all? Shouldn't fs-verity 790 be implemented entirely at the VFS level? 791:A: There are many reasons why this is not possible or would be very 792 difficult, including the following: 793 794 - To prevent bypassing verification, folios must not be marked 795 Uptodate until they've been verified. Currently, each 796 filesystem is responsible for marking folios Uptodate via 797 ``->readahead()``. Therefore, currently it's not possible for 798 the VFS to do the verification on its own. Changing this would 799 require significant changes to the VFS and all filesystems. 800 801 - It would require defining a filesystem-independent way to store 802 the verity metadata. Extended attributes don't work for this 803 because (a) the Merkle tree may be gigabytes, but many 804 filesystems assume that all xattrs fit into a single 4K 805 filesystem block, and (b) ext4 and f2fs encryption doesn't 806 encrypt xattrs, yet the Merkle tree *must* be encrypted when the 807 file contents are, because it stores hashes of the plaintext 808 file contents. 809 810 So the verity metadata would have to be stored in an actual 811 file. Using a separate file would be very ugly, since the 812 metadata is fundamentally part of the file to be protected, and 813 it could cause problems where users could delete the real file 814 but not the metadata file or vice versa. On the other hand, 815 having it be in the same file would break applications unless 816 filesystems' notion of i_size were divorced from the VFS's, 817 which would be complex and require changes to all filesystems. 818 819 - It's desirable that FS_IOC_ENABLE_VERITY uses the filesystem's 820 transaction mechanism so that either the file ends up with 821 verity enabled, or no changes were made. Allowing intermediate 822 states to occur after a crash may cause problems. 823