1##################### 2Unit Tests Guidelines 3##################### 4 5Purpose 6======= 7 8This document defines the requirements for unit testing within RSE ROM 9development process. Additionally, this document includes a user guide for writing 10unit tests, which provides detailed instructions on creating, running, and 11managing unit tests effectively. 12 13Scope 14===== 15 16This policy is applicable to all developers and contributors engaged in the 17development, review, and maintenance of RSE ROM. 18 19Policy Statement 20================ 21 221. Unit Test Requirement 23------------------------ 24 25- All new RSE ROM code should be accompanied by relevant unit tests. 26 272. Updating Unit Tests 28---------------------- 29 30- When modifying existing functionality, developers must update the 31 corresponding unit tests to ensure they accurately test the new behavior. 32- Deprecated or obsolete unit tests must be removed or refactored to maintain 33 the relevance and efficiency of the unit test suite. 34 35Introduction 36============ 37 38Documentation is available at `Unity`_ and `CMock`_ to get started with the 39test and mocking frameworks. 40 41.. note:: Test runners are compiled for the host system. 42 43Required Tools 44-------------- 45 46The following tools are required for writing and running unit tests: 47 48- **LCOV**: A graphical front-end for GCC's coverage testing tool gcov. 49 LCOV is used to generate test coverage reports. 50- **genhtml**: A tool to convert LCOV output files into HTML reports for 51 easy viewing of code coverage. 52 53Terminology 54----------- 55 56- **Unit Test**: A test that verifies the functionality of a small, 57 isolated piece of code. 58- **Unit Under Test**: A unit under test (UUT) is the object that is 59 being tested. 60- **Mock**: A simulated object that mimics the behavior of real objects 61 in controlled ways. 62- **Assertion**: A statement that checks if a condition is true. 63 64Building all available tests 65---------------------------- 66 67.. code:: sh 68 69 $ cmake -GNinja -S unittests -B <build-dir> \ 70 -DTFM_ROOT_DIR=<tfm-root-dir> \ 71 -DTFM_UNITTESTS_PATHS=<path1;path2;...> 72 $ cmake --build <build-dir> 73 74e.g. :: 75 76 path1 = <absolute-path>/platform/ext/target/arm/rse/common/unittests 77 path2 = <absolute-path-to-new-unittests> 78 ... 79 80Executing tests 81--------------- 82 83Every unit covered by tests has at least one test suite with one or more 84tests. A test runner is generated for every test suite that executes all 85tests in the suite. 86 87Individual test suites can be executed by directly invoking the test 88runner executable. 89 90.. code:: sh 91 92 $ ./<build-dir>/test_dummy 93 test_dummy.c:19:test_dummy_ALWAYS_OK:PASS 94 95 ----------------------- 96 1 Tests 0 Failures 0 Ignored 97 OK 98 99Or using `CTest`_ for finer control (e.g. categorization using labels, match 100name by regex etc) 101 102.. code:: sh 103 104 $ ctest --test-dir <build-dir> -R dummy 105 Start 1: dummy 106 1/1 Test #1: dummy ...................... Passed 0.00 sec 107 108 100% tests passed, 0 tests failed out of 1 109 110 Total Test time (real) = 0.01 sec 111 112To run a single test from a specific unit 113 114.. code:: sh 115 116 $ ./<build-dir>/test_dummy -f ALWAYS_OK 117 # Run all tests with name containing 'ALWAYS_OK' 118 119Creating a new unit test 120------------------------ 121 122The build system checks each subdirectory in TFM_UNITTESTS_PATHS for a “valid” 123unit test directory. A “valid” unit test directory must contain: 124 1251. ``utcfg.cmake`` 1262. One or more source/header files 127 128Unit test configuration 129----------------------- 130 131The unit test configuration *must set* the following 132 1331. ``UNIT_UNDER_TEST``: File implementing the unit under test. 1342. ``UNIT_TEST_SUITE``: File containing tests for the UUT 1353. ``UNIT_TEST_DEPS``: List of files that the UUT depend on. 1364. ``UNIT_TEST_INCLUDE_DIRS``: List of directories to be included in the build. 1375. ``MOCK_HEADERS``: List of headers that contain interfaces to be 138 mocked. 139 140The unit test configuration *may also set* the following 1411. ``UNIT_TEST_LABELS``: List of labels for the test suite. 1422. ``UNIT_TEST_COMPILE_DEFS``: Macros to build the unit test 143 144Debugging unit tests 145-------------------- 146 147For each valid unit test directory, the corresponding unit test runners 148are available at the root of the ``<build-dir>``. 149 150.. note:: All unit test runners are built with ``-g3``. 151 152To debug a segfaulting test case in a unit, e.g. ``critical_system`` 153 154.. code:: sh 155 156 $ gdb <build-dir>/test_critical_system 157 (gdb) r 158 ... <segfault> ... 159 (gdb) bt 160 ... <backtrace> ... 161 162 163.. note:: A common reason for segfaults in test runners is an invalid memory 164 access on the host system. GDB populates 165 ``$_siginfo._sifields._sigfault.si_addr`` with the offending address. 166 167Components 168========== 169 170:: 171 172 platform/ext/target/arm/rse/common/unittests 173 ├── CMakeLists.txt 174 ├── framework 175 │ ├── cmock 176 │ │ ├── <patch-files> 177 │ │ ├── cfg.yml 178 │ │ └── CMakeLists.txt 179 │ ├── cmsis 180 │ │ └── CMakeLists.txt 181 │ └── unity 182 │ ├── <patch-files> 183 │ ├── cfg.yml 184 │ └── CMakeLists.txt 185 ├── <dir1> 186 │ ├── <unit1-tests> 187 │ │ ├── test_unit1.c 188 │ │ └── utcfg.cmake 189 │ └── <unit2-tests> 190 │ ├── test_unit2.c 191 │ └── utcfg.cmake 192 └── <dir2> 193 └── <unit3-tests> 194 ├── test_unit3.c 195 └── utcfg.cmake 196 197- Framework & dependencies are downloaded into the build tree directly. 198- ``cmock/cfg.yml``: CMock configuration for mocking interfaces. 199- ``unity/cfg.yml``: Unity configuration for the framework features. 200- ``include``: Mocked headers common to all unit tests. 201 202Writing Unit Tests 203------------------ 204 205As an example, a simple test is implemented for ``tfm_plat_provisioning_is_required`` 206in ``platform/ext/target/arm/rse/common/provisioning/bl1_provisioning.c`` 207 208.. code:: c 209 210 static void gpio_set(enum rse_gpio_val_t val) 211 { 212 volatile uint32_t *gretreg = 213 &((struct rse_sysctrl_t *)RSE_SYSCTRL_BASE_S)->gretreg; 214 215 *gretreg &= ~0b1111; 216 *gretreg |= val & 0b1111; 217 } 218 219 enum tfm_plat_err_t tfm_plat_provisioning_is_required(bool *provisioning_required) 220 { 221 enum lcm_error_t err; 222 enum lcm_lcs_t lcs; 223 224 if (provisioning_required == NULL) { 225 return TFM_PLAT_ERR_INVALID_INPUT; 226 } 227 228 err = lcm_get_lcs(&LCM_DEV_S, &lcs); 229 if (err != LCM_ERROR_NONE) { 230 return err; 231 } 232 233 *provisioning_required = (lcs == LCM_LCS_CM || lcs == LCM_LCS_DM); 234 if (!*provisioning_required) { 235 if (lcs == LCM_LCS_RMA) { 236 gpio_set(RSE_GPIO_STATE_RMA_IDLE); 237 } else if (lcs == LCM_LCS_SE) { 238 gpio_set(RSE_GPIO_STATE_SE_ROM_BOOT); 239 } 240 } 241 242 return TFM_PLAT_ERR_SUCCESS; 243 } 244 245To have a good unit test, we need to cover all possible paths that can lead to 246different outputs or function calls. Some of the possible execution paths are 247 2481. The return value of ``lcm_get_lcs`` is not ``LCM_ERROR_NONE`` 2492. The return value of ``lcm_get_lcs`` is ``LCM_ERROR_NONE`` with 250 provisioning required. 2513. The return value of ``lcm_get_lcs`` is ``LCM_ERROR_NONE`` with 252 provisioning not required amd ``lcs = LCM_LCS_RMA``. 2534. The return value of ``lcm_get_lcs`` is ``LCM_ERROR_NONE with`` 254 provisioning not required amd ``lcs = LCM_LCS_SE``. 255 256Each path will return a value and may set a ``gretreg`` value. 257 258Before writing the test, we need to mock all the external interfaces used by the 259UUT (``tfm_plat_provisioning_is_required``). External interfaces are all APIs 260and variables out of the C file. In this case ``lcm_get_lcs`` is external 261function from ``lcm_drv.c``. `CMock`_ generates a mocked version of the function 262which the unit test can use to inject values into the UUT. 263 264.. code:: c 265 266 #include "unity.h" 267 268 #include "platform_regs.h" 269 #include "mock_lcm_drv.h" 270 271 static struct rse_sysctrl_t mock_sysctrl; 272 volatile struct rse_sysctrl_t * RSE_SYSCTRL_BASE_S = &mock_sysctrl; 273 274 void test_bl1_provisionig_IsRequired_FalseRMA(void) 275 { 276 int ret; 277 enum lcm_lcs_t expected_lcs = LCM_LCS_RMA; 278 279 /* Prepare */ 280 281 /* Mocked function return 0 (LCM_ERROR_NONE) */ 282 lcm_get_lcs_ExpectAnyArgsAndReturn(0); 283 284 /* Mocked function return through the argument 285 * pointer the value LCM_LCS_CM 286 */ 287 lcm_get_lcs_ReturnThruPtr_lcs(&expected_lcs); 288 289 /* Act */ 290 ret = tfm_plat_provisioning_is_required(); 291 292 /* Assert */ 293 TEST_ASSERT_EQUAL(0, ret); 294 TEST_ASSERT_EQUAL((RSE_SYSCTRL_BASE_S->gretreg & 0x000F), 295 RSE_GPIO_STATE_RMA_IDLE); 296 } 297 298The test now covers one of the possible execution paths, and we can add test 299cases for the other paths in the same way. 300 301Setting the expectations from the external interfaces is important for writing 302a good test case. Please refer to the `CMock`_ documentation. 303 304Registers and memory accesses 305----------------------------- 306 307In the example above, the UUT writes values to the ``gretreg`` register directly. 308The memory address defined in the software for the target may be invalid on the host. 309 310For the test, the UUT would need to include a dummy ``platform_base_address.h`` with 311 312.. code:: c 313 314 extern volatile struct rse_sysctrl_t * RSE_SYSCTRL_BASE_S; 315 316And in the unit test file 317 318.. code:: c 319 320 static struct rse_sysctrl_t mock_sysctrl; 321 volatile struct rse_sysctrl_t * RSE_SYSCTRL_BASE_S = &mock_sysctrl; 322 323This will use the address of ``mock_sysctrl`` from the unit test, where test cases 324can assert expectations or set initial values of the ``mock_sysctrl`` registers. 325 326Unittest Style Guide 327==================== 328 329For additions and changes in unit tests, it is preferable to follow the 330guidelines outlined below: 331 332#. The format for the test names is ``test_`` followed by the function being 333 tested and a pass or fail expectation, for example: 334 ``test_function_being_tested_init_success``. 335#. Each test case should cover one scenario. For example, if testing one case 336 for a function, have a test function for that case only. 337#. Name the test functions according to the test being performed. 338 339-------------- 340 341*SPDX-License-Identifier: BSD-3-Clause* 342 343*SPDX-FileCopyrightText: Copyright The TrustedFirmware-M Contributors* 344 345.. _Unity: https://github.com/ThrowTheSwitch/Unity 346.. _CMock: https://github.com/ThrowTheSwitch/CMock 347.. _CTest: https://cmake.org/cmake/help/latest/manual/ctest.1.html 348