1@page page_component_utest utest Framework 2 3# utest Introduction 4 5utest (unit test) is a unit testing framework developed by RT-Thread. The original intention of designing utest is to make it easier for RT-Thread developers to write test programs using a unified framework interface for unit testing, coverage testing, and integration testing. 6 7## Test Case Definition 8 9A test case (tc) is a single test performed to achieve a specific test objective. It is a specification that includes test input, execution conditions, test procedures, and expected results. It is a infinite loop with clear end conditions and test results. 10 11The utest (unit test) framework defines user-written test programs as **test cases**, and a test case contains only one *testcase* function (similar to the main function), which can contain multiple *test unit* functions. 12 13The test code for a function, specifically through the API provided by the utest framework, is a test case. 14 15## Test Unit Definition 16 17The test unit is a test point subdivided by the function to be tested. Each test point can be the smallest measurable unit of the function to be tested. Of course, different classification methods will subdivide different test units. 18 19## utest Application Block Diagram 20 21 22 23As shown in the figure above, the test case is designed based on the service interface provided by the test framework utest, which supports compiling multiple test cases together for testing. In addition, as you can see from the figure, a test case corresponds to a unique *testcase* function, and multiple test units are included in *testcase*. 24 25# utest API 26 27To enable uniform test case code, the test framework utest provides a common API interface for test case writing. 28 29## Macros of assertion 30 31> NOTE: 32> Here assert only records the number of passes and failures, it does not generate assertions or terminates program execution. Its function is not equivalent to RT_ASSERT. 33 34 35| assert Macro | Description | 36| :------ | :------ | 37| uassert_true(value) | If the value is true then the test passes, otherwise the test fails. | 38| uassert_false(value) | If the value is false then the test passes, otherwise the test fails. | 39| uassert_null(value) | If the Value is null then the test passes, otherwise the test fails | 40| uassert_not_null(value)| If the value is a non-null value, the test passes, otherwise the test fails. | 41| uassert_int_equal(a, b)| If the values of a and b are equal, the test passes, otherwise the test fails. | 42| uassert_int_not_equal(a, b)| If the values of a and b are not equal, the test passes, otherwise the test fails. | 43| uassert_str_equal(a, b) | If the string a and the string b are the same, the test passes, otherwise the test fails. | 44| uassert_str_not_equal(a, b)| If the string a and the string b are not the same, the test passes, otherwise the test fails. | 45| uassert_in_range(value, min, max) | If the value is in the range of min and max, the test passes, otherwise the test fails. | 46| uassert_not_in_range(value, min, max)| If the value is not in the range of min and max, the test passes, otherwise the test fails. | 47 48## Macros for Running Test Units 49 50```c 51UTEST_UNIT_RUN(test_unit_func) 52``` 53 54In the test case, the specified test unit function `test_unit_func` is executed using the `UTEST_UNIT_RUN` macro. The test unit must be executed using the `UTEST_UNIT_RUN` macro. 55 56## Macros for Exporting Test Cases 57 58```c 59UTEST_TC_EXPORT(testcase, name, init, cleanup, timeout) 60``` 61 62| Parameters | Description | 63| :----- | :------ | 64| testcase | Test case main-bearing function (**specifies** using a function called *static void testcase(void)* | 65| name | Test case name (uniqueness). Specifies the naming format for connecting relative names of test cases relative to `testcases directory` with `.` | 66| init | the initialization function before Test case startup | 67| cleanup | Cleanup function after the end of the test case | 68| timeout | Test case expected test time (in seconds) | 69 70**Test case naming requirements:** 71 72Test cases need to be named in the prescribed format. Specifies the naming format for the connection of the current test case relative to the `testcases directory ` linked with `.` . The name contains the file name of the current test case file (the file name except the suffix name). 73 74**Test case naming example:** 75 76Assuming that there is a `testcases\components\filesystem\dfs\dfs_api_tc.c` test case file in the test case `testcases` directory, the test case name in the `dfs_api_tc.c` is named `components.filesystem.dfs.dfs_api_tc`. 77 78## Test Case LOG Output Interface 79 80The utest framework relies on the *ulog log module* for log output and the log output level in the utest framework. So just add `#include "utest.h"` to the test case to use all level interfaces (LOG_D/LOG_I/LOG_E) of the ulog log module. 81 82In addition, the utest framework adds an additional log control interface as follows: 83 84```c 85#define UTEST_LOG_ALL (1u) 86#define UTEST_LOG_ASSERT (2u) 87 88void utest_log_lv_set(rt_uint8_t lv); 89``` 90 91Users can use the `utest_log_lv_set` interface to control the log output level in test cases. The `UTEST_LOG_ALL` configuration outputs all logs, and the `UTEST_LOG_ASSERT` configuration only outputs logs after the failure of uassert. 92 93# Configuration Enable 94 95Using the utest framework requires the following configuration in the ENV tool using menuconfig: 96 97```c 98RT-Thread Kernel ---> 99 Kernel Device Object ---> 100 (256) the buffer size for console log printf /* The minimum buffer required by the utest log */ 101RT-Thread Components ---> 102 Utilities ---> 103 -*- Enable utest (RT-Thread test framework) /* Enable utest framework */ 104 (4096) The utest thread stack size /* Set the utest thread stack (required for -thread mode) */ 105 (20) The utest thread priority /* Set utest thread priority (required for -thread mode) */ 106``` 107 108# Application Paradigm 109 110The utest framework and related APIs were introduced earlier. The basic test case code structure is described here. 111 112The code blocks necessary for the test case file are as follows: 113 114```c 115/* 116 * Copyright (c) 2006-2019, RT-Thread Development Team 117 * 118 * SPDX-License-Identifier: Apache-2.0 119 * 120 * Change Logs: 121 * Date Author Notes 122 * 2019-01-16 MurphyZhao the first version 123 */ 124 125#include <rtthread.h> 126#include "utest.h" 127 128static void test_xxx(void) 129{ 130 uassert_true(1); 131} 132 133static rt_err_t utest_tc_init(void) 134{ 135 return RT_EOK; 136} 137 138static rt_err_t utest_tc_cleanup(void) 139{ 140 return RT_EOK; 141} 142 143static void testcase(void) 144{ 145 UTEST_UNIT_RUN(test_xxx); 146} 147UTEST_TC_EXPORT(testcase, "components.utilities.utest.sample.sample_tc", utest_tc_init, utest_tc_cleanup, 10); 148``` 149 150A basic test case must contain the following: 151 152- File comment header (Copyright) 153 154 The test case file must contain a file comment header containing `Copyright`, time, author, and description information. 155 156- utest_tc_init(void) 157 158 The initialization function before the test run is generally used to initialize the environment required for the test. 159 160- utest_tc_cleanup(void) 161 162 The cleanup function after the test is used to clean up the resources (such as memory, threads, semaphores, etc.) applied during the test. 163 164- testcase(void) 165 166 The mainly function of testcase, a test case implementation can only contain one testcase function (similar to the main function). Usually this function is only used to run the test unit execution function `UTEST_UNIT_RUN`. 167 168 A testcase can contain multiple test units, each of which is executed by `UTEST_UNIT_RUN`. 169 170- UTEST_UNIT_RUN 171 172 Test unit execution function. 173 174- test_xxx(void) 175 176 Test implementation of each functional unit. The user determines the function name and function implementation based on the requirements. 177 178- uassert_true 179 180 The assertion macro used to determine the test result (this assertion macro does not terminate the program run). Test cases must use the `uassert_xxx` macro to determine the test results, otherwise the test framework does not know if the test passed. 181 182 After all the `uassert_xxx` macros have been passed, the entire test case is passed. 183 184- UTEST_TC_EXPORT 185 186 Export the test case testcase function to the test framework. 187 188# Requirements for running test cases 189 190The test framework utest exports all test cases to the `UtestTcTab` code segment. The `UtestTcTab` section is not required to be defined in the link script in the IAR and MDK compilers, but it needs to be explicitly set in the link script when GCC is compiled. 191 192Therefore, in order for test cases to be compiled and run under GCC, the `UtestTcTab` code segment must be defined in the *link script* of GCC. 193 194In the `.text` of the GCC link script, add the definition of the `UtestTcTab` section in the following format: 195 196```c 197/* section information for utest */ 198. = ALIGN(4); 199__rt_utest_tc_tab_start = .; 200KEEP(*(UtestTcTab)) 201__rt_utest_tc_tab_end = .; 202``` 203 204# Running Test Cases 205 206The test framework provides the following commands to make it easy for users to run test cases on the RT-Thread MSH command line. The commands are as follows: 207 208***utest_list* command** 209 210Lists the test cases supported by the current system, including the name of the test case and the time required for the test. This command has no parameters. 211 212***utest_run* command** 213 214Test case execution command, the format of the command is as follows: 215 216```c 217utest_run [-thread or -help] [testcase name] [loop num] 218``` 219 220| utest_run Command Parameters | Description | 221| :---- | :----- | 222| -thread | Run the test framework in threaded mode | 223| -help | Print help information | 224| testcase name | Specify the test case name. Using the wildcard `*` is supported, specifying front byte of the test case name is supported. | 225| loop num | the number of iterations of test cases | 226 227**Example of test command usage:** 228 229```c 230msh />utest_list 231[14875] I/utest: Commands list : 232[14879] I/utest: [testcase name]:components.filesystem.dfs.dfs_api_tc; [run timeout]:30 233[14889] I/utest: [testcase name]:components.filesystem.posix.posix_api_tc; [run timeout]:30 234[14899] I/utest: [testcase name]:packages.iot.netutils.iperf.iperf_tc; [run timeout]:30 235msh /> 236msh />utest_run components.filesystem.dfs.dfs_api_tc 237[83706] I/utest: [==========] [ utest ] started 238[83712] I/utest: [----------] [ testcase ] (components.filesystem.dfs.dfs_api_tc) started 239[83721] I/testcase: in testcase func... 240[84615] D/utest: [ OK ] [ unit ] (test_mkfs:26) is passed 241[84624] D/testcase: dfs mount rst: 0 242[84628] D/utest: [ OK ] [ unit ] (test_dfs_mount:35) is passed 243[84639] D/utest: [ OK ] [ unit ] (test_dfs_open:40) is passed 244[84762] D/utest: [ OK ] [ unit ] (test_dfs_write:74) is passed 245[84770] D/utest: [ OK ] [ unit ] (test_dfs_read:113) is passed 246[85116] D/utest: [ OK ] [ unit ] (test_dfs_close:118) is passed 247[85123] I/utest: [ PASSED ] [ result ] testcase (components.filesystem.dfs.dfs_api_tc) 248[85133] I/utest: [----------] [ testcase ] (components.filesystem.dfs.dfs_api_tc) finished 249[85143] I/utest: [==========] [ utest ] finished 250msh /> 251``` 252 253## Test result analysis 254 255 256 257As shown in the figure above, the log of the test case run is divided into four columns from left to right, which are `(1) log header information`, `(2) result bar`, `(3) property bar`, and `(4) detail information display bar`. The test case test result (PASSED or FAILED) is identified in the log using the `result` attribute. 258 259# Test Case Run Process 260 261 262 263From the above flow chart you can get the following: 264 265* The utest framework is a sequential execution of all **test units** in the *testcase* function 266* Assert of the previous UTEST_UNIT_RUN macro has occurred, and all subsequent UTEST_UNIT_RUN will skip execution. 267 268# NOTE 269 270- Determine whether the link script has the `UtestTcTab` section added before compiling with GCC. 271- Make sure `RT-Thread Kernel -> Kernel Device Object -> (256) the buffer size for console log printf` is at least 256 bytes before compiling. 272- The resources (threads, semaphores, timers, memory, etc.) created in the test case need to be released before the test ends. 273- A test case implementation can only export a test body function (testcase function) using `UTEST_TC_EXPORT` 274- Write a `README.md` document for the your test case to guide the user through configuring the test environment. 275
