1==================== 2Scheduler Statistics 3==================== 4 5Version 17 of schedstats removed 'lb_imbalance' field as it has no 6significance anymore and instead added more relevant fields namely 7'lb_imbalance_load', 'lb_imbalance_util', 'lb_imbalance_task' and 8'lb_imbalance_misfit'. The domain field prints the name of the 9corresponding sched domain from this version onwards. 10 11Version 16 of schedstats changed the order of definitions within 12'enum cpu_idle_type', which changed the order of [CPU_MAX_IDLE_TYPES] 13columns in show_schedstat(). In particular the position of CPU_IDLE 14and __CPU_NOT_IDLE changed places. The size of the array is unchanged. 15 16Version 15 of schedstats dropped counters for some sched_yield: 17yld_exp_empty, yld_act_empty and yld_both_empty. Otherwise, it is 18identical to version 14. Details are available at 19 20 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/scheduler/sched-stats.txt?id=1e1dbb259c79b 21 22Version 14 of schedstats includes support for sched_domains, which hit the 23mainline kernel in 2.6.20 although it is identical to the stats from version 2412 which was in the kernel from 2.6.13-2.6.19 (version 13 never saw a kernel 25release). Some counters make more sense to be per-runqueue; other to be 26per-domain. Note that domains (and their associated information) will only 27be pertinent and available on machines utilizing CONFIG_SMP. 28 29In version 14 of schedstat, there is at least one level of domain 30statistics for each cpu listed, and there may well be more than one 31domain. Domains have no particular names in this implementation, but 32the highest numbered one typically arbitrates balancing across all the 33cpus on the machine, while domain0 is the most tightly focused domain, 34sometimes balancing only between pairs of cpus. At this time, there 35are no architectures which need more than three domain levels. The first 36field in the domain stats is a bit map indicating which cpus are affected 37by that domain. Details are available at 38 39 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/sched-stats.txt?id=b762f3ffb797c 40 41The schedstat documentation is maintained version 10 onwards and is not 42updated for version 11 and 12. The details for version 10 are available at 43 44 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/sched-stats.txt?id=1da177e4c3f4 45 46These fields are counters, and only increment. Programs which make use 47of these will need to start with a baseline observation and then calculate 48the change in the counters at each subsequent observation. A perl script 49which does this for many of the fields is available at 50 51 http://eaglet.pdxhosts.com/rick/linux/schedstat/ 52 53Note that any such script will necessarily be version-specific, as the main 54reason to change versions is changes in the output format. For those wishing 55to write their own scripts, the fields are described here. 56 57CPU statistics 58-------------- 59cpu<N> 1 2 3 4 5 6 7 8 9 60 61First field is a sched_yield() statistic: 62 63 1) # of times sched_yield() was called 64 65Next three are schedule() statistics: 66 67 2) This field is a legacy array expiration count field used in the O(1) 68 scheduler. We kept it for ABI compatibility, but it is always set to zero. 69 3) # of times schedule() was called 70 4) # of times schedule() left the processor idle 71 72Next two are try_to_wake_up() statistics: 73 74 5) # of times try_to_wake_up() was called 75 6) # of times try_to_wake_up() was called to wake up the local cpu 76 77Next three are statistics describing scheduling latency: 78 79 7) sum of all time spent running by tasks on this processor (in nanoseconds) 80 8) sum of all time spent waiting to run by tasks on this processor (in 81 nanoseconds) 82 9) # of timeslices run on this cpu 83 84 85Domain statistics 86----------------- 87One of these is produced per domain for each cpu described. (Note that if 88CONFIG_SMP is not defined, *no* domains are utilized and these lines 89will not appear in the output.) 90 91domain<N> <name> <cpumask> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 92 93The <name> field prints the name of the sched domain and is only supported 94with schedstat version >= 17. On previous versions, <cpumask> is the first 95field. 96 97The <cpumask> field is a bit mask indicating what cpus this domain operates 98over. 99 100The next 33 are a variety of sched_balance_rq() statistics in grouped into types 101of idleness (busy, idle and newly idle): 102 103 1) # of times in this domain sched_balance_rq() was called when the 104 cpu was busy 105 2) # of times in this domain sched_balance_rq() checked but found the 106 load did not require balancing when busy 107 3) # of times in this domain sched_balance_rq() tried to move one or 108 more tasks and failed, when the cpu was busy 109 4) Total imbalance in load in this domain when the cpu was busy 110 5) Total imbalance in utilization in this domain when the cpu was busy 111 6) Total imbalance in number of tasks in this domain when the cpu was busy 112 7) Total imbalance due to misfit tasks in this domain when the cpu was 113 busy 114 8) # of times in this domain detach_task() was called when busy 115 9) # of times in this domain detach_task() was called even though the 116 target task was cache-hot when busy 117 10) # of times in this domain sched_balance_rq() was called but did not 118 find a busier queue while the cpu was busy 119 11) # of times in this domain a busier queue was found while the cpu 120 was busy but no busier group was found 121 122 12) # of times in this domain sched_balance_rq() was called when the 123 cpu was idle 124 13) # of times in this domain sched_balance_rq() checked but found 125 the load did not require balancing when the cpu was idle 126 14) # of times in this domain sched_balance_rq() tried to move one or 127 more tasks and failed, when the cpu was idle 128 15) Total imbalance in load in this domain when the cpu was idle 129 16) Total imbalance in utilization in this domain when the cpu was idle 130 17) Total imbalance in number of tasks in this domain when the cpu was idle 131 18) Total imbalance due to misfit tasks in this domain when the cpu was 132 idle 133 19) # of times in this domain detach_task() was called when the cpu 134 was idle 135 20) # of times in this domain detach_task() was called even though 136 the target task was cache-hot when idle 137 21) # of times in this domain sched_balance_rq() was called but did 138 not find a busier queue while the cpu was idle 139 22) # of times in this domain a busier queue was found while the 140 cpu was idle but no busier group was found 141 142 23) # of times in this domain sched_balance_rq() was called when the 143 cpu was just becoming idle 144 24) # of times in this domain sched_balance_rq() checked but found the 145 load did not require balancing when the cpu was just becoming idle 146 25) # of times in this domain sched_balance_rq() tried to move one or more 147 tasks and failed, when the cpu was just becoming idle 148 26) Total imbalance in load in this domain when the cpu was just becoming 149 idle 150 27) Total imbalance in utilization in this domain when the cpu was just 151 becoming idle 152 28) Total imbalance in number of tasks in this domain when the cpu was just 153 becoming idle 154 29) Total imbalance due to misfit tasks in this domain when the cpu was 155 just becoming idle 156 30) # of times in this domain detach_task() was called when newly idle 157 31) # of times in this domain detach_task() was called even though the 158 target task was cache-hot when just becoming idle 159 32) # of times in this domain sched_balance_rq() was called but did not 160 find a busier queue while the cpu was just becoming idle 161 33) # of times in this domain a busier queue was found while the cpu 162 was just becoming idle but no busier group was found 163 164 Next three are active_load_balance() statistics: 165 166 34) # of times active_load_balance() was called 167 35) # of times active_load_balance() tried to move a task and failed 168 36) # of times active_load_balance() successfully moved a task 169 170 Next three are sched_balance_exec() statistics: 171 172 37) sbe_cnt is not used 173 38) sbe_balanced is not used 174 39) sbe_pushed is not used 175 176 Next three are sched_balance_fork() statistics: 177 178 40) sbf_cnt is not used 179 41) sbf_balanced is not used 180 42) sbf_pushed is not used 181 182 Next three are try_to_wake_up() statistics: 183 184 43) # of times in this domain try_to_wake_up() awoke a task that 185 last ran on a different cpu in this domain 186 44) # of times in this domain try_to_wake_up() moved a task to the 187 waking cpu because it was cache-cold on its own cpu anyway 188 45) # of times in this domain try_to_wake_up() started passive balancing 189 190/proc/<pid>/schedstat 191--------------------- 192schedstats also adds a new /proc/<pid>/schedstat file to include some of 193the same information on a per-process level. There are three fields in 194this file correlating for that process to: 195 196 1) time spent on the cpu (in nanoseconds) 197 2) time spent waiting on a runqueue (in nanoseconds) 198 3) # of timeslices run on this cpu 199 200A program could be easily written to make use of these extra fields to 201report on how well a particular process or set of processes is faring 202under the scheduler's policies. A simple version of such a program is 203available at 204 205 http://eaglet.pdxhosts.com/rick/linux/schedstat/v12/latency.c 206