summaryrefslogtreecommitdiffstats
path: root/Documentation/dev-tools/kselftest.rst
blob: 79d15421e9a5453241e252a984c717b1287b976d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
======================
Linux Kernel Selftests
======================

The kernel contains a set of "self tests" under the tools/testing/selftests/
directory. These are intended to be small tests to exercise individual code
paths in the kernel. Tests are intended to be run after building, installing
and booting a kernel.

You can find additional information on Kselftest framework, how to
write new tests using the framework on Kselftest wiki:

https://kselftest.wiki.kernel.org/

On some systems, hot-plug tests could hang forever waiting for cpu and
memory to be ready to be offlined. A special hot-plug target is created
to run the full range of hot-plug tests. In default mode, hot-plug tests run
in safe mode with a limited scope. In limited mode, cpu-hotplug test is
run on a single cpu as opposed to all hotplug capable cpus, and memory
hotplug test is run on 2% of hotplug capable memory instead of 10%.

Running the selftests (hotplug tests are run in limited mode)
=============================================================

To build the tests::

  $ make -C tools/testing/selftests

To run the tests::

  $ make -C tools/testing/selftests run_tests

To build and run the tests with a single command, use::

  $ make kselftest

Note that some tests will require root privileges.

Kselftest supports saving output files in a separate directory and then
running tests. To locate output files in a separate directory two syntaxes
are supported. In both cases the working directory must be the root of the
kernel src. This is applicable to "Running a subset of selftests" section
below.

To build, save output files in a separate directory with O= ::

  $ make O=/tmp/kselftest kselftest

To build, save output files in a separate directory with KBUILD_OUTPUT ::

  $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest

The O= assignment takes precedence over the KBUILD_OUTPUT environment
variable.

The above commands by default run the tests and print full pass/fail report.
Kselftest supports "summary" option to make it easier to understand the test
results. Please find the detailed individual test results for each test in
/tmp/testname file(s) when summary option is specified. This is applicable
to "Running a subset of selftests" section below.

To run kselftest with summary option enabled ::

  $ make summary=1 kselftest

Running a subset of selftests
=============================

You can use the "TARGETS" variable on the make command line to specify
single test to run, or a list of tests to run.

To run only tests targeted for a single subsystem::

  $ make -C tools/testing/selftests TARGETS=ptrace run_tests

You can specify multiple tests to build and run::

  $  make TARGETS="size timers" kselftest

To build, save output files in a separate directory with O= ::

  $ make O=/tmp/kselftest TARGETS="size timers" kselftest

To build, save output files in a separate directory with KBUILD_OUTPUT ::

  $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest

See the top-level tools/testing/selftests/Makefile for the list of all
possible targets.

Running the full range hotplug selftests
========================================

To build the hotplug tests::

  $ make -C tools/testing/selftests hotplug

To run the hotplug tests::

  $ make -C tools/testing/selftests run_hotplug

Note that some tests will require root privileges.


Install selftests
=================

You can use the kselftest_install.sh tool to install selftests in the
default location, which is tools/testing/selftests/kselftest, or in a
user specified location.

To install selftests in default location::

   $ cd tools/testing/selftests
   $ ./kselftest_install.sh

To install selftests in a user specified location::

   $ cd tools/testing/selftests
   $ ./kselftest_install.sh install_dir

Running installed selftests
===========================

Kselftest install as well as the Kselftest tarball provide a script
named "run_kselftest.sh" to run the tests.

You can simply do the following to run the installed Kselftests. Please
note some tests will require root privileges::

   $ cd kselftest
   $ ./run_kselftest.sh

Contributing new tests
======================

In general, the rules for selftests are

 * Do as much as you can if you're not root;

 * Don't take too long;

 * Don't break the build on any architecture, and

 * Don't cause the top-level "make run_tests" to fail if your feature is
   unconfigured.

Contributing new tests (details)
================================

 * Use TEST_GEN_XXX if such binaries or files are generated during
   compiling.

   TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by
   default.

   TEST_CUSTOM_PROGS should be used by tests that require custom build
   rules and prevent common build rule use.

   TEST_PROGS are for test shell scripts. Please ensure shell script has
   its exec bit set. Otherwise, lib.mk run_tests will generate a warning.

   TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests.

   TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the
   executable which is not tested by default.
   TEST_FILES, TEST_GEN_FILES mean it is the file which is used by
   test.

 * First use the headers inside the kernel source and/or git repo, and then the
   system headers.  Headers for the kernel release as opposed to headers
   installed by the distro on the system should be the primary focus to be able
   to find regressions.

 * If a test needs specific kernel config options enabled, add a config file in
   the test directory to enable them.

   e.g: tools/testing/selftests/android/config

Test Harness
============

The kselftest_harness.h file contains useful helpers to build tests.  The tests
from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as example.

Example
-------

.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
    :doc: example


Helpers
-------

.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
    :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP
                FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN

Operators
---------

.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
    :doc: operators

.. kernel-doc:: tools/testing/selftests/kselftest_harness.h
    :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE
                ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE
                ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT
                EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE
                EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE