Those were added before 0.61 which is our minimum version.
Dots were dropped from the end of some messages. Most messages did not have the
trailing dot.
0.63 added support for multi-line f-strings. We can't use those yet.
We went back and forth between 'prog.sh', files('prog.sh'), and
find_program('prog.sh'). We want to use files() or find_program() so that we
get a good error message if the file is missing. Behaviour of meson changed
over time, and in the past not all forms could be used in all places. For
example 0f4c4f3824 added find_program() in many
places to avoid repeated messages. But it seems that all recent meson versions
work fine with files().
find_program prints silly messages:
Program tools/make-man-index.py found: YES
(/home/zbyszek/src/systemd/tools/make-man-index.py)
Program tools/meson-render-jinja2.py found: YES
(/home/zbyszek/src/systemd/tools/meson-render-jinja2.py)
...
We know that those files will be found, they are part of the git checkout.
With files() this is gone and the meson output is easier to read.
[1] says:
> Since 0.60.0 the name argument is optional and defaults to the basename of
> the first output
We specify >= 0.62 as the supported version, so drop the duplicate name in all cases
where it is the same as outputs[0], i.e. almost all cases.
[1] https://mesonbuild.com/Reference-manual_functions.html#custom_target
Currently, when fuzzers are enabled, we run meson from within meson
to build the fuzzer executables with sanitizers. The idea is that
we can build the fuzzers with different kinds of sanitizers
independently from the main build.
The issue with this setup is that we don't actually make use of it.
We only build the fuzzers with one set of sanitizers (address,undefined)
so we're adding a bunch of extra complexity without any benefit as we
can just setup the top level meson build with these sanitizers and get
the same result.
The other issue with this setup is that we don't pass on all the options
passed to the top level meson build to the nested meson build. The only things
we pass on are extra compiler arguments and the value of the auto_features
option, but none of the individual feature options if overridden are passed on,
which can lead to very hard to debug issues as an option enabled in the top
level build is not enabled in the nested build.
Since we're not getting anything useful out of this setup, let's simplify
and get rid of the nested meson build. Instead, sanitizers should be enabled
for the top level meson.build. This currently didn't work as we were overriding
the sanitizers passed to the meson build with the fuzzer sanitizer, so we
fix that as well by making sure we combine the fuzzer sanitizer with the ones
passed in by the user.
We also drop support for looking up libFuzzer as a separate library as
it has been shipped builtin in clang since clang 6.0, so we can assume
that -fsanitize=fuzzer is available.
To make sure we still run the fuzzing tests, we enable the fuzz-tests option
by default now to make sure they still always run (without instrumentation unless
one of llvm-fuzz or oss-fuzz is enabled).
meson's `cpp_args` option is defined only if it detects a C++ compiler,
otherwise we get an error:
../test/fuzz/meson.build:56:28: ERROR: Tried to access unknown option 'cpp_args'.
Prompted by #29972, because right now it's practically impossible to pass
-fno-sanitize=function to the fuzzer targets without some extensive
sed'ing.
This splits both c_args and cpp_args to separate arguments for
tools/meson-build.sh, because the other way would be to use `eval`, so
the space-separated but quoted strings passed to these options are not
split where they shouldn't, and I'd rather avoid using `eval` if
possible.
Also, this switches the positional arguments we pass to `meson setup`,
as they were in incorrect order (docs say it should be buildir followed
by sourcedir); meson is apparently clever enough to figure this out and
switch the arguments around if necessary, so it didn't complain.
Now that we use meson feature options for our dependencies, we can just
rely on '--auto-features=disabled' to do the same. One benefit of this
is that specific features can still be force-enabled by overriding it
with the appropriate '-Dfeature=enabled' flag.
The two remaining uses for skip-deps can simply rely on their default
logic that sets the value to 'no' when the dependency is disabled.
When mkosi is run from git-worktree(1), the .git is not a repository
directory but a textfile pointing to the real git dir
(e.g. /home/user/systemd/.git/worktrees/systemd-worktree). This git dir
is not bind mounted into build environment and it fails with:
> fatal: not a git repository: /home/user/systemd/.git/worktrees/systemd-worktree
> test/meson.build:190:16: ERROR: Command `/usr/bin/env -u GIT_WORK_TREE /usr/bin/git --git-dir=/root/src/.git ls-files ':/test/dmidecode-dumps/*.bin'` failed with status 128.
There is already a fallback to use shell globbing instead of ls-files,
use it with git worktrees as well.
The lists of directives for fuzzer tests are maintained manually in the
repo. There is a tools/check-directives.sh script that runs during test
phase and reports stale directive lists.
Let's rework the script into a generator so that these directive files
are created on-the-flight and needn't be updated whenever a unit file
directives change. The scripts is rewritten in Python to get rid of gawk
dependency and each generated file is a separate meson target so that
incremental builds refresh what is just necessary (and parallelize
(negligible)).
Note: test/fuzz/fuzz-unit-file/directives-all.slice is kept since there
is not automated way to generate it (it is not covered by the check
script neither).
to let them use reproducible identifiers, which should make it possible
to really use files copied from OSS-Fuzz to reproduce issues on
GHActions and locally. Prompted by https://github.com/systemd/systemd/pull/22365
No need to invoke ls when we are just interested in file names.
Also, the cd to source root makes the output identical to
"git ls-files" (relative instead of absolute paths).
Meson 0.58 has gotten quite bad with emitting a message every time
a quoted command is used:
Program /home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh found: YES (/home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program xsltproc found: YES (/usr/bin/xsltproc)
Configuring custom-entities.ent using configuration
Message: Skipping bootctl.1 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping journal-remote.conf.5 because HAVE_MICROHTTPD is false
Message: Skipping journal-upload.conf.5 because HAVE_MICROHTTPD is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping loader.conf.5 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
...
Let's suffer one message only for each command. Hopefully we can silence
even this when https://github.com/mesonbuild/meson/issues/8642 is
resolved.
Normally ls-files prints the full path to files from the repo root. But when
$GIT_WORK_TREE is set, ls-files prints paths relative to the current
directory. When rebasing, $GIT_WORK_TREE is set in the commands executed from
'rebase -x'. This causes problems if meson config is touched and the meson
reconfigures itself. ($GIT_WORK_TREE shouldn't be relevant, since the paths that
ls-files reports don't depend on the work tree, but whatever.) Let's unset
GIT_WORK_TREE to avoid the issue.
$ (cd test; git --git-dir=$PWD/../.git ls-files ':/test/dmidecode-dumps/*.bin')
test/dmidecode-dumps/HP-Z600.bin
test/dmidecode-dumps/Lenovo-ThinkPad-X280.bin
test/dmidecode-dumps/Lenovo-Thinkcentre-m720s.bin
$ (cd test; GIT_WORK_TREE=$PWD/.. git --git-dir=$PWD/../.git ls-files ':/test/dmidecode-dumps/*.bin')
dmidecode-dumps/HP-Z600.bin
dmidecode-dumps/Lenovo-ThinkPad-X280.bin
dmidecode-dumps/Lenovo-Thinkcentre-m720s.bin
Fixes#18148.
318/365 fuzz-bus-message:crash-26bba7182dedc8848939931d9fcefcb7922f2e56:address OK 0.03 s
319/365 fuzz-bus-message:crash-29ed3c202e0ffade3cad42c8bbeb6cc68a21eb8e:address OK 0.03 s
320/365 fuzz-bus-message:crash-b88ad9ecf4aacf4a0caca5b5543953265367f084:address OK 0.03 s
321/365 fuzz-bus-message:crash-c1b37b4729b42c0c05b23cba4eed5d8102498a1e:address OK 0.03 s
322/365 fuzz-bus-message:crash-d8f3941c74219b4c03532c9b244d5ea539c61af5:address OK 0.03 s
323/365 fuzz-bus-message:crash-e1b811da5ca494e494b77c6bd8e1c2f2989425c5:address OK 0.03 s
324/365 fuzz-bus-message:leak-c09c0e2256d43bc5e2d02748c8d8760e7bc25d20:address OK 0.04 s
325/365 fuzz-bus-message:message1:address OK 0.03 s
326/365 fuzz-bus-message:timeout-08ee8f6446a4064db064e8e0b3d220147f7d0b5b:address OK 0.03 s
327/365 fuzz-dhcp-server:discover-existing:address OK 0.04 s
328/365 fuzz-dhcp-server:discover-new:address OK 0.03 s
329/365 fuzz-dhcp-server:release:address OK 0.04 s
330/365 fuzz-dhcp-server:request-existing:address OK 0.03 s
331/365 fuzz-dhcp-server:request-new:address OK 0.03 s
332/365 fuzz-dhcp-server:request-reboot:address OK 0.03 s
333/365 fuzz-dhcp-server:request-renew:address OK 0.03 s
334/365 fuzz-dns-packet:issue-7888:address OK 0.03 s
335/365 fuzz-dns-packet:oss-fuzz-5465:address OK 0.03 s
336/365 fuzz-journal-remote:crash-5a8f03d4c3a46fcded39527084f437e8e4b54b76:address OK 0.06 s
337/365 fuzz-journal-remote:crash-96dee870ea66d03e89ac321eee28ea63a9b9aa45:address OK 0.04 s
338/365 fuzz-journal-remote:invalid-ts.txt:address OK 0.04 s
339/365 fuzz-journal-remote:oss-fuzz-8659:address OK 0.06 s
340/365 fuzz-journal-remote:oss-fuzz-8686:address OK 0.04 s
341/365 fuzz-journal-remote:sample.txt:address OK 0.07 s
342/365 fuzz-unit-file:directives.service:address OK 0.03 s
343/365 fuzz-unit-file:empty.scope:address OK 0.04 s
344/365 fuzz-unit-file:machine.slice:address OK 0.03 s
345/365 fuzz-unit-file:oss-fuzz-6884:address OK 0.05 s
346/365 fuzz-unit-file:oss-fuzz-6885:address OK 0.03 s
347/365 fuzz-unit-file:oss-fuzz-6886:address OK 0.04 s
348/365 fuzz-unit-file:oss-fuzz-6892:address OK 0.03 s
349/365 fuzz-unit-file:oss-fuzz-6897:address OK 0.05 s
350/365 fuzz-unit-file:oss-fuzz-6897-evverx:address OK 0.04 s
351/365 fuzz-unit-file:oss-fuzz-6908:address OK 0.05 s
352/365 fuzz-unit-file:oss-fuzz-6917:address OK 0.06 s
353/365 fuzz-unit-file:oss-fuzz-6977:address OK 0.08 s
354/365 fuzz-unit-file:oss-fuzz-6977-unminimized:address OK 0.10 s
355/365 fuzz-unit-file:oss-fuzz-7004:address OK 0.03 s
356/365 fuzz-unit-file:oss-fuzz-8064:address OK 0.03 s
357/365 fuzz-unit-file:oss-fuzz-8827:address OK 0.50 s
358/365 fuzz-unit-file:proc-sys-fs-binfmt_misc.automount:address OK 0.03 s
359/365 fuzz-unit-file:syslog.socket:address OK 0.03 s
360/365 fuzz-unit-file:systemd-ask-password-console.path:address OK 0.03 s
361/365 fuzz-unit-file:systemd-machined.service:address OK 0.03 s
362/365 fuzz-unit-file:systemd-resolved.service:address OK 0.03 s
363/365 fuzz-unit-file:systemd-tmpfiles-clean.timer:address OK 0.03 s
364/365 fuzz-unit-file:timers.target:address OK 0.03 s
365/365 fuzz-unit-file:var-lib-machines.mount:address OK 0.04 s
This gives us slightly nicer coverage in the normal test run.
When in a git repo, git ls-files is used to get a list of files known to git.
This mirrors what update-man-rules does for man files. Only looking at files
known to git makes it easier to not forget to commit the test file to git,
and also makes bisecting easier if some files are left in repo.
When outside of a git repo, we expect to be unpacked from a tarball, so just
using all files reported by ls is OK.
There isn't really much need to keep them separate. Anything which is a good
corpus entry can be used as a smoke test, and anything which which is a
regression test can just as well be inserted into the corpus.
The only functional difference from this patch (apart from different paths in
output) is that the regression tests are now zipped together with the rest of
the corpus.
$ meson configure build -Dslow-tests=true && ninja -C build test
...
307/325 fuzz-dns-packet:issue-7888:address OK 0.06 s
308/325 fuzz-dns-packet:oss-fuzz-5465:address OK 0.04 s
309/325 fuzz-journal-remote:crash-5a8f03d4c3a46fcded39527084f437e8e4b54b76:address OK 0.07 s
310/325 fuzz-journal-remote:crash-96dee870ea66d03e89ac321eee28ea63a9b9aa45:address OK 0.05 s
311/325 fuzz-journal-remote:oss-fuzz-8659:address OK 0.05 s
312/325 fuzz-journal-remote:oss-fuzz-8686:address OK 0.07 s
313/325 fuzz-unit-file:oss-fuzz-6884:address OK 0.06 s
314/325 fuzz-unit-file:oss-fuzz-6885:address OK 0.05 s
315/325 fuzz-unit-file:oss-fuzz-6886:address OK 0.05 s
316/325 fuzz-unit-file:oss-fuzz-6892:address OK 0.05 s
317/325 fuzz-unit-file:oss-fuzz-6897:address OK 0.05 s
318/325 fuzz-unit-file:oss-fuzz-6897-evverx:address OK 0.06 s
319/325 fuzz-unit-file:oss-fuzz-6908:address OK 0.07 s
320/325 fuzz-unit-file:oss-fuzz-6917:address OK 0.07 s
321/325 fuzz-unit-file:oss-fuzz-6977:address OK 0.13 s
322/325 fuzz-unit-file:oss-fuzz-6977-unminimized:address OK 0.12 s
323/325 fuzz-unit-file:oss-fuzz-7004:address OK 0.05 s
324/325 fuzz-unit-file:oss-fuzz-8064:address OK 0.05 s
325/325 fuzz-unit-file:oss-fuzz-8827:address OK 0.52 s
