Demi Marie Obenour <demiobenour@gmail.com> writes:
Right now, the makefiles in host/rootfs, vm/sys/net, and img/app have manually-maintained lists of files and symlinks. These duplicate the information in the git repository and can easily get out of sync or cause unnecessary merge conflicts. Fix all of these issues by having the git repository be the source of truth, and using a script to generate the file lists. Developers can regenerate the lists before every commit, or even add a git hook to do that.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
diff --git a/scripts/genfiles.awk b/scripts/genfiles.awk new file mode 100644 index 0000000000000000000000000000000000000000..891ad162ea9748e275f7a048db3acbd9e7895a9b --- /dev/null +++ b/scripts/genfiles.awk @@ -0,0 +1,90 @@ +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2021-2024 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com> +BEGIN { + RS = "\n"; + FS = "\t"; + modes["120000"] = "symlink"; + modes["100644"] = "regular"; + modes["100755"] = "regular"; +} + +function fail(msg) { + exit_code = 1;
This line doesn't do anything now, right?
+ print msg > "/dev/stderr"; + exit 1; +} + +done { fail("Junk after DONE", 1); } + +$0 == "DONE" { + done = 1; + next; +} + +# Extract data from built-in variables. +{ + filename = $2; + raw_mode = $1; + # awk autocreates empty string entries if the key is invalid, + # but the code exits in this case so that is okay. + mode = modes[raw_mode]; +} + +filename !~ /^[[:alnum:]_.+@/-]+$/ { + fail("filename '" filename "' has forbidden characters"); +} + +# Skip license files +/\.license$/ { next } + +filename ~ /^image\/etc\/s6-rc\// { + if (mode != "regular") { + fail("s6-rc-compile input '" filename "' isn't a regular file"); + } + rc_files[rc_count++] = filename; + next; +} + +mode == "symlink" { + symlinks[symlink_count++] = filename; + next; +} + +mode == "regular" { + files[file_count++] = filename; + next; +} + +{ fail("File '" filename "' is not regular file or symlink (mode " raw_mode ")"); } + +END { + if (exit_code) { + exit exit_code; + } + if (!done) { + fail("Did not receive DONE line"); + } + printf ("# SPDX-License-Identifier: EUPL-1.2+\n" \ +"# SPDX-FileCopyrightText: 2021-2024 Alyssa Ross <hi@alyssa.is>\n" \ +"# Generated by scripts/genfile.sh. Any changes will be overwritten.\n" \ +"FILES =") > out_file; + for (array_index = 0; array_index < file_count; array_index += 1) { + printf " \\\n\t%s", files[array_index] > out_file; + } + printf ("\n\n" \ +"# These are separate because they need to be included, but putting\n" \ +"# them as make dependencies would confuse make.\n" \ +"LINKS =") > out_file; + for (array_index = 0; array_index < symlink_count; array_index += 1) { + printf " \\\n\t%s", symlinks[array_index] > out_file; + } + printf "\n\nS6_RC_FILES =" > out_file; + for (array_index = 0; array_index < rc_count; array_index += 1) { + printf " \\\n\t%s", rc_files[array_index] > out_file; + } + print > out_file; + if (close(out_file)) { + fail("Cannot close output file: " ERRNO); + } +} diff --git a/scripts/genfiles.sh b/scripts/genfiles.sh new file mode 100755 index 0000000000000000000000000000000000000000..65e8b56654448f4c9529e00807e68adb0bcfefbf --- /dev/null +++ b/scripts/genfiles.sh @@ -0,0 +1,28 @@ +#!/bin/sh -- +set -euo pipefail +export LC_ALL=C LANGUAGE=C +# shell strips trailing newlines, so add something after the newline +dir=$(git rev-parse --show-toplevel && echo a) +cd "${dir%' +a'}"
What's this for? In case the directory name ends with a newline? All sorts of stuff is going to break if somebody decides to do this. We don't need to go out of our way to accomodate it.
+for i in host/rootfs img/app vm/sys/net; do + output_file=$i/file-list.mk + { + git -C "$i" -c core.quotePath=true ls-files $'--format=%(objectmode)\t%(path)' -- image | + sort -t $'\t' -k 2 + echo DONE
I still don't understand what the DONE is for. Can you describe a circumstance in which it would be necessary?
+ } | + awk -v "out_file=$output_file.tmp" -f scripts/genfiles.awk
This was unresolved from last time too. This could just be stdout and a simpler awk script. If you really want to make sure a temporary file isn't left around if something goes wrong, you could trap EXIT, but it's also just really not a big deal.
+ if [ -f "$output_file" ]; then + # Avoid changing output file if it is up to date, as that + # would cause unnecessary rebuilds. + if cmp -s -- "$output_file.tmp" "$output_file"; then + rm -- "$output_file.tmp" + continue + else + astatus=$? + if [ "$astatus" != 1 ]; then exit "$astatus"; fi + fi + fi + mv -- "$output_file.tmp" "$output_file" +done