[PATCH] scripts/make-gpt.sh: allow setting partition size
We want non-minimally-sized partitions to leave space for updates. All such partitions will also be labelled, so we can just add another optional field at the end. Since we don't parse partition specifications in sh, we can't keep a running total any more, so instead we just go through the table at the end and add up all the sizes, taking advantage of our knowledge that the size will always be the last thing in each line in our tables. Signed-off-by: Alyssa Ross <hi@alyssa.is> --- This is an alternative to Demi's implementation[1], which required Bash features. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20251126-updates-v5-3-... scripts/make-gpt.sh | 28 +++++++++++++++------------- scripts/sfdisk-field.awk | 22 +++++++++++++++++----- 2 files changed, 32 insertions(+), 18 deletions(-) diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 96f0d2c..07eac50 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,10 +1,10 @@ #!/bin/sh -eu # -# SPDX-FileCopyrightText: 2021-2023 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2021-2023, 2025 Alyssa Ross <hi@alyssa.is> # SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # -# usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID[:PARTLABEL]]... +# usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID[:PARTLABEL[:PARTMiB]]]... ONE_MiB=1048576 @@ -28,7 +28,8 @@ fillPartition() { lseek -S 1 "$start" cat "$3" 1<>"$1" } -# Prints the partition path from a PATH:PARTTYPE[:PARTUUID[:PARTLABEL]] string. +# Prints the partition path from a +# PATH:PARTTYPE[:PARTUUID[:PARTLABEL[:PARTMiB]]] string. partitionPath() { awk -F: '{print $1}' <<EOF $1 @@ -40,21 +41,22 @@ scriptsDir="$(dirname "$0")" out="$1" shift -nl=' -' -table="label: gpt" +table=$(for partition; do + awk -f "$scriptsDir/sfdisk-field.awk" \ + -v partition="$partition" \ + -v size="$(sizeMiB "$(partitionPath "$partition")")" +done) # Keep 1MiB free at the start, and 1MiB free at the end. -gptBytes=$((ONE_MiB * 2)) -for partition; do - sizeMiB="$(sizeMiB "$(partitionPath "$partition")")" - table="$table${nl}size=${sizeMiB}MiB,$(awk -f "$scriptsDir/sfdisk-field.awk" -v partition="$partition")" - gptBytes="$((gptBytes + sizeMiB * ONE_MiB))" -done +# Here we rely on sfdisk-field.awk always putting size last +gptMiB=$(printf "%s\n" "$table" | awk -F= '{size += $NF} END {print size + 2}') rm -f "$out" -truncate -s "$gptBytes" "$out" +truncate -s "${gptMiB}M" "$out" + sfdisk --no-reread --no-tell-kernel "$out" <<EOF +label: gpt +sector-size: $ONE_MiB $table EOF diff --git a/scripts/sfdisk-field.awk b/scripts/sfdisk-field.awk index e13c86d..78b438e 100644 --- a/scripts/sfdisk-field.awk +++ b/scripts/sfdisk-field.awk @@ -1,7 +1,7 @@ #!/usr/bin/awk -f # # SPDX-License-Identifier: EUPL-1.2+ -# SPDX-FileCopyrightText: 2022, 2024 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2022, 2024-2025 Alyssa Ross <hi@alyssa.is> BEGIN { types["root.aarch64"] = "b921b045-1df0-41c3-af44-4c6f280d3fae" @@ -9,12 +9,11 @@ BEGIN { types["verity.aarch64"] = "df3300ce-d69f-4c92-978c-9bfb0f38d820" types["verity.x86_64"] = "2c7357ed-ebd2-46d9-aec1-23d437ec2bf5" - # Field #1 is the partition path, which make-gpt.sh will turn into - # the size field. Since it's handled elsewhere, we skip that - # first field. + # Field #1 is the partition path, which is read by make-gpt.sh + # but not relevant for running sfdisk, so skip it. skip=1 - split("type uuid name", keys) + split("type uuid name size", keys) split(partition, fields, ":") arch = ENVIRON["ARCH"] @@ -31,8 +30,21 @@ BEGIN { if (keys[n - skip] == "type") { if (uuid = types[fields[n] "." arch]) fields[n] = uuid + } else if (keys[n - skip] == "size") { + if (fields[n] < size) { + printf "%s MiB partition content is too big for %s MiB partition\n", + size, fields[n] > "/dev/stderr" + exit 1 + } + + size = fields[n] + continue # Handled at the end. } printf "%s=%s,", keys[n - skip], fields[n] } + + # Always output a size field, either supplied in input or + # default value of the size variable. + printf "size=%s\n", size } base-commit: c43e5c63a028994d5f66a15db19f415bf3cb7736 -- 2.51.0
On 11/27/25 12:40, Alyssa Ross wrote:
We want non-minimally-sized partitions to leave space for updates. All such partitions will also be labelled, so we can just add another optional field at the end.
Since we don't parse partition specifications in sh, we can't keep a running total any more, so instead we just go through the table at the end and add up all the sizes, taking advantage of our knowledge that the size will always be the last thing in each line in our tables.
Signed-off-by: Alyssa Ross <hi@alyssa.is> --- This is an alternative to Demi's implementation[1], which required Bash features.
[1]: https://spectrum-os.org/lists/archives/spectrum-devel/20251126-updates-v5-3-...
scripts/make-gpt.sh | 28 +++++++++++++++------------- scripts/sfdisk-field.awk | 22 +++++++++++++++++----- 2 files changed, 32 insertions(+), 18 deletions(-)
diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 96f0d2c..07eac50 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,10 +1,10 @@ #!/bin/sh -eu # -# SPDX-FileCopyrightText: 2021-2023 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2021-2023, 2025 Alyssa Ross <hi@alyssa.is> # SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # -# usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID[:PARTLABEL]]... +# usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID[:PARTLABEL[:PARTMiB]]]...
ONE_MiB=1048576
@@ -28,7 +28,8 @@ fillPartition() { lseek -S 1 "$start" cat "$3" 1<>"$1" }
-# Prints the partition path from a PATH:PARTTYPE[:PARTUUID[:PARTLABEL]] string. +# Prints the partition path from a +# PATH:PARTTYPE[:PARTUUID[:PARTLABEL[:PARTMiB]]] string. partitionPath() { awk -F: '{print $1}' <<EOF $1 @@ -40,21 +41,22 @@ scriptsDir="$(dirname "$0")" out="$1" shift
-nl=' -' -table="label: gpt" +table=$(for partition; do + awk -f "$scriptsDir/sfdisk-field.awk" \ + -v partition="$partition" \ + -v size="$(sizeMiB "$(partitionPath "$partition")")" +done)
# Keep 1MiB free at the start, and 1MiB free at the end. -gptBytes=$((ONE_MiB * 2)) -for partition; do - sizeMiB="$(sizeMiB "$(partitionPath "$partition")")" - table="$table${nl}size=${sizeMiB}MiB,$(awk -f "$scriptsDir/sfdisk-field.awk" -v partition="$partition")" - gptBytes="$((gptBytes + sizeMiB * ONE_MiB))" -done +# Here we rely on sfdisk-field.awk always putting size last +gptMiB=$(printf "%s\n" "$table" | awk -F= '{size += $NF} END {print size + 2}')
Needs -o pipefail, though I believe this will be in the next revision of POSIX if it isn't already. You can avoid that with some awk magic but the script gets harder to read.
rm -f "$out" -truncate -s "$gptBytes" "$out" +truncate -s "${gptMiB}M" "$out" + sfdisk --no-reread --no-tell-kernel "$out" <<EOF +label: gpt +sector-size: $ONE_MiB
I'm concerned about this line. GPT sector sizes aren't just used to compute partition sizes. They are part of the on-disk format, and moving an image from a disk to a different disk with a different sector size requires fixing on-disk data structures. I wrote a tool to do this for Qubes OS, but it is also possible to generate a polyglot GPT that works for both 512-byte and 4096-byte sectors. I recommend we use this for the installer eventually. That said, if the installer boots with this it should not have a problem.
$table EOF
diff --git a/scripts/sfdisk-field.awk b/scripts/sfdisk-field.awk index e13c86d..78b438e 100644 --- a/scripts/sfdisk-field.awk +++ b/scripts/sfdisk-field.awk @@ -1,7 +1,7 @@ #!/usr/bin/awk -f # # SPDX-License-Identifier: EUPL-1.2+ -# SPDX-FileCopyrightText: 2022, 2024 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2022, 2024-2025 Alyssa Ross <hi@alyssa.is>
BEGIN { types["root.aarch64"] = "b921b045-1df0-41c3-af44-4c6f280d3fae" @@ -9,12 +9,11 @@ BEGIN { types["verity.aarch64"] = "df3300ce-d69f-4c92-978c-9bfb0f38d820" types["verity.x86_64"] = "2c7357ed-ebd2-46d9-aec1-23d437ec2bf5"
- # Field #1 is the partition path, which make-gpt.sh will turn into - # the size field. Since it's handled elsewhere, we skip that - # first field. + # Field #1 is the partition path, which is read by make-gpt.sh + # but not relevant for running sfdisk, so skip it. skip=1
- split("type uuid name", keys) + split("type uuid name size", keys) split(partition, fields, ":")
arch = ENVIRON["ARCH"] @@ -31,8 +30,21 @@ BEGIN { if (keys[n - skip] == "type") { if (uuid = types[fields[n] "." arch]) fields[n] = uuid + } else if (keys[n - skip] == "size") { + if (fields[n] < size) { + printf "%s MiB partition content is too big for %s MiB partition\n", + size, fields[n] > "/dev/stderr" + exit 1 + } + + size = fields[n] + continue # Handled at the end. }
printf "%s=%s,", keys[n - skip], fields[n] } + + # Always output a size field, either supplied in input or + # default value of the size variable. + printf "size=%s\n", size }
base-commit: c43e5c63a028994d5f66a15db19f415bf3cb7736
This should work if I read the code correctly, so if it passes tests: Acked-by: Demi Marie Obenour <demiobenour@gmail.com> The check for not fitting in the partition is a nice improvement over my patch. -- Sincerely, Demi Marie Obenour (she/her/hers)
Demi Marie Obenour <demiobenour@gmail.com> writes:
On 11/27/25 12:40, Alyssa Ross wrote:
We want non-minimally-sized partitions to leave space for updates. All such partitions will also be labelled, so we can just add another optional field at the end.
Since we don't parse partition specifications in sh, we can't keep a running total any more, so instead we just go through the table at the end and add up all the sizes, taking advantage of our knowledge that the size will always be the last thing in each line in our tables.
Signed-off-by: Alyssa Ross <hi@alyssa.is> --- This is an alternative to Demi's implementation[1], which required Bash features.
[1]: https://spectrum-os.org/lists/archives/spectrum-devel/20251126-updates-v5-3-...
scripts/make-gpt.sh | 28 +++++++++++++++------------- scripts/sfdisk-field.awk | 22 +++++++++++++++++----- 2 files changed, 32 insertions(+), 18 deletions(-)
diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 96f0d2c..07eac50 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,10 +1,10 @@ #!/bin/sh -eu # -# SPDX-FileCopyrightText: 2021-2023 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2021-2023, 2025 Alyssa Ross <hi@alyssa.is> # SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # -# usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID[:PARTLABEL]]... +# usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID[:PARTLABEL[:PARTMiB]]]...
ONE_MiB=1048576
@@ -28,7 +28,8 @@ fillPartition() { lseek -S 1 "$start" cat "$3" 1<>"$1" }
-# Prints the partition path from a PATH:PARTTYPE[:PARTUUID[:PARTLABEL]] string. +# Prints the partition path from a +# PATH:PARTTYPE[:PARTUUID[:PARTLABEL[:PARTMiB]]] string. partitionPath() { awk -F: '{print $1}' <<EOF $1 @@ -40,21 +41,22 @@ scriptsDir="$(dirname "$0")" out="$1" shift
-nl=' -' -table="label: gpt" +table=$(for partition; do + awk -f "$scriptsDir/sfdisk-field.awk" \ + -v partition="$partition" \ + -v size="$(sizeMiB "$(partitionPath "$partition")")" +done)
# Keep 1MiB free at the start, and 1MiB free at the end. -gptBytes=$((ONE_MiB * 2)) -for partition; do - sizeMiB="$(sizeMiB "$(partitionPath "$partition")")" - table="$table${nl}size=${sizeMiB}MiB,$(awk -f "$scriptsDir/sfdisk-field.awk" -v partition="$partition")" - gptBytes="$((gptBytes + sizeMiB * ONE_MiB))" -done +# Here we rely on sfdisk-field.awk always putting size last +gptMiB=$(printf "%s\n" "$table" | awk -F= '{size += $NF} END {print size + 2}')
Needs -o pipefail, though I believe this will be in the next revision of POSIX if it isn't already. You can avoid that with some awk magic but the script gets harder to read.
It is now POSIX! Will add.
rm -f "$out" -truncate -s "$gptBytes" "$out" +truncate -s "${gptMiB}M" "$out" + sfdisk --no-reread --no-tell-kernel "$out" <<EOF +label: gpt +sector-size: $ONE_MiB
I'm concerned about this line. GPT sector sizes aren't just used to compute partition sizes. They are part of the on-disk format, and moving an image from a disk to a different disk with a different sector size requires fixing on-disk data structures. I wrote a tool to do this for Qubes OS, but it is also possible to generate a polyglot GPT that works for both 512-byte and 4096-byte sectors. I recommend we use this for the installer eventually.
That said, if the installer boots with this it should not have a problem.
sfdisk is explicitly documented to only consider this for computing partition size, as I understand it. The actual sector size used will always be 512, 1024, 2048, or 4096: --sector-size sectorsize Specify the sector size of the disk. Valid values are 512, 1024, 2048, and 4096. The kernel is aware of the sector size for regular block devices. Use this option only on very old kernels, when working with disk images, or to override the kernel’s default sector size. Since util-linux-2.17, fdisk distinguishes between logical and physical sector size. This option changes both sector sizes to the specified sectorsize. … sector-size Specifies the sector size used in the input. sfdisk always internally uses the device sector size provided by the kernel for the block device, or as specified by the user on the command line (see --sector-size). Starting with version 2.39, sfdisk recalculates sizes from the input if the sector-size header and device sector size are different.
participants (2)
-
Alyssa Ross -
Demi Marie Obenour