Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New features, minor fixes #276

Open
wants to merge 21 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
93b3ffa
Handling the edge case when original user is unknown
kincsescsaba Feb 20, 2024
f27b4b9
Fixed invalid qemu option
kincsescsaba Feb 20, 2024
8aa0ec4
Added settings file for volatile settings
kincsescsaba Feb 22, 2024
52ff4d1
Sourced volatile settings to be expanded
kincsescsaba Feb 22, 2024
5b41d64
Referenced volatile settings, caching downloads as a default
kincsescsaba Feb 22, 2024
21510c8
Fixed Kernel URL
kincsescsaba Feb 22, 2024
ae7718d
Restricting kernel directory reach using setting var expansion instea…
kincsescsaba Feb 22, 2024
8661592
Typo fix
kincsescsaba Feb 22, 2024
e24c33f
Bug fixes (missing settings source, variable substitution)
kincsescsaba Feb 22, 2024
7766ed5
Handling non-exiting Qemu child process to let parent process exit
kincsescsaba Feb 23, 2024
3161f09
Volatile settings expansion for Busybox, Glibc
kincsescsaba Feb 23, 2024
c72d114
kernel.config and busybox.config now function as patches
kincsescsaba Feb 24, 2024
ae48132
make defconfig is now non-interactive - use kernel.config, busybox.co…
kincsescsaba Feb 24, 2024
6a163fc
Qemu test cleanup
kincsescsaba Feb 24, 2024
ae238a4
Qemu console mode now accepts input
kincsescsaba Feb 25, 2024
c2f6034
Skipping rescue shell options, jumping straight to inittab reading (/…
kincsescsaba Feb 26, 2024
a6caeb0
Bundles volatile config proof-of-concept
kincsescsaba Feb 27, 2024
ecf072f
passwd bundle added to settings
kincsescsaba Feb 27, 2024
45cbcd5
Default empty disks and virtfs option for qemu bios console
kincsescsaba Feb 29, 2024
6c93d18
Added jq bundle
kincsescsaba Mar 1, 2024
84b5f4e
Added e2fsprogs bundle, Log to virt disk test
kincsescsaba Mar 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions src/.config
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,19 @@
#
# https://kernel.org
#
KERNEL_SOURCE_URL=https://kernel.org/pub/linux/kernel/v5.x/linux-5.18.3.tar.xz
KERNEL_SOURCE_URL=https://mirrors.edge.kernel.org/pub/linux/kernel/v$(echo $KERNEL_VERSION | cut -d '.' -f 1).x/linux-$KERNEL_VERSION.tar.xz

# You can find the latest GNU C library source bundles here:
#
# https://gnu.org/software/libc
#
GLIBC_SOURCE_URL=https://ftp.gnu.org/gnu/glibc/glibc-2.35.tar.bz2
GLIBC_SOURCE_URL=https://ftp.gnu.org/gnu/glibc/glibc-$GLIBC_VERSION.tar.bz2

# You can find the latest Busybox source bundles here:
#
# https://busybox.net
#
BUSYBOX_SOURCE_URL=https://busybox.net/downloads/busybox-1.34.1.tar.bz2
BUSYBOX_SOURCE_URL=https://busybox.net/downloads/busybox-$BUSYBOX_VERSION.tar.bz2

# You can find the latest Syslinux source bundles here:
#
Expand Down Expand Up @@ -65,7 +65,7 @@ FIRMWARE_TYPE=bios
#
# minimal_config/kernel.config
#
USE_PREDEFINED_KERNEL_CONFIG=false
USE_PREDEFINED_KERNEL_CONFIG=true

# Build kernel modules and make them available in the overlayfs / rootfs in the
# /lib/modules directory. The modules can be loaded via 'insmod' to support
Expand All @@ -80,7 +80,7 @@ BUILD_KERNEL_MODULES=false
#
# minimal_config/busybox.config
#
USE_PREDEFINED_BUSYBOX_CONFIG=false
USE_PREDEFINED_BUSYBOX_CONFIG=true

# Define the overlay type to use. Possible values are 'sparse' and 'folder'.
# You can use any other value, no value, or comment the property in order to
Expand Down Expand Up @@ -190,7 +190,7 @@ OVERLAY_TYPE=folder
# * mll_logo
# * mll_source
#
OVERLAY_BUNDLES=dhcp,mll_hello,mll_logo,mll_source
OVERLAY_BUNDLES=$ADD_BUNDLES

# The location where the overlay bundle software will be stored.
#
Expand All @@ -208,7 +208,7 @@ OVERLAY_LOCATION=iso
# sources and have no internet connection, or if you want to share your version
# of 'Minimal Linux Live' with predefined sources.
#
USE_LOCAL_SOURCE=false
USE_LOCAL_SOURCE=true

# This property enables the standard penguin boot logo in the upper left corner
# of the screen. The property is used in 'xx_build_kernel.sh'. The default
Expand Down
115 changes: 59 additions & 56 deletions src/02_build_kernel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@ set -e

# Load common properties and functions in the current script.
. ./common.sh
. ./settings

echo "*** BUILD KERNEL BEGIN ***"

# Change to the kernel source directory which ls finds, e.g. 'linux-4.4.6'.
cd `ls -d $WORK_DIR/kernel/linux-*`
cd `ls -d $WORK_DIR/kernel/linux-$KERNEL_VERSION`

# Cleans up the kernel sources, including configuration files.
echo "Preparing kernel work area."
Expand All @@ -23,87 +24,89 @@ if [ "$USE_PREDEFINED_KERNEL_CONFIG" = "true" -a ! -f $SRC_DIR/minimal_config/ke
USE_PREDEFINED_KERNEL_CONFIG=false
fi

if [ "$USE_PREDEFINED_KERNEL_CONFIG" = "true" ] ; then
# Use predefined configuration file for the kernel.
echo "Using config file '$SRC_DIR/minimal_config/kernel.config'."
cp -f $SRC_DIR/minimal_config/kernel.config .config
else
# Create default configuration file for the kernel.
make defconfig -j $NUM_JOBS
echo "Generated default kernel configuration."
# Create default configuration file for the kernel.
yes | make defconfig -j $NUM_JOBS

# Changes the name of the system to 'minimal'.
sed -i "s/.*CONFIG_DEFAULT_HOSTNAME.*/CONFIG_DEFAULT_HOSTNAME=\"minimal\"/" .config
echo "Generated default kernel configuration."

# OVERLAYFS - BEGIN - most features are disabled (you don't really need them)
# Changes the name of the system to 'minimal'.
sed -i "s/.*CONFIG_DEFAULT_HOSTNAME.*/CONFIG_DEFAULT_HOSTNAME=\"minimal\"/" .config

# Enable overlay support, e.g. merge ro and rw directories (3.18+).
sed -i "s/.*CONFIG_OVERLAY_FS.*/CONFIG_OVERLAY_FS=y/" .config
# OVERLAYFS - BEGIN - most features are disabled (you don't really need them)

# Turn on redirect dir feature by default (4.10+).
echo "# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set" >> .config
# Enable overlay support, e.g. merge ro and rw directories (3.18+).
sed -i "s/.*CONFIG_OVERLAY_FS.*/CONFIG_OVERLAY_FS=y/" .config

# Turn on inodes index feature by default (4.13+).
echo "# CONFIG_OVERLAY_FS_INDEX is not set" >> .config
# Turn on redirect dir feature by default (4.10+).
echo "# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set" >> .config

# Follow redirects even if redirects are turned off (4.15+).
echo "CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y" >> .config
# Turn on inodes index feature by default (4.13+).
echo "# CONFIG_OVERLAY_FS_INDEX is not set" >> .config

# Turn on NFS export feature by default (4.16+).
echo "# CONFIG_OVERLAY_FS_NFS_EXPORT is not set" >> .config
# Follow redirects even if redirects are turned off (4.15+).
echo "CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y" >> .config

# Auto enable inode number mapping (4.17+).
echo "# CONFIG_OVERLAY_FS_XINO_AUTO is not set" >> .config
# Turn on NFS export feature by default (4.16+).
echo "# CONFIG_OVERLAY_FS_NFS_EXPORT is not set" >> .config

# Тurn on metadata only copy up feature by default (4.19+).
echo "# CONFIG_OVERLAY_FS_METACOPY is not set" >> .config
# Auto enable inode number mapping (4.17+).
echo "# CONFIG_OVERLAY_FS_XINO_AUTO is not set" >> .config

# OVERLAYFS - END
# Тurn on metadata only copy up feature by default (4.19+).
echo "# CONFIG_OVERLAY_FS_METACOPY is not set" >> .config

# Step 1 - disable all active kernel compression options (should be only one).
sed -i "s/.*\\(CONFIG_KERNEL_.*\\)=y/\\#\\ \\1 is not set/" .config
# OVERLAYFS - END

# Step 2 - enable the 'xz' compression option.
sed -i "s/.*CONFIG_KERNEL_XZ.*/CONFIG_KERNEL_XZ=y/" .config
# Step 1 - disable all active kernel compression options (should be only one).
sed -i "s/.*\\(CONFIG_KERNEL_.*\\)=y/\\#\\ \\1 is not set/" .config

# Enable the VESA framebuffer for graphics support.
sed -i "s/.*CONFIG_FB_VESA.*/CONFIG_FB_VESA=y/" .config
# Step 2 - enable the 'xz' compression option.
sed -i "s/.*CONFIG_KERNEL_XZ.*/CONFIG_KERNEL_XZ=y/" .config

# Read the 'USE_BOOT_LOGO' property from '.config'
USE_BOOT_LOGO=`read_property USE_BOOT_LOGO`
# Enable the VESA framebuffer for graphics support.
sed -i "s/.*CONFIG_FB_VESA.*/CONFIG_FB_VESA=y/" .config

# Read the 'USE_BOOT_LOGO' property from '.config'
USE_BOOT_LOGO=`read_property USE_BOOT_LOGO`

if [ "$USE_BOOT_LOGO" = "true" ] ; then
sed -i "s/.*CONFIG_LOGO_LINUX_CLUT224.*/CONFIG_LOGO_LINUX_CLUT224=y/" .config
echo "Boot logo is enabled."
else
sed -i "s/.*CONFIG_LOGO_LINUX_CLUT224.*/\\# CONFIG_LOGO_LINUX_CLUT224 is not set/" .config
echo "Boot logo is disabled."
fi

if [ "$USE_BOOT_LOGO" = "true" ] ; then
sed -i "s/.*CONFIG_LOGO_LINUX_CLUT224.*/CONFIG_LOGO_LINUX_CLUT224=y/" .config
echo "Boot logo is enabled."
else
sed -i "s/.*CONFIG_LOGO_LINUX_CLUT224.*/\\# CONFIG_LOGO_LINUX_CLUT224 is not set/" .config
echo "Boot logo is disabled."
fi
# Disable debug symbols in kernel => smaller kernel binary.
sed -i "s/^CONFIG_DEBUG_KERNEL.*/\\# CONFIG_DEBUG_KERNEL is not set/" .config

# Disable debug symbols in kernel => smaller kernel binary.
sed -i "s/^CONFIG_DEBUG_KERNEL.*/\\# CONFIG_DEBUG_KERNEL is not set/" .config
# Enable the EFI stub
sed -i "s/.*CONFIG_EFI_STUB.*/CONFIG_EFI_STUB=y/" .config

# Enable the EFI stub
sed -i "s/.*CONFIG_EFI_STUB.*/CONFIG_EFI_STUB=y/" .config
# Request that the firmware clear the contents of RAM after reboot (4.14+).
echo "CONFIG_RESET_ATTACK_MITIGATION=y" >> .config

# Request that the firmware clear the contents of RAM after reboot (4.14+).
echo "CONFIG_RESET_ATTACK_MITIGATION=y" >> .config
# Disable Apple Properties (Useful for Macs but useless in general)
echo "CONFIG_APPLE_PROPERTIES=n" >> .config

# Disable Apple Properties (Useful for Macs but useless in general)
echo "CONFIG_APPLE_PROPERTIES=n" >> .config
# Check if we are building 64-bit kernel.
if [ "`grep "CONFIG_X86_64=y" .config`" = "CONFIG_X86_64=y" ] ; then
# Enable the mixed EFI mode when building 64-bit kernel.
echo "CONFIG_EFI_MIXED=y" >> .config
fi

# Check if we are building 64-bit kernel.
if [ "`grep "CONFIG_X86_64=y" .config`" = "CONFIG_X86_64=y" ] ; then
# Enable the mixed EFI mode when building 64-bit kernel.
echo "CONFIG_EFI_MIXED=y" >> .config
fi
if [ "$USE_PREDEFINED_KERNEL_CONFIG" = "true" ] ; then
# Use predefined configuration file for the kernel.
echo "Using config file '$SRC_DIR/minimal_config/kernel.config' to update defaults."
$SRC_DIR/./update_config.sh .config $SRC_DIR/minimal_config/kernel.config
fi

# Compile the kernel with optimization for 'parallel jobs' = 'number of processors'.
# Good explanation of the different kernels:
# http://unix.stackexchange.com/questions/5518/what-is-the-difference-between-the-following-kernel-makefile-terms-vmlinux-vmlinux
echo "Building kernel."
make \

yes | make \
CFLAGS="$CFLAGS" \
bzImage -j $NUM_JOBS

Expand Down
3 changes: 2 additions & 1 deletion src/04_build_glibc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ set -e

# Load common properties and functions in the current script.
. ./common.sh
. ./settings

echo "*** BUILD GLIBC BEGIN ***"

Expand All @@ -18,7 +19,7 @@ rm -rf $GLIBC_INSTALLED
mkdir $GLIBC_INSTALLED

# Find the glibc source directory, e.g. 'glibc-2.23' and remember it.
GLIBC_SRC=`ls -d $WORK_DIR/glibc/glibc-*`
GLIBC_SRC=`ls -d $WORK_DIR/glibc/glibc-$GLIBC_VERSION`

# All glibc work is done from the working area.
cd $GLIBC_OBJECTS
Expand Down
17 changes: 9 additions & 8 deletions src/07_build_busybox.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ set -e

# Load common properties and functions in the current script.
. ./common.sh
. ./settings

echo "*** BUILD BUSYBOX BEGIN ***"

Expand All @@ -12,7 +13,7 @@ echo "Removing old Busybox artifacts. This may take a while."
rm -rf $BUSYBOX_INSTALLED

# Change to the source directory ls finds, e.g. 'busybox-1.24.2'.
cd `ls -d $WORK_DIR/busybox/busybox-*`
cd `ls -d $WORK_DIR/busybox/busybox-$BUSYBOX_VERSION`

# Remove previously generated artifacts.
echo "Preparing Busybox work area. This may take a while."
Expand All @@ -26,14 +27,14 @@ if [ "$USE_PREDEFINED_BUSYBOX_CONFIG" = "true" -a ! -f $SRC_DIR/minimal_config/b
USE_PREDEFINED_BUSYBOX_CONFIG="false"
fi

# Create default configuration file.
echo "Generating default Busybox configuration."
yes | make defconfig -j $NUM_JOBS

if [ "$USE_PREDEFINED_BUSYBOX_CONFIG" = "true" ] ; then
# Use predefined configuration file for Busybox.
echo "Using config file $SRC_DIR/minimal_config/busybox.config"
cp -f $SRC_DIR/minimal_config/busybox.config .config
else
# Create default configuration file.
echo "Generating default Busybox configuration."
make defconfig -j $NUM_JOBS
echo "Using config file $SRC_DIR/minimal_config/busybox.config to update defaults."
$SRC_DIR/./update_config.sh .config $SRC_DIR/minimal_config/busybox.config
fi

# Now we tell Busybox to use the sysroot area.
Expand All @@ -44,7 +45,7 @@ sed -i "s|.*CONFIG_EXTRA_CFLAGS.*|CONFIG_EXTRA_CFLAGS=\"$CFLAGS -L$SYSROOT/lib\"

# Compile busybox with optimization for "parallel jobs" = "number of processors".
echo "Building Busybox."
make \
yes | make \
busybox -j $NUM_JOBS

# Create the symlinks for busybox. The file 'busybox.links' is used for this.
Expand Down
1 change: 1 addition & 0 deletions src/08_prepare_bundles.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ set -e

# Load common properties and functions in the current script.
. ./common.sh
. ./settings

echo "*** PREPARE OVERLAY BEGIN ***"

Expand Down
15 changes: 10 additions & 5 deletions src/16_cleanup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,19 @@ set -e
echo "*** CLEANUP BEGIN ***"

if [ "$(id -u)" = "0" ] ; then
echo "Applying original ownership to all affected files. This may take a while."

# Find the original user. Note that this may not always be correct.
ORIG_USER=`who | head -n 1 | awk '{print \$1}'`
echo "Original user is '$ORIG_USER'."

# Apply ownership back to original owner for all affected files.
chown -R $ORIG_USER:$ORIG_USER *
if [ -n "$var" ]; then
echo "Applying original ownership to all affected files. This may take a while."

echo "Original user is '$ORIG_USER'."

# Apply ownership back to original owner for all affected files.
chown -R $ORIG_USER:$ORIG_USER *
else
echo "Original user unknown"
fi
else
echo "No need to perform cleanup."
fi
Expand Down
5 changes: 4 additions & 1 deletion src/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

set -e

. ./settings

SRC_DIR=$PWD
CONFIG=$SRC_DIR/.config
SOURCE_DIR=$SRC_DIR/source
Expand Down Expand Up @@ -32,7 +34,8 @@ read_property() (

if [ ! "$prop_name" = "" ] ; then
# Search in the main '.config' file.
prop_value=`grep -i ^${prop_name}= $CONFIG | cut -f2- -d'=' | xargs`
value_def=`grep -i ^${prop_name}= $CONFIG | cut -f2- -d'=' | xargs`
prop_value=`eval echo $value_def`
fi

echo $prop_value
Expand Down
1 change: 1 addition & 0 deletions src/minimal_overlay/bundles/e2fsprogs/.config
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
E2FSPROGS_SOURCE_URL=https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/snapshot/e2fsprogs-$E2FSPROGS_VERSION.tar.gz
39 changes: 39 additions & 0 deletions src/minimal_overlay/bundles/e2fsprogs/01_get.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
#!/bin/sh

set -e

. ../../common.sh
. ../../../settings

# Read the common configuration properties.
DOWNLOAD_URL=`read_property E2FSPROGS_SOURCE_URL`
USE_LOCAL_SOURCE=`read_property USE_LOCAL_SOURCE`

# Grab everything after the last '/' character.
ARCHIVE_FILE=${DOWNLOAD_URL##*/}

if [ "$USE_LOCAL_SOURCE" = "true" -a ! -f $MAIN_SRC_DIR/source/overlay/$ARCHIVE_FILE ] ; then
echo "Source bundle $MAIN_SRC_DIR/source/overlay/$ARCHIVE_FILE is missing and will be downloaded."
USE_LOCAL_SOURCE="false"
fi

cd $MAIN_SRC_DIR/source/overlay

if [ ! "$USE_LOCAL_SOURCE" = "true" ] ; then
# Downloading util-linux source bundle file. The '-c' option allows the download to resume.
echo "Downloading jq source bundle from $DOWNLOAD_URL"
wget -c $DOWNLOAD_URL
else
echo "Using local jq source bundle $MAIN_SRC_DIR/source/overlay/$ARCHIVE_FILE"
fi

# Delete folder with previously extracted jq.
echo "Removing util-linux work area. This may take a while."
rm -rf $WORK_DIR/overlay/$BUNDLE_NAME
mkdir $WORK_DIR/overlay/$BUNDLE_NAME

# Extract util-linux to folder 'work/overlay/util_linux'.
# Full path will be something like 'work/overlay/util_linux/util-linux-2.31'.
tar -xvf $ARCHIVE_FILE -C $WORK_DIR/overlay/$BUNDLE_NAME

cd $SRC_DIR
Loading