Skip to content

Commit 83b2098

Browse files
chrisjsimpsonchrisjsimpson
committed
repeatable minimal boot linux with openssh & ip utils
1 parent 85de59d commit 83b2098

13 files changed

+239
-96
lines changed

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
*.swp

‎README.md

Lines changed: 22 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
# Minimal boot linux coreutils bash glibc
1+
# Minimal build & boot linux kernel + coreutils (via busybox), openssh & iputils
22

33
![qemu showing boot into minimal linux](./img/boot-qemu-example.png)
44

5-
Goal is to have mimimal but realistic node with network capabilities build in (`ip`) and bootstrapped (similar to [alpine's netbot](https://boot.alpinelinux.org/) ([example](https://github.com/KarmaComputing/server-bootstrap/blob/494089caa2c88bbf37a739aa96561231d5847be5/.github/workflows/build-alpine-netboot-image-zfs.yml#L1))). [Clearlinux](https://github.com/clearlinux/distribution) is interesting. This is using glibc over musl (which alpine uses).
5+
Goal is to have mimimal but realistic node with network capabilities build in (`ip`) and bootstrapped (similar to [alpine's netbot](https://boot.alpinelinux.org/) ([example](https://github.com/KarmaComputing/server-bootstrap/blob/494089caa2c88bbf37a739aa96561231d5847be5/.github/workflows/build-alpine-netboot-image-zfs.yml#L1))). [Clearlinux](https://github.com/clearlinux/distribution) is interesting. This is using musl (which alpine uses) rather than glibc to support staticaly built binaries more easily.
66

77

88
## The process in a nutshell
@@ -12,38 +12,26 @@ Goal is to have mimimal but realistic node with network capabilities build in (`
1212
- Write an `init` script
1313
- Build all the binaries + `init` script into a initramfs
1414
- Run/boot with qemu
15+
- Rememebr busybox needs to be static build (see .config)
16+
- <strike>dropbear needs at least a `dropbear_rsa_host_key` key config or will not start see [gist](https://gist.github.com/mad4j/7983719) </strike>
17+
- Prefering openssh for end user compatability (statically built)
1518

1619

1720

1821
TODO: add [iproute2](https://github.com/iproute2/iproute2) for minimal routing.
1922

2023
## Things you need to build
2124

22-
- git clone & build GNU `bash` is in it's own repo
23-
- git clone & build coreutils (`ls` , `date` , `touch` etc(`mount` is not in here- who knew)
24-
- To get `mount` you need to build util-linux https://en.wikipedia.org/wiki/Util-linux
25-
- is `ls` failing, did you forget to include `/usr/lib/x86_64-linux-gnu/libcap.so.2`? See `mkchroot.sh` helper from https://landley.net/writing/rootfs-programming.html
26-
27-
- Don't forget to `mount` linux virtual filesystems (oh wait, did you forget to build util-linux into your init?
28-
29-
Example built rootfs:
30-
```
31-
ls -lh rootfs.cpio.gz
32-
-rw-rw-r-- 1 chris chris 16M Dec 8 23:40 rootfs.cpio.gz
33-
```
25+
See [./build-all.sh](./build-all.sh)
3426

3527
# What does this repo not include (yet)
3628

37-
- Full clone/compile instructions
3829
- Automated ci
3930

40-
### How do I build statically coreutils, do I even need to?
31+
<strike>### How do I build statically coreutils, do I even need to?
4132

42-
See https://lists.gnu.org/archive/html/coreutils/2019-04/msg00001.html
33+
See https://lists.gnu.org/archive/html/coreutils/2019-04/msg00001.html </strike> switched to using `musl`.
4334

44-
## `ls` , `cat` and `date` etc won't run without libc!
45-
46-
git clone https://sourceware.org/git/glibc.git
4735

4836
# Reading
4937
See also
@@ -52,3 +40,17 @@ https://wiki.gentoo.org/wiki/Custom_Initramfs
5240
https://unix.stackexchange.com/a/305406
5341
https://landley.net/writing/rootfs-howto.html
5442
https://landley.net/writing/rootfs-programming.html
43+
- https://unix.stackexchange.com/questions/193066/how-to-unlock-account-for-public-key-ssh-authorization-but-not-for-password-aut
44+
- https://stackoverflow.com/a/79151188
45+
- https://z49x2vmq.github.io/2020/12/24/linux-tiny-qemu/
46+
47+
> "Stuff like this is slowly becoming a lost art" [src](https://www.linuxquestions.org/questions/linux-general-1/bin-bash-as-primary-init-4175543547/#post5367386) ooopse.
48+
49+
50+
TODO: kernel inital ram disk support https://stackoverflow.com/questions/14430551/qemu-boot-error-swapper-used-greatest-stack-depth
51+
TODO READ: https://bbs.archlinux.org/viewtopic.php?pid=1378903#p1378903
52+
53+
## Notes
54+
55+
> "busybox qemu /bin/sh: can't access tty; job control turned off"
56+
> https://github.com/brgl/busybox/blob/master/shell/cttyhack.c

‎build-all.sh

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,42 @@
22

33
set -exu
44

5+
# We will
6+
#
7+
# - Empty ./build-dir (to build from scratch)
8+
# - Build linux kernel
9+
# - Build busybox
10+
# - Build musl
11+
# - Build openssh statically (using musl)
12+
# - iproute2 is built into busybox
13+
14+
SCRIPT_START_DIR=$PWD
15+
BUILD_DIR=$(realpath -s ./build-dir)
16+
rm -rf "$BUILD_DIR"
17+
mkdir -p "$BUILD_DIR"/build-artifacts
18+
BUILD_ARTIFACTS_DIR="$BUILD_DIR"/build-artifacts
19+
20+
cd "$BUILD_DIR"
21+
22+
# Build linux kernel
23+
BUILD_ARTIFACTS_DIR=$BUILD_ARTIFACTS_DIR ../build-kernel.sh
24+
25+
cd "$BUILD_DIR"
26+
# Build busybox
27+
BUILD_ARTIFACTS_DIR=$BUILD_ARTIFACTS_DIR ../build-busybox.sh
28+
29+
# Build musl
30+
../build-musl.sh
31+
cd "$BUILD_DIR"
32+
33+
# Build openssh
34+
BUILD_ARTIFACTS_DIR=$BUILD_ARTIFACTS_DIR ../build-openssh-statically.sh
35+
cd "$BUILD_DIR"
36+
37+
cd "$SCRIPT_START_DIR"
38+
539
./create-init.sh
640
./build-rootfs.sh
41+
42+
echo If all is wel, you\'re now good to run the vm in qemu
43+
echo see ./run-qemu.sh

‎build-busybox.sh

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/bin/bash
2+
3+
set -eux
4+
5+
INITAL_WORKING_DIR=$PWD
6+
7+
git clone git://git.busybox.net/busybox
8+
9+
cd busybox
10+
git checkout 1_37_0
11+
make defconfig
12+
echo CONFIG_STATIC=y >> .config
13+
sed -i 's/CONFIG_TC=y/# CONFIG_TC is not set/g' .config
14+
make -j$(nproc)
15+
16+
mkdir "$BUILD_ARTIFACTS_DIR"/busybox
17+
cp ./busybox "$BUILD_ARTIFACTS_DIR"/busybox
18+
19+
cd $INITAL_WORKING_DIR

‎build-coreutils.sh

Lines changed: 0 additions & 15 deletions
This file was deleted.

‎build-kernel.sh

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/bin/bash
2+
3+
set -eux
4+
5+
INITAL_WORKING_DIR=$PWD
6+
7+
mkdir linux-kernel
8+
cd linux-kernel
9+
wget https://www.kernel.org/pub/linux/kernel/v6.x/linux-6.9.tar.xz
10+
11+
tar xf linux-6.9.tar.xz
12+
cd linux-6.9
13+
mkdir -p $BUILD_ARTIFACTS_DIR/linux-kernel
14+
make defconfig
15+
make -j$(nproc)
16+
cp ./arch/x86_64/boot/bzImage $BUILD_ARTIFACTS_DIR/linux-kernel
17+
18+
19+
cd $INITAL_WORKING_DIR

‎build-libc.sh

Lines changed: 0 additions & 22 deletions
This file was deleted.

‎build-musl.sh

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/bash
2+
# Via https://wiki.musl-libc.org/getting-started.html
3+
4+
set -xu
5+
6+
INITAL_WORKING_DIR=$PWD
7+
8+
git clone git://git.musl-libc.org/musl
9+
cd musl/
10+
git checkout v1.2.5
11+
./configure --prefix=$HOME/musl --exec-prefix=$HOME/bin --syslibdir=$HOME/musl/lib --disable-shared
12+
13+
make
14+
make install
15+
16+
17+
cd $INITAL_WORKING_DIR

‎build-openssh-statically.sh

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#!/bin/bash
2+
3+
# 1. clone openssh-portable
4+
git clone https://github.com/openssh/openssh-portable
5+
cd openssh-portable
6+
git checkout V_9_9_P1
7+
autoconf
8+
9+
# Note this uses only the **experimental** internal (reduced) cryp algos
10+
# built-into openssh. TODO actuall include libcrypto
11+
CC="musl-gcc -static" ./configure --prefix=/usr/bin --sysconfdir=/etc/ssh --without-zlib --without-openssl
12+
make -j$(nproc)
13+
14+
# Copy over openssh binaries to build artifacts dir
15+
mkdir -p $BUILD_ARTIFACTS_DIR/openssh
16+
17+
echo $PWD
18+
19+
for sshUtility in $(find ./ -maxdepth 1 -type f -executable | grep -E -v '(\.sh|\.in|\.rc|\.sub|\.sample|\.status|\.guess|configure|fixpaths|install-sh|mkinstalldirs|fixalgorithms)'); do
20+
echo Copying over "$sshUtility"
21+
cp "$sshUtility" $BUILD_ARTIFACTS_DIR/openssh
22+
done
23+
24+
cp $(find ./ -name sshd_config) $BUILD_ARTIFACTS_DIR/openssh
25+

‎create-init.sh

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,38 @@
1-
#!/bin/bash
1+
#!/bin/sh
22

33
set -eux
44

5-
echo "#!/usr/bin/bash" > scratch-space/init
6-
echo "exec /usr/bin/bash" >> scratch-space/init
7-
chmod +x scratch-space/init
5+
INIT_FILE_PATH=scratch-space/init
6+
7+
echo "#!/bin/busybox sh" > $INIT_FILE_PATH
8+
echo 'echo YOLOOooooooooooooooooooooooooooo' >> $INIT_FILE_PATH
9+
echo 'echo YOLOOooooooooooooooooooooooooooo' >> $INIT_FILE_PATH
10+
echo 'echo YOLOOooooooooooooooooooooooooooo' >> $INIT_FILE_PATH
11+
echo 'echo YOLOOooooooooooooooooooooooooooo' >> $INIT_FILE_PATH
12+
echo 'echo YOLOOooooooooooooooooooooooooooo' >> $INIT_FILE_PATH
13+
echo 'mount -t sysfs sysfs /sys' >> $INIT_FILE_PATH
14+
echo 'mount -t proc proc /proc' >> $INIT_FILE_PATH
15+
# (sshd needs openpty: No such file or directory )
16+
echo 'mount -t devtmpfs udev /dev' >> $INIT_FILE_PATH
17+
echo 'mkdir /dev/pts' >> $INIT_FILE_PATH
18+
echo 'mount -t devpts devpts /dev/pts' >> $INIT_FILE_PATH
19+
echo 'sysctl -w kernel.printk="2 4 1 7"' >> $INIT_FILE_PATH
20+
echo 'chown -R root:root /var/empty' >> $INIT_FILE_PATH
21+
echo 'chmod -R 400 /var/empty' >> $INIT_FILE_PATH
22+
23+
echo 'echo Bringing up loopback interface' >> $INIT_FILE_PATH
24+
echo 'ip link set lo up' >> $INIT_FILE_PATH
25+
echo 'ip addr show lo' >> $INIT_FILE_PATH
26+
27+
echo 'echo Generating ssh host keys' >> $INIT_FILE_PATH
28+
echo 'ssh-keygen -A' >> $INIT_FILE_PATH
29+
echo 'ls -l /etc/ssh' >> $INIT_FILE_PATH
30+
31+
echo 'echo Starting sshd' >> $INIT_FILE_PATH
32+
echo '/usr/bin/sshd -E ssh_log' >> $INIT_FILE_PATH
33+
34+
# Curious? See https://github.com/brgl/busybox/blob/master/shell/cttyhack.c
35+
echo 'setsid cttyhack /bin/sh' >> $INIT_FILE_PATH
36+
37+
chmod +x $INIT_FILE_PATH
838

0 commit comments

Comments
 (0)