Posted by astralbijection 17 hours ago
However a fun fact is that you can (do not actually do this!) boot a qemu VM from /dev/sda. You have to use an overlay (eg. qemu -drive snapshot=on flag) so that qemu won't write through to /dev/sda. I use this trick in supernested, a script I wrote that runs nested within nested within nested VMs ad infinitum until your hypervisor crashes. http://git.annexia.org/?p=supernested.git;a=blob;f=run-super...
It's not a real dual boot if you don't boot both partitions at the same time.
As long as you don't install guest VBox drivers, those would make it hang when it boots as the host on physical hardware, since there's no longer someone above to answer the hypercalls.
I had no idea that this was possible, and I learned something new today. Thank you!
Without spoiling too much, the command at the very end of the series does something adjacent to this.
On the topic itself: wow, what a journey. And I personally fully support "come on, you should totally be able to just dump the system image onto your disk and reboot/exec it!"
Taking control at the initrd stage, as in the second page of the article, is significantly more reliable.
But have busybox in your initrd so you don't have to suffer. It takes up 0.5% of the size of my initrd file.
Or if you have access to the boot command line you can also usually stop the boot process before pivot_root happens (hence you’ll be left running in the initramfs environment)
On Fedora/EL it would be done by putting `rd.break` in the kernel command line
[0]: https://www.man7.org/linux/man-pages/man8/xfs_freeze.8.html
unmount the efi and boot partitions, write your image to the head of the disk, power cycle, then grow the last filesystem from the image to cover the rest of the disk.
you might get lucky and have all three of uefi/boot/swap to work with.
of course with the advent of uefi, you could instead just drop an installer image directly into the efi parition and boot that.
I went down a similar rabbit-hole myself, with the goal of safely replacing the Linux installation on a disk that a machine is already running from (e.g. replace a VPS's setup image with one of your own) without needing a KVM-style remote access tool to the console.
The problem there is if you directly modify the disk when a filesystem is mounted on that disk then all bets are off in terms of corruption of the filesystem that's already on there and also the filesystem(s) you're writing over the top.
My solution was to kexec into a new kernel+initramfs which has a DHCP client and cURL in it - that effectively stops any filesystem access while the image is being written over the disk, then to just reboot.
That's what I was expecting from the article.
Update: It's not obvious, but it turns out that this is a multipart article, and kexec is reserved for part 3: https://astrid.tech/2026/03/24/2/how-to-pass-secrets-between...
https://www.kernel.org/doc/html/latest/admin-guide/sysrq.htm...
It's technically not an unmount, but still a pretty strong guarantee OS will not corrupt the image being written.
When done, reboot has to be done from the same sysrq handler, of course.
We had a big drive with the source of truth image used to boot all our machines on it, and we added rsync to the init image. When each machine booted init would rsync everything from the storage box to the local machine. We'd keep the storage machine up to date and when we wanted to update other machines in the fleet we'd just do a reboot and it would sync up the latest files (provisioning for whatever each machine was supposed to do happened later, can't remember how that was handled now). The storage machine was running ZFS so we also took a snapshot before doing any rolling reboots, so if anything did go wrong you could just revert to the previous snapshot and reboot again as long as you didn't break the init image.
Sounds jank saying it out loud, but I don't remember it ever causing us any problems.
QEMU defaults to 512B sectors, which isn't true for many NVMe drives. There are some flags to change that. https://unix.stackexchange.com/a/722450
I think it should be possible to make an image with many headers at different locations, so that it works on all types of disks at once, but I don't think any tools do it for you by default.
> The OS may stop you from unmounting /dev/sda1, but it won’t stop you from writing to /dev/sda1 or /dev/sda even if there’s something mounted!
Not always true. There's a kernel config option that allows it. CONFIG_BLK_DEV_WRITE_MOUNTED
After waiting for a little while, the program terminated with the following output:
astrid@chungus infra gzip -vc result/nixos.img | ssh root@myhost.example -- bash -c 'gunzip -vc > /dev/sda'
root@myhost.example's password:
77.8% -- replaced with stdout
What happened here?
I do like this idea - with sufficient prep of the system before writing the image, namely stopping as many processes as possible especially those that might do some writing, it's a quick and dirty way to replace a stock OS with a ready-made image. Could perhaps be safer doing it twice, once into a minimal image that does very little beyond network bringup & runs ssh, followed by final OS replacement in a (more) controlled manner.
They'd netboot.. not mount the disks, then download an ISO/IMG and write it directly to the primary boot disk.
If netbooting is a heavy lift, why not boot into a custom initramfs you built, with i.e. dd/curl installed, and flash the disk that way, without mounting / at all? Then kexec/chroot into it?
I'd much prefer this as a way to provision Raspberry Pis.
So ending up at:
> From a 292MB initramfs, we now have a 6.1MB initramfs, smaller than almost every other distro's initramfs and made entirely to run busybox wget dd.
Is pretty great achievement today - but way bigger than something that can fit on a floppy.
Stupid penguin trick I learned: Add a file inside ramdisk (i use /dev/shm) as LVM PV.
pvmove off the hard drive
Boom, now your OS lives entirely in RAM
You can now even replace the hard disk, put a new one and migrate back.
Or migrate to network storage (nbd,iSCSI etc.), re-sequence disks into whatever RAID you need, and migrate back
Need to fix /boot after that tho, and probably make sure to not have power failure in meantime
Also, I once burned an iso straight from ftp using a fifo. I was low on disk space and really needed that CD. Worked fine because the Internet was already faster than the CDR.