Rescuing Linux - revision history

Dank: /* moving a root filesystem + bootloader */

2026-02-04T10:51:19Z

moving a root filesystem + bootloader

← Older revision		Revision as of 10:51, 4 February 2026
Line 95:		Line 95:

	<b>ESP, bootloader, initramfs...</b>		<b>ESP, bootloader, initramfs...</b>

			==reassembling a degraded MDRAID==
			...

	==External links==		==External links==
	* Debian wiki page for [https://wiki.debian.org/initramfs initramfs]		* Debian wiki page for [https://wiki.debian.org/initramfs initramfs]
	* Kernel documentation for [https://www.kernel.org/doc/html/latest/filesystems/ramfs-rootfs-initramfs.html Ramfs, Rootfs, and Initramfs]		* Kernel documentation for [https://www.kernel.org/doc/html/latest/filesystems/ramfs-rootfs-initramfs.html Ramfs, Rootfs, and Initramfs]

Dank: /* kernel command line */

2026-02-04T10:50:13Z

kernel command line

← Older revision		Revision as of 10:50, 4 February 2026
Line 18:		Line 18:

	==kernel command line==		==kernel command line==
	The bootloader can provide command line parameters to the kernel (they can also be specified at build time, see <tt>CONFIG_CMDLINE</tt>). [[systemd]] can take many parameters off of the kernel command line.		The bootloader can provide command line parameters to the kernel (they can also be specified at build time, see <tt>CONFIG_CMDLINE</tt>). GRUB supports providing them in interactive mode. With UEFI firmware, they can be specified in the EFI shell.

			[[systemd]] can take many parameters off of the kernel command line, including <tt>systemd.unit=</tt> to specify a boot target.

	For more output, ensure "quiet" is not present. For still more output, add "debug=vc" (by default, <tt>debug</tt> writes to <tt>/run/initramfs/initramfs.debug</tt>; this sends it to the console).		For more output, ensure "quiet" is not present. For still more output, add "debug=vc" (by default, <tt>debug</tt> writes to <tt>/run/initramfs/initramfs.debug</tt>; this sends it to the console).
Line 25:		Line 27:

	Note that the kernel does not by default reboot following a panic. This can be undesirable on remote machines. <tt>panic=N</tt> will reboot N seconds after a panic, if N is positive. <tt>/proc/sys/kernel/panic</tt> exposes this.		Note that the kernel does not by default reboot following a panic. This can be undesirable on remote machines. <tt>panic=N</tt> will reboot N seconds after a panic, if N is positive. <tt>/proc/sys/kernel/panic</tt> exposes this.

			Modules can be blacklisted with <tt>modprobe.blacklist=</tt>.

	===my video is borked===		===my video is borked===

Dank: /* firmware */

2026-02-04T10:47:56Z

firmware

← Older revision		Revision as of 10:47, 4 February 2026
Line 2:		Line 2:

	==firmware==		==firmware==
	System firmware configuration can usually be entered by holding one of Delete, F2, or F1. A boot menu can commonly be accessed with F10.		System firmware configuration can usually be entered by holding one of Delete, F2, or F1. A boot menu can commonly be accessed with F10. All of these are of course motherboard-specific.

	If you need clear a firmware password, try removing the battery and unplugging the machine for 30s.		If you need clear a firmware password, try removing the battery and unplugging the machine for 30s.

			<tt>systemctl reboot --firmware-setup</tt> will attempt to reboot into firmware configuration.

	==grub==		==grub==

Dank: /* fsck on boot */

2026-02-04T10:46:29Z

fsck on boot

← Older revision		Revision as of 10:46, 4 February 2026
Line 47:		Line 47:

	==fsck on boot==		==fsck on boot==
			this doesn't happen much anymore since most everyone's running at least ext3, but ext2 required <tt>fsck</tt> on boot when not cleanly unmounted, and every so often just to make sure things hadn't shifted around.

			if you don't want a filesystem to block booting under [[systemd]], use <tt>nofail</tt> in <tt>/etc/fstab</tt>.

	==access sans password==		==access sans password==

Dank: /* initramfs can't mount root */

2026-02-04T10:38:19Z

initramfs can't mount root

← Older revision		Revision as of 10:38, 4 February 2026
Line 43:		Line 43:
	===initramfs can't mount root===		===initramfs can't mount root===
	An unpleasant situation is one where initramfs fails to mount the root partition, in which case you will be dumped to the dreaded BusyBox or klibc shells (ash, as in "a shitty shell"). A day when one sees BusyBox is never a good day. If the machine is remote, you are fucked without server-style out of band access (e.g. Dell iDRAC, BMC, KVM-over-IP). Otherwise, if you have a valid root partition somewhere, you can manually continue the boot by mounting that partition to <tt>/mnt/root</tt> and running <tt>exec switch_root /mnt/root /sbin/init</tt> or its non-union equivalent. Usually this means you've specified the wrong root partition in your bootloader; check the <tt>root</tt> command line option to the kernel.		An unpleasant situation is one where initramfs fails to mount the root partition, in which case you will be dumped to the dreaded BusyBox or klibc shells (ash, as in "a shitty shell"). A day when one sees BusyBox is never a good day. If the machine is remote, you are fucked without server-style out of band access (e.g. Dell iDRAC, BMC, KVM-over-IP). Otherwise, if you have a valid root partition somewhere, you can manually continue the boot by mounting that partition to <tt>/mnt/root</tt> and running <tt>exec switch_root /mnt/root /sbin/init</tt> or its non-union equivalent. Usually this means you've specified the wrong root partition in your bootloader; check the <tt>root</tt> command line option to the kernel.

			If you have [[systemd]] running in the emergency shell (its <tt>emergency.target</tt>), try manually mounting the root filesystem to <tt>/sysroot</tt> and then running <tt>systemctl restart mount-sysroot</tt>. If this succeeds, exiting the shell ought continue the boot.

	==fsck on boot==		==fsck on boot==

Dank: /* firmware */

2024-04-02T06:15:02Z

firmware

← Older revision		Revision as of 06:15, 2 April 2024
Line 3:		Line 3:
	==firmware==		==firmware==
	System firmware configuration can usually be entered by holding one of Delete, F2, or F1. A boot menu can commonly be accessed with F10.		System firmware configuration can usually be entered by holding one of Delete, F2, or F1. A boot menu can commonly be accessed with F10.

			If you need clear a firmware password, try removing the battery and unplugging the machine for 30s.

	==grub==		==grub==

Dank at 06:14, 2 April 2024

2024-04-02T06:14:10Z

← Older revision		Revision as of 06:14, 2 April 2024
Line 1:		Line 1:
	Sometimes you need perform irregular, unpleasant tasks involving a Linux box. One generally picks up the techniques via experience. This kind of thing tends to become less necessary as one learns Linux better, and thus it's easy to lose currency. The introduction of [[systemd]], for instance, changed almost all of this shit up. Almost everything here requires access to the console, relevant only long before sshd is running (though take a look at e.g. [https://wiki.debian.org/RescueInitramfs RescueInitramfs]).		Sometimes you need perform irregular, unpleasant tasks involving a Linux box. One generally picks up the techniques via experience. This kind of thing tends to become less necessary as one learns Linux better, and thus it's easy to lose currency. The introduction of [[systemd]], for instance, changed almost all of this shit up. Almost everything here requires access to the console, relevant only long before sshd is running (though take a look at e.g. [https://wiki.debian.org/RescueInitramfs RescueInitramfs]).

			==firmware==
			System firmware configuration can usually be entered by holding one of Delete, F2, or F1. A boot menu can commonly be accessed with F10.

	==grub==		==grub==

Dank: /* systemd emergency mode with a locked root account */

2024-04-02T03:22:16Z

systemd emergency mode with a locked root account

← Older revision		Revision as of 03:22, 2 April 2024
Line 46:		Line 46:
	<b>what if we can't use interactive mode?</b>		<b>what if we can't use interactive mode?</b>
	===systemd emergency mode with a locked root account===		===systemd emergency mode with a locked root account===
	If you've locked the root account with <tt>passwd -l</tt>, systemd's <tt>sulogin</tt> won't let you enter rescue mode. Use the instructions from "access sans password" above to get a shell if possible, and enable root with <tt>passwd</tt>. Reboot, and <tt>sulogin</tt> will admit you using the new ~~passwork~~.		If you've locked the root account with <tt>passwd -l</tt>, systemd's <tt>sulogin</tt> won't let you enter rescue mode. Use the instructions from "access sans password" above to get a shell if possible, and enable root with <tt>passwd</tt>. Reboot, and <tt>sulogin</tt> will admit you using the new password.

	==all my binaries are gone==		==all my binaries are gone==

Dank: /* grub */

2024-04-01T06:03:05Z

grub

← Older revision		Revision as of 06:03, 1 April 2024
Line 7:		Line 7:

	Do yourself a favor and install [https://packages.debian.org/sid/memtest86+ memtest86+], which will be automatically added to your grub menu on [[UEFI]] machines.		Do yourself a favor and install [https://packages.debian.org/sid/memtest86+ memtest86+], which will be automatically added to your grub menu on [[UEFI]] machines.

			If you're in the interactive GRUB command prompt, and wish not to be (i.e. to return to the menu), use the command <tt>normal</tt>.

	==kernel command line==		==kernel command line==

Dank: /* grub */

2024-04-01T06:00:45Z

grub

← Older revision		Revision as of 06:00, 1 April 2024
Line 2:		Line 2:

	==grub==		==grub==
	If there is no boot delay, but you need interactive control, try holding down space when grub comes up.		If there is no boot delay, but you need interactive control, try holding down space (or with some versions shift) when grub comes up.

	Don't go editing grub's configuration files directly if at all possible. Edit <tt>/etc/default/grub</tt> and rebuild them with <tt>update-grub</tt>. Note that <tt>update-grub2</tt> is these days just a symlink to <tt>update-grub</tt>. If what you want to change isn't available via <tt>/etc/default/grub</tt>, try to do it via <tt>/etc/grub.d</tt>. If you're directly editing <tt>grub.cfg</tt>, you're gonna have a bad time (and your changes will be blown away the next time someone runs <tt>update-grub</tt>).		Don't go editing grub's configuration files directly if at all possible. Edit <tt>/etc/default/grub</tt> and rebuild them with <tt>update-grub</tt>. Note that <tt>update-grub2</tt> is these days just a symlink to <tt>update-grub</tt>. If what you want to change isn't available via <tt>/etc/default/grub</tt>, try to do it via <tt>/etc/grub.d</tt>. If you're directly editing <tt>grub.cfg</tt>, you're gonna have a bad time (and your changes will be blown away the next time someone runs <tt>update-grub</tt>).