Kernel

OS in Rust

Announcements

  • Action Items:
    • Is qemu working at all.
      • The coolest assignment ever for a second week in a row.
      • I’m glad you all love it

Today

  • Background
  • Kernel

Citations

Background

The Boot Process

  • POV: You are an inanimate piece of silicon.
    • You contain wires connected to logic gates.
    • Somewhere, a switch is flipped.
    • Electrons flow into some of your wires, through some gates.

First Steps

  • What determines the initial arrangement of gates?
  • Where do electrons flow?
  • This is determined by the boot process
    • Occurs on every power-up
    • Determined by hardware design
    • More fundamental than the OS

Firmware

  • What is between hardware and software?
    • Firmware.
  • On power-up, devices execute code embedded in physical read only memory (ROM).
    • Read more: ROM
  • Is it software? Is it hardware? Who can say.

Enter the CPU

  • Usually, power-up occurs on a “motherboard” hosting, among other things, the bus.
    • Named “mother” after the “MU/TH/UR” on ship computer in Alien (1979)
    • This is a lie.

Not “code” but “ware”

  • So firmware isn’t really like CPU code (like Rust or C), but it does:
    • Tell circuitry where to direct electrons within the CPU.
    • Also wake up e.g. the MMU.

Enter the OS

  • With the CPU primed but not yet ticking through clock cycles, all that remains is to either
    • Execute a bare metal executable, or
    • Boot an operating system to enable the next higher-level task.

What it sounds like

  • We regard, then, the operating system as a system which operates the hardware on behalf of higher level software.
    • Hence, “systems computing”.
    • Hopefully the contrast to software is a bit more clear here.

Aside

  • I am supposed to teach you about the “power-on self-test”.
  • A bit electrical engineering for me.

A power-on self-test (POST) is a process performed by firmware or software routines immediately after a computer or other digital electronic device is powered on.

  • Neat! Moving on.

Competing Standards

  • Lucky us, there is no widely agreed upon way to do firmware.
  • There’s the cool, old way that doesn’t work well but is easy.
    • “Basic Input/Output System“ BIOS
    • 1981

Competing Standards

  • There’s the new way that is too hard to use for normal people like us.
    • As in people who do anything else ever.
    • It’s good though we promise.
    • “Unified Extensible Firmware Interface” UEFI.
    • May make you use it for a lab. More.
    • 2006

BIOS Boot

  • I actually started fooling around with the BIOS before UEFI even existed.
    • I was a normal kid though 100%.
  • Fortunately it’s still basically around. Quoth Blog:

This is great, because you can use the same boot logic across all machines from the last century.

Upsides

  • Blog says it’s a downside that this means you have to do 16-bit mode.
    • I say: that’s cool.
    • I can’t count higher than about 0xFFFF anyways.
  • The blog impolitely calls 1980s bootloaders “archaic” instead of “vintage”, “retro”, or “foundational”.

Bootable Disks

  • I should also tell you about bootable disks
  • Nowadays we all boot from SSD or rarely HDD.
  • But you have probably at some point booted from USB.
    • Usually when removing a virus like Microsoft Windows from your system.
  • Olden days computers could boot from floppy disks, etc.

Bootlaoder

  • I mentioned 1980s bootloaders.
    • No relation to bootleggers or boatloaders.
  • 512-byte portion of executable code stored at the bootable disk’s “beginning”.
    • On a HDD this is physically the outermost ring of addressable magnetic regions.
    • I don’t know how SSDs work as Samsung, SK Hynix, or Micron (shout out Boise).

Data Structures

  • Most bootloaders are larger than 512 bytes.
  • So bootloaders are commonly split into a 512 byte first stage that loads a latter stage.
    • This is why we should still be teaching linked lists, basically.

Location, Location

  • The bootloader lives in a reserved physical (HDD) or logical (SSD) location.
  • Does the OS?
    • With respect to itself, yes, the OS probably says it lives at memory location zero.
    • With respect to underlying hardware? Probably not.
      • Gotta find it.

Booting the OS

  • The bootloader has to determine the location of the kernel image on disk and load it into memory.
    • Basically this is the definition of the kernel, the minimal OS internal that runs first.
    • Image here means we have physical bits capturing some information, so copies of the bits may live in different places.
      • SSD and RAM, for example.

Switcheroo

  • The OS probably is not a 16-bit OS.
    • Unless? Lab idea? Hold me back!
  • Big OS wants me to tell you that:
    • 16-bit mode is called “real mode”
    • 32-bit mode is called “protected mode”
    • 64-bit mode is called “long mode”.
  • Recall we were writing 64-bit bare metal.

Hand-wave

Writing a bootloader is a bit cumbersome as it requires assembly language and “write this magic value to this processor register”.

Instead use a bootimage that automatically prepends a bootloader to your kernel.

  • This is called “cheating” and is a good way to get ahead in life.

Kernel

A Minimal Kernel

  • Let’s make a kernel.
  • Specifically, a disk image that prints a “Hello World!” to the screen when booted.
  • We extending our bare metal executable.

Target Triple

  • We recall the “target triple
  • Imagine host triple is x86_64-unknown-linux-gnu
    • CPU architecture (x86_64),
    • Vendor (unknown) - It’s Intel #Portland
    • Operating system (linux)
    • The ABI (gnu).

Our Target

  • I am aware of no existing target triple suitable for this course.
  • So, make our own.
  • It’s not too bad, just JSON.
    • We’ll specify some easy stuff, like architecture.
    • Some weird stuff, like manual memory layouts.
    • And get on with things.

JSON

  • JSON rules by the way.
{
    "llvm-target": "x86_64-unknown-linux-gnu",
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
    "arch": "x86_64",
    "target-endian": "little",
    "target-pointer-width": 64,
    "target-c-int-width": 32,
    "os": "linux",
    "executables": true,
    "linker-flavor": "gcc",
    "pre-link-args": ["-m64"],
    "morestack": false
}

Some context

  • Most fields are required by LLVM.
  • Data layout field defines the size of integer, float (ew), and pointer types.
  • Rust uses conditional compilation, such as via target-pointer-width.
  • The pre-link-args field specifies arguments passed to the linker.

ARM64 Take Notes

  • We also target x86_64.
  • Start here:
x86_64-osirs.json
{
    "llvm-target": "x86_64-unknown-none",
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
    "arch": "x86_64",
    "target-endian": "little",
    "target-pointer-width": 64,
    "target-c-int-width": 32,
    "os": "none",
    "executables": true
}

Changes

  • Note that we changed the OS in the llvm-target and the os field to none, because we will run on bare metal.
x86_64-osirs.json
{
    "llvm-target": "x86_64-unknown-none",
    ...
    "os": "none",
    ...
}

Linking

  • We’ll add the following build-related entries:
x86_64-osirs.json
{
    "linker-flavor": "ld.lld",
    "linker": "rust-lld",
}
  • For OS-agnosticism, we use the cross-platform “LLD” linker that is shipped with Rust for linking our kernel.

Panic Abort

  • You know how I feel about unwinding.
    • I have never relaxed in my life.
    • I’ve only panicked and given up.
x86_64-osirs.json
{
    "panic-strategy": "abort",
}

Target vs. TOML

Red Zone

  • Okay red zone is not particularly relevant to this class.
  • But it is extremely cool.
x86_64-osirs.json
{
    "disable-redzone": true,
}

Ancient Nemesis

  • You all know I love floating point numbers.
    • And with good reason!
x86_64-osirs.json
{
    "features": "-mmx,-sse,+soft-float",
}

Turn off floats

  • features enables/disables target features.
  • We disable the mmx and sse features by prefixing them with a minus
  • We enable the soft-float feature by prefixing it with a plus.
  • Note that there must be no spaces between different flags!

MMX/SSE

  • The mmx and sse features are performance optimizing vector operations from when Intel though they’d be able to hold off NVIDIA in the 90s.
  • These are braodly called Single Instruction Multiple Data (SIMD) instructions and are historically important.
    • Foundation of numpy
  • We aren’t using data frames in our kernel.

Soft Float

  • Floating point operations on x86_64 require SIMD registers by default.
    • That’s right - floats are worse than you thought!
  • To solve this problem, we add the soft-float feature, which emulates all floating point operations through software functions based on normal integers.
    • Just like f16

Aside

  • We also need to tell the Rust compiler rustc that we want to use the corresponding ABI.
x86_64-osirs.json
{
    "rustc-abi": "x86-softfloat"
}
  • I am 100% sure I can write an OS without hard or soft floats but I haven’t worked far enough ahead to be absolutely certain.

Altogether

x86_64-osirs.json
{
    "llvm-target": "x86_64-unknown-none",
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
    "arch": "x86_64",
    "target-endian": "little",
    "target-pointer-width": 64,
    "target-c-int-width": 32,
    "os": "none",
    "executables": true,
    "linker-flavor": "ld.lld",
    "linker": "rust-lld",
    "panic-strategy": "abort",
    "disable-redzone": true,
    "features": "-mmx,-sse,+soft-float",
    "rustc-abi": "x86-softfloat"
}

I just curl

curl https://raw.githubusercontent.com/phil-opp/blog_os/refs/heads/post-02/x86_64-blog_os.json -o x86_64-osirs.json

Tree

  • For me looks like this.
$ tree
.
├── Cargo.lock
├── Cargo.toml
├── src
│   ├── main.rs
│   └── old.rs
└── x86_64-osirs.json

1 directory, 5 files

Back to loops

  • By the way, I’ve switched from cool, good recursion back to unexciting, drab loops
  • Infinite recursion blows up the call stack and instantly segmentation faults.
    • This is because someone other than me wrote rustc.
    • I am not about to write rustc!

Current main

src/main.rs
#![no_std]
#![no_main]

#[unsafe(no_mangle)]
pub extern "C" fn _start() -> ! {
    loop {}
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    loop {}
}

A note

  • Remember the earlier mention of core
    • How we need core to also panic abort?
    • We note when looking at src/main.rs we do have a core reference.

Linux Everywhere

  • Okay so we aren’t going to use Linux on our device.
  • But we are going to use Linux conventions
    • Not Linux software, but Linux as a social technology.
  • The ld.lld “linker-flavor” instructs LLVM to compile with the -flavor gnu flag.
  • This means that we need an entry point named _start - same as before!

Build it

  • I bet it will work now.
    • Use our new target by passing the name of the JSON file as --target:
$ cargo b --target x86_64-osirs.json
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/user/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc - --crate-name ___ --print=file-names --target /home/user/tmp/32/x86_64-osirs.json --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg -Wwarnings` (exit status: 1)
  --- stderr
  error: Error loading target specification: Field target-pointer-width in target specification is required. Run `rustc --print target-list` for a list of built-in targets

Aside

  • If you did not see that error, that is okay!
  • It concerns an unstable language feature and may differ.
  • Just skip two slides!

Okay so

  • Huh?
Field target-pointer-width in target specification is required.
  • We very explicitly included that.
  • In the most annoying thing in the universe, rustc expect pointer width as a JSON string and not a JSON integer.
    • Read more
    • It’s fixed in Nightly, but I sleep at night, so I’m busy.

Fix it?

  • I kid you not this is the solution.
    • “Rust has a type system!”
    • Sure it does.
$ diff bad.wrong x86_64-osirs.json
6,7c6,7
<     "target-pointer-width": 64,
<     "target-c-int-width": 32,
---
>     "target-pointer-width": "64",
>     "target-c-int-width": "32",

Now it works

  • This time it will work.
$ cargo b --target x86_64-osirs.json
   Compiling osirs v0.1.0 (/home/user/tmp/32)
error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-osirs` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-osirs`

For more information about this error, try `rustc --explain E0463`.
error: could not compile `osirs` (bin "osirs") due to 1 previous error
  • Okay I was bamboozled.

The Core

Wrong Core

  • We actually meant the Rust compiler core library.
  • This library contains basic Rust types such as Result, Option, and iterators, and is implicitly linked to all no_std crates.

The Problem

  • core is usually pre-compiled (and then, of course, linked).
  • But we made a new target which needs a new core.
  • No problem, we’ll just tell rustc to do some compilation.

Config

  • The most graceful to do this that I am aware of is with a .cargo/config.toml
  • Basically, we can write down some things we always want cargo to do, and store them in TOML file in the hidden .cargo folder.
  • I just made the folder then opened it up in my most beloved neon vimothy.
mdkir .cargo
nvim .cargo/config.toml

Check in

mdkir .cargo
nvim .cargo/config.toml
  • Understanding check - what happens if you don’t make .cargo first?

The build-std Option

  • build-std is a feature of Cargo.
  • We can recompile core and other standard library crates on demand.
    • Vs. using the precompiled versions shipped with the Rust installation.
  • Read more.

My Config

  • We can use a pretty sparse config though we’ll want to add more latter.
  • I specify the “build-std” option in TOML.
  • We want to build-std
  • We want to build the core
.cargo/config.toml
build-std = ["core"]
  • I bet this will work.

Oh.

  • The exact same error as before?
$ cargo b --target x86_64-osirs.json
   Compiling osirs v0.1.0 (/home/user/tmp/32)
error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-osirs` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-osirs`

For more information about this error, try `rustc --explain E0463`.
error: could not compile `osirs` (bin "osirs") due to 1 previous error

Unstable

  • So apparently build-std is not a stable feature of the Rust language.
    • This means the Rust designers can change or remove it at any time.
  • To use unstable features, we have to tell cargo they are unstable.

Mental model

  • Think of it a bit like unsafe, but for the language instead of the executables.
    • When an executable is unsafe, it may crash or leak your private key.
    • When a language is unstable, it may not compile or may compile then leak your private key.

Update config

  • We prepend an [unstable] label to our build-std configuration.
.cargo/config.toml
[unstable]
build-std = ["core"]
  • I bet it will work now (it won’t).

Oh.

  • The exact same error as before?
$ cargo b --target x86_64-osirs.json
   Compiling osirs v0.1.0 (/home/user/tmp/32)
error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-osirs` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-osirs`

For more information about this error, try `rustc --explain E0463`.
error: could not compile `osirs` (bin "osirs") due to 1 previous error

Nightly

  • Okay folks here’s the deal.
  • I’m not happy about it either.
  • build-std - which we need - is unstable and
  • [unstable] is only available in nightly Rust.
    • The version of Rust for nerds.
  • Not to worry, we are nerds and can use it.
    • Simply add a +nightly right after cargo.

Try Two Things

  • Here’s two things that also still won’t work.
cargo +nightly b --target x86_64-osirs.json
  • Gives this:
error: `.json` target specs require -Zjson-target-spec

Try Two Things

  • Here’s two things that also still won’t work.
cargo +nightly b
  • Gives this:
error: `.json` target specs require -Zjson-target-spec
   Compiling compiler_builtins v0.1.160 (/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/compiler-builtins/compiler-builtins)
   Compiling core v0.0.0 (/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core)
   Compiling osirs v0.1.0 (/home/user/tmp/32)
error: linking with `cc` failed: exit status: 1
  |
  = note:  "cc" "-m64" "/home/user/tmp/32/target/debug/deps/rustcMmXrbF/symbols.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/home/user/tmp/32/target/debug/deps/{libcore-0c26ef2bd74962c1,libcompiler_builtins-40a77a01cbdbd500}.rlib" "-L" "/home/user/tmp/32/target/debug/deps/rustcMmXrbF/raw-dylibs" "-Wl,-Bdynamic" "-B<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/tmp/32/target/debug/deps/osirs-62b17f4aa5d3b391" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: rust-lld: error: duplicate symbol: _start
          >>> defined at /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o:(_start)
          >>> defined at main.rs:6 (src/main.rs:6)
          >>>            /home/user/tmp/32/target/debug/deps/osirs-62b17f4aa5d3b391.3hpwl9lytayxk9wu897na7tu0.0wpyfaj.rcgu.o:(.text._start+0x0)
          collect2: error: ld returned 1 exit status


error: could not compile `osirs` (bin "osirs") due to 1 previous error

The Problem

  • Not everything works the same way with nightly and stable rust.
    • That’s why it’s not stable.
  • We will encounter two examples immediately, both related to JSON.
  • Let’s look at this:
$ cargo +nightly b --target x86_64-osirs.json
error: `.json` target specs require -Zjson-target-spec

One Solution

  • We can of course just put -Zjson-target-spec in there.
  • We get an error, but one we are clever enough to handle.
$ cargo +nightly b --target x86_64-osirs.json -Zjson-target-spec
  • Gives this:
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc - --crate-name ___ --print=file-names --target /home/user/tmp/32/x86_64-osirs.json -Zunstable-options --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg -Wwarnings` (exit status: 1)
  --- stderr
  error: error loading target specification: target-pointer-width: invalid type: string "64", expected u16 at line 6 column 32
    |
    = help: run `rustc --print target-list` for a list of built-in targets

Another Solution

  • However, that -Zjson-target-spec looks an awful lot like a .cargo/config.toml option…
    • I believe that is also unstable…
    • … after all, it works fine on stable!
.cargo/config.toml
[unstable]
build-std = ["core"]
  • We can then get the same error with less typing:
cargo +nightly b --target x86_64-osirs.json

The error

error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc - --crate-name ___ --print=file-names --target /home/user/tmp/32/x86_64-osirs.json -Zunstable-options --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg -Wwarnings` (exit status: 1)
  --- stderr
  error: error loading target specification: target-pointer-width: invalid type: string "64", expected u16 at line 6 column 32
    |
    = help: run `rustc --print target-list` for a list of built-in targets
  • Does that remind you of anything?
$ diff bad.wrong x86_64-osirs.json
6,7c6,7
<     "target-pointer-width": 64,
<     "target-c-int-width": 32,
---
>     "target-pointer-width": "64",
>     "target-c-int-width": "32",

Aside

  • If you did not see that error, that is okay!
  • It concerns an unstable language feature and may differ.
  • Just skip one slide!

Revert!

  • Change the string integers back to integer integers in your JSON file and you will be living a charmed and blessed life.
    • A long one, compilation is 10+ seconds for me.
$ cargo +nightly b --target x86_64-osirs.json
   Compiling compiler_builtins v0.1.160 (/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/compiler-builtins/compiler-builtins)
   Compiling core v0.0.0 (/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core)
^[[A    Building [==========>                  ] 2/5: core, compiler_builtins                                                                Compiling osirs v0.1.0 (/home/user/tmp/32)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 10.18s

Aside

  • By the way, are you tired of specifying the target every time?
    • Sounds like a problem for .cargo/config.toml!
.cargo/config.toml
[unstable]
build-std = ["core"]
json-target-spec = true

[build]
target = "x86_64-osirs.json"

Run again

  • By the way it will be fast now.
$ cargo +nightly b
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.08s

Aside: Nightly

  • Surely you can also update .cargo/config.toml to use nightly!
    • You can’t. You can solve the problem other ways though.
    • Left as an exercise to the interested student.

Aside: \(\not\)Futureproofing

  • I was pretty sure I can make a kernel without floats.
  • So I removed the two soft-float lines and it still worked for now.
$ diff old.boring x86_64-osirs.json
13,15c13
<     "disable-redzone": true,
<     "features": "-mmx,-sse,+soft-float",
<     "rustc-abi": "x86-softfloat"
---
>     "disable-redzone": true

Aside: \(\not\)Futureproofing

  • I was pretty sure I could make a kernel without compiler-builtins
  • These are memory related functions where Rust often uses linked C implementations.
  • I’m leaving them out for now and will add them in when I need them.
.cargo/config.toml
[unstable]
build-std-features = ["compiler-builtins-mem"]
build-std = ["core", "compiler_builtins"]

Boot it

  • And with that, I bet this will totally work.
  • Let’s break out qemu
$ qemu-system-x86_64 -kernel target/x86_64-osirs/debug/osirs
Command 'qemu-system-x86_64' not found, but can be installed with:
sudo apt install qemu-system-x86      # version 1:6.2+dfsg-2ubuntu6.27, or
sudo apt install qemu-system-x86-xen  # version 1:6.2+dfsg-2ubuntu6.27
  • Oh right, we only installed “misc” qemu
    • Real ones will use apt
    sudo apt install qemu-system-x86

Boot it

$ qemu-system-x86_64 -kernel target/x86_64-osirs/debug/osirs
qemu-system-x86_64: Error loading uncompressed kernel without PVH ELF Note
  • Oh right, we did precisely nothing with the BIOS and the bootloader and all that and still need to do those things.
    • I bet that would be a fun topic for a lab.

Fin

Main File

  • Unaltered except loops for stability.
src/main.rs
#![no_std]
#![no_main]

#[unsafe(no_mangle)]
pub extern "C" fn _start() -> ! {
    loop {}
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    loop {}
}

Config File

  • All new, only works with nightly.
.cargo/config.toml
[unstable]
build-std = ["core"]
json-target-spec = true

[build]
target = "x86_64-osirs.json"

Target File

  • All new, this is the nightly version.
x86_64-osirs.json
{
    "llvm-target": "x86_64-unknown-none",
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
    "arch": "x86_64",
    "target-endian": "little",
    "target-pointer-width": 64,
    "target-c-int-width": 32,
    "os": "none",
    "executables": true,
    "linker-flavor": "ld.lld",
    "linker": "rust-lld",
    "panic-strategy": "abort",
    "disable-redzone": true
}

Cargo File

  • Move panic specification to target.
Cargo.toml
[package]
name = "osirs"
version = "0.1.0"
edition = "2024"

[dependencies]