Book Notes: How Linux Works (in progress)
→ 日本語版を読むOverview
Notes on important points from "How Linux Works."
Computer System Layers
Layer
User programs OS external libraries OS libraries Kernel Hardware
- In reality, it's not this neatly layered
Terminology
Program
- For compiled languages like Go, the executable file after building is the program
- For scripting languages like Python, the source code itself is the program
- The kernel is also a type of program. When the machine is powered on, the kernel starts, and all other programs start after it
- A program is a collection of instructions and data that operate on a computer
Process
- A running program is called a process
- A running process is sometimes called a program, so "program" is the broader concept
Kernel
- We don't want processes to access devices directly
- Hardware (CPU) has a feature called "mode"
- There is kernel mode and user mode
- In kernel mode, the CPU places no restrictions
- In user mode, the CPU prevents certain instructions from executing
- In Linux, only the kernel operates in kernel mode and can access devices
- Processes operate in user mode and cannot access devices
- The kernel centrally manages and distributes resources shared by processes
System Call
- A request from a process to the kernel for processing
- System calls are issued when requesting operations like creating new processes or hardware manipulation
- When a process issues a system call to the kernel, an event called an "exception" occurs in the CPU, and the CPU mode transitions from user mode to kernel mode. After the kernel finishes processing, it returns to user mode
Standard C Library
- Linux also provides the standard C library (libc)
- Almost all programs written in C are linked with libc
- bash, echo, and Python are also linked with standard libraries
libc.so.6refers to the standard C library
$ ldd /bin/bash
linux-vdso.so.1 (0x00007ffd3afbb000)
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007efda6879000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efda6873000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efda6681000)
/lib64/ld-linux-x86-64.so.2 (0x00007efda69e0000)
$ ldd /bin/echo
linux-vdso.so.1 (0x00007ffcc7529000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fba60498000)
/lib64/ld-linux-x86-64.so.2 (0x00007fba606a0000)
$ ldd /usr/bin/python3
linux-vdso.so.1 (0x00007fff9bcdb000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f99cd902000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f99cd8df000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f99cd8d9000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f99cd8d4000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f99cd785000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f99cd757000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f99cd739000)
/lib64/ld-linux-x86-64.so.2 (0x00007f99cdaff000)
System Call Wrappers
System calls must be invoked using assembly code.
For X86_64 architecture CPUs, the system call to get the parent process ID is issued with the following assembly code:
mov $0x6e,%eax
syscall
For arm64 architecture, it looks like this:
mov x8, <system call number>
svc #0
In C, the wrapper function getppid() is provided, which can be used to get the parent process ID.
pause() is also a wrapper function.
Wrapper functions eliminate the need to write assembly code manually.
CPU Architecture
X86
A series name for microprocessors (MPU/CPU) developed and manufactured by Intel for PCs and other devices. Also the name of the instruction set architecture for operating processors in this series. Named after its ancestor, the 16-bit MPU "8086" developed in 1978.
AMD64 (x86_64)
https://e-words.jp/w/AMD64.html
A 64-bit instruction set for microprocessors (CPU/MPU) developed by AMD, which enables execution of 64-bit code while maintaining compatibility with the 32-bit x86 instruction set.
Intel64
A 64-bit instruction set built into x86-series microprocessors (CPU/MPU) by Intel, which enables execution of 64-bit code while maintaining compatibility with the 32-bit x86 instruction set. Almost identical to AMD's AMD64 (x86-64).
ARM64
https://e-words.jp/w/ARM64.html
One of the fundamental designs (architecture) of microprocessors (MPU/CPU) developed by Arm Ltd., for processing programs and data in 64-bit units.
Static Libraries and Shared Libraries
Program generation flow:
-
Compile source code to create object files
-
Link libraries used by the object files to create an executable
-
Static library
- At link time, library functions are embedded directly into the program
-
Shared library
- At link time, references to library functions are embedded in the executable
- During program execution, the library is loaded into memory and library functions are used
Prepare code called pause.c as follows:
#include <unistd.h>
int main(void) {
pause(5);
return 0;
}
For static linking, compile as follows:
This creates an executable named pause.
cc -static -o pause pause.c
Use ls -l and ldd to check size and link status.
For shared library linking, compile as follows:
cc -o pause pause.c
The fork() Function for Splitting Processes
- The fork() function is a wrapper function for the system call that splits a process
- Creates a copy of the parent process's memory for the child process
- The return value of fork(): from the parent process it's the child process ID, from the child process it's 0
- The return value can be used to branch processing
#!/usr/bin/python3
import os, sys
ret = os.fork()
if ret == 0:
print("Child process: pid{}, parent process pid={}".format(os.getpid(), os.getppid()))
exit()
elif ret > 0:
print("Parent process: pid={}, child process pid={}".format(os.getpid(), ret))
exit()
sys.exit(1)
- When executed, PIDs for both parent and child processes are displayed:
$ ./fork.py
Parent process: pid=355, child process pid=356
Child process: pid356, parent process pid=355
The execve() Function for Starting Another Program
- The execve() function overwrites the current process's memory with content read from an executable file
- This creates new process memory for the new process, enabling the new process to start
- Combined with fork(), new processes can be created from the parent process
#!/usr/bin/python3
import os, sys
ret = os.fork()
if ret == 0:
print("Child process: pid={}, parent process pid={}".format(os.getpid(), os.getppid()))
os.execve("/bin/echo", ["echo", "Hello from child process pid={}".format(os.getpid())], {})
exit()
elif ret > 0:
print("Parent process: pid={}, child process pid={}".format(os.getpid(), ret))
exit()
sys.exit(1)
$ ./fork-and-exec.py
Parent process: pid=387, child process pid=388
Child process: pid=388, parent process pid=387
Hello from child process pid=388
Flow Until the init Process Starts
- Computer power on
- Firmware such as BIOS, UEFI starts and initializes hardware
- Firmware starts a bootloader such as GRUB
- Bootloader starts the OS kernel (Linux kernel)
- Linux kernel starts the init process
- init process starts child processes
↓ View of init process starting child processes:
$ pstree -p
init(1)─┬─init(13)─┬─init(14)───bash(15)───sudo(303)───unshare(304)───bash(305)
│ └─init(84)
├─init(320)───init(321)───bash(322)───pstree(390)
└─{init}(6)
Process States
- Runnable state: CPU execution rights not yet obtained
- Running state: CPU execution rights obtained
- Sleep state: Idle process running. Transitions to runnable state when an event occurs
- Zombie state
- Process terminated
Process Termination
A process can be terminated with the exit_group() system call.
During this system call, the kernel reclaims process resources such as memory.
After a process terminates, calling the wait() or waitpid() system calls allows you to check the process's return value, whether it terminated via a signal, and how much CPU time it used.
The fact that wait() system call can be used to get the above information means that even after process termination, the terminated child process continues to exist in the system in some form.
Offset, Size, Memory Map Start Address, Entry Point
Executable files hold the following in addition to program code and data:
- File offset, size, and memory map start address for the code region
- File offset, size, and memory map start address for the data region
- Entry point (memory address of the first instruction to execute)
File offsets, sizes, and memory map start addresses can be checked with readelf -S <executable name>.
The entry point can be checked with readelf -h <executable name>.
$ cc -o pause -no-pie pause.c
$ readelf -S pause
There are 31 section headers, starting at offset 0x3938:
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .interp PROGBITS 0000000000400318 00000318
000000000000001c 0000000000000000 A 0 0 1
[ 2] .note.gnu.propert NOTE 0000000000400338 00000338
0000000000000020 0000000000000000 A 0 0 8
[ 3] .note.gnu.build-i NOTE 0000000000400358 00000358
0000000000000024 0000000000000000 A 0 0 4
[ 4] .note.ABI-tag NOTE 000000000040037c 0000037c
0000000000000020 0000000000000000 A 0 0 4
[ 5] .gnu.hash GNU_HASH 00000000004003a0 000003a0
000000000000001c 0000000000000000 A 6 0 8
[ 6] .dynsym DYNSYM 00000000004003c0 000003c0
0000000000000060 0000000000000018 A 7 1 8
[ 7] .dynstr STRTAB 0000000000400420 00000420
000000000000003e 0000000000000000 A 0 0 1
[ 8] .gnu.version VERSYM 000000000040045e 0000045e
0000000000000008 0000000000000002 A 6 0 2
[ 9] .gnu.version_r VERNEED 0000000000400468 00000468
0000000000000020 0000000000000000 A 7 1 8
[10] .rela.dyn RELA 0000000000400488 00000488
0000000000000030 0000000000000018 A 6 0 8
[11] .rela.plt RELA 00000000004004b8 000004b8
0000000000000018 0000000000000018 AI 6 24 8
[12] .init PROGBITS 0000000000401000 00001000
000000000000001b 0000000000000000 AX 0 0 4
[13] .plt PROGBITS 0000000000401020 00001020
0000000000000020 0000000000000010 AX 0 0 16
[14] .plt.sec PROGBITS 0000000000401040 00001040
0000000000000010 0000000000000010 AX 0 0 16
[15] .text PROGBITS 0000000000401050 00001050
0000000000000175 0000000000000000 AX 0 0 16
[16] .fini PROGBITS 00000000004011c8 000011c8
000000000000000d 0000000000000000 AX 0 0 4
[17] .rodata PROGBITS 0000000000402000 00002000
0000000000000004 0000000000000004 AM 0 0 4
[18] .eh_frame_hdr PROGBITS 0000000000402004 00002004
0000000000000044 0000000000000000 A 0 0 4
[19] .eh_frame PROGBITS 0000000000402048 00002048
0000000000000100 0000000000000000 A 0 0 8
[20] .init_array INIT_ARRAY 0000000000403e10 00002e10
0000000000000008 0000000000000008 WA 0 0 8
[21] .fini_array FINI_ARRAY 0000000000403e18 00002e18
0000000000000008 0000000000000008 WA 0 0 8
[22] .dynamic DYNAMIC 0000000000403e20 00002e20
00000000000001d0 0000000000000010 WA 7 0 8
[23] .got PROGBITS 0000000000403ff0 00002ff0
0000000000000010 0000000000000008 WA 0 0 8
[24] .got.plt PROGBITS 0000000000404000 00003000
0000000000000020 0000000000000008 WA 0 0 8
[25] .data PROGBITS 0000000000404020 00003020
0000000000000010 0000000000000000 WA 0 0 8
[26] .bss NOBITS 0000000000404030 00003030
0000000000000008 0000000000000000 WA 0 0 1
[27] .comment PROGBITS 0000000000000000 00003030
000000000000002b 0000000000000001 MS 0 0 1
[28] .symtab SYMTAB 0000000000000000 00003060
00000000000005e8 0000000000000018 29 45 8
[29] .strtab STRTAB 0000000000000000 00003648
00000000000001ca 0000000000000000 0 0 1
[30] .shstrtab STRTAB 0000000000000000 00003812
000000000000011f 0000000000000000 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)
$ readelf -h pause
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x401050
Start of program headers: 64 (bytes into file)
Start of section headers: 14648 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 13
Size of section headers: 64 (bytes)
Number of section headers: 31
Section header string table index: 30
The entry point value is as follows:
Entry point address: 0x401050
namespace
For various types of resources that exist in the system, there is a corresponding namespace for each.
A feature that makes processes belonging to a namespace see apparently independent resources.
Examples include:
- pid namespace: shows an independent pid namespace
- user namespace: shows independent uid and gid
- mount namespace: shows an independent filesystem mount status
pid namespace
In the following example, the pid ns is 4026532192.
$ ls -l /proc/$$/ns/pid
lrwxrwxrwx 1 bluen bluen 0 Mar 16 23:21 /proc/15/ns/pid -> 'pid:[4026532192]'
Using the unshare --pid command, a new pid ns is created and bash is run in the newly created namespace.
The pid ns is now 4026532211, showing that its ID differs from the parent pid ns.
Furthermore, it can be seen that child pid ns cannot see processes from the parent pid ns.
$ sudo unshare --fork --pid --mount-proc bash
$ echo $$
1
$ ls -l /proc/1/ns/pid
lrwxrwxrwx 1 root root 0 Mar 16 23:23 /proc/1/ns/pid -> 'pid:[4026532211]'
$ ps ax
PID TTY STAT TIME COMMAND
1 pts/0 S 0:00 bash
9 pts/0 R+ 0:00 ps ax
Opening another terminal and checking from the parent pid ns, bash (PID=305) belonging to the child pid ns can be confirmed.
In other words, the parent pid ns can see processes in the child pid ns.
Also note that the process ID of bash visible in the parent pid ns (PID=305) differs from the process ID visible within the child pid ns (PID=1).
$ pstree -p
init(1)─┬─init(13)─┬─init(14)───bash(15)───sudo(303)───unshare(304)───bash(305)
│ └─init(84)
├─init(320)───init(321)───bash(322)───pstree(387)
├─{init}(6)
└─{init}(386)
The number of namespaces available in the Linux kernel continues to grow.
Containers
A container is a process that uses the namespaces described above to have a separate execution environment from other processes.
Which namespaces (pid ns, user ns, mount ns) are separated depends on the container runtime.
Since the execution environment is separated from other processes, be aware that if there is a problem caused by the host OS or another container, it may not be visible from inside the container.
Various Container Runtimes
With containers, the host system and all containers on the host share the kernel.
If the kernel has a vulnerability, there is a risk that container users could spy on information from the host OS or other containers.
Various container runtimes have emerged besides Docker:
- Kata Container
- gVisor