[Arm][Linux] Understanding Stacks in User and Kernel space

It is important to figure out how the stacks are organized in user space and kernel space in Linux system.

This post will outline the user space stack and kernel space stack:

User Stack

Every time a user task (process) is created, a user stack and a kernel stack are allocated.

The size of the user stack can be set when creating a thread. The default size depends on the CPU architecture.

To check the maximum size of the user stack, use the following command:

Kernel Stack

Each task (process) has its own kernel stack. The kernel stack is much smaller than the user stack. Its size depends on the Linux kernel version and configuration. The kernel stack usually uses 1 to 4 pages of memory.

Examples of kernel stack sizes:

• ARM32: 2 pages (8KB) - 0x2000
• ARM64 with 4KB pages: 4 pages (16KB) - 0x4000
• ARM64 with 16KB pages: 1 page (16KB) - 0x4000

Exception Stack (ARM32)

ARM32 architecture uses small mini-stacks for handling exceptions. When an exception occurs, the system first uses a 3-word mini-stack. Then, it switches to SVC mode(supervisor mode) and continues using the SVC stack (kernel stack) for further processing.

Stack Usage During Context Switching

• When a task (thread) runs in user space, it uses the user stack.
• When a system call is made, the task switches to kernel space and starts using the kernel stack.
• When switching back to user mode, the task resumes using the user stack.

Stack Registers in ARM32 and ARM64

• In ARM32, different stack pointers exist for each mode:

sp_usr, sp_svc, sp_irq, etc.

• In ARM64, stack pointers exist for each exception level:

el0_sp, el1_sp, el2_sp, etc.

Stack Usage During Interrupts

• If an interrupt occurs while a user task is running, the kernel stack of that task is used.
• If an interrupt occurs while the CPU is in kernel space, the behavior depends on the architecture:

ARM32: Uses the same kernel stack as the running kernel thread.

ARM64: Uses a separate interrupt stack.

• If a second interrupt (higher priority) occurs during an interrupt:

ARM32: Uses the same kernel stack.

ARM64: Uses a separate interrupt stack.

Some architectures separate stacks for hard IRQs and soft IRQs:

• ARM32: Hard IRQ uses the appropriate kernel stack as explained earlier. Soft IRQ runs in ksoftirqd, which has its own kernel stack.
• ARM64: Hard IRQ uses the interrupt stack. Soft IRQ can be configured to use either the interrupt stack or the ksoftirqd kernel stack.