Introduce Kernel Partitioning

[etiaro]

Description

Introduce spatial and temporal partitioning support in the kernel.

Key changes:

• Add syspage_part_t partition abstraction holding partition configuration
• Full MPU reconfiguration on context switch (pmap_switch) for flexible per-process memory isolation on MPU targets
• Temporal partitioning via configurable scheduler windows for partition time separation
• Partition memory limit accounting on MMU targets for resource safety
• Disable inter-partition synchronous messaging (IPC on shared-memory recommended instead)
• Separate per-partition sleeping trees to reduce inter-partition interference

NOTE: this is a starting point version, related partitioning features will be in active development during upcoming weeks/months.

Work-In-Progress partitioning documentation

Motivation and Context

Separation mechanisms for safety-critical and mixed-criticality systems where spatial and temporal isolation between groups of processes must be guaranteed.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Chore (refactoring, style fixes, git/CI config, submodule management, no code logic changes)

How Has This Been Tested?

• Already covered by automatic testing.
• New test added: (add PR link here).
• Tested by hand on: (list targets here).

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing linter checks and tests passed.
• My changes generate no new compilation warnings for any of the targets.

Special treatment

• This PR needs additional PRs to work (list the PRs, preferably in merge-order).
• Introduce Kernel Partitioning plo#419
• Introduce Kernel Partitioning phoenix-rtos-build#261
• target cofiguration changes in Phoenix-RTOS-project (not yet on PR)
• I will merge this PR by myself when appropriate.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a fundamental kernel partitioning mechanism, significantly enhancing resource isolation and management capabilities. It refactors core operating system components, including memory management, scheduling, and inter-process communication, to be partition-aware. This change allows for stricter control over how processes consume resources and interact with each other, leveraging hardware features like MMUs/MPUs for robust separation. The scheduler now supports distinct scheduling windows for different partitions, and memory allocations are tracked per partition, laying the groundwork for more secure and predictable real-time systems.

Highlights

• Kernel Partitioning Introduction: Introduced the concept of kernel partitioning to enable resource isolation and management for processes. This involves new syspage_part_t and hal_syspage_part_t structures to define partitions and their hardware-specific configurations.
• Memory Management Unit (MMU)/Memory Protection Unit (MPU) Integration: Updated architecture-specific pmap implementations (ARMv7M, ARMv7R, ARMv8M) to utilize partition-specific MPU configurations, allowing dynamic switching of memory access rules based on the active process's partition. This enhances memory protection and isolation between different software components.
• Partition-Aware Memory Allocation: Modified vm_pageAlloc and vm_pageFree functions to accept a syspage_part_t argument, enabling memory allocation and deallocation to be tracked and managed on a per-partition basis. This ensures that memory resources are correctly attributed and constrained by their assigned partition.
• Partitioned Scheduler: Refactored the thread scheduler to support multiple scheduling windows and per-partition ready/sleeping queues. This allows for more flexible and isolated scheduling policies, where processes within different partitions can have their own scheduling parameters and resource guarantees.
• Process and IPC Partition Enforcement: Integrated partition awareness into process creation (proc_start, proc_spawn, proc_vfork) and inter-process communication (IPC) mechanisms (proc_send, proc_recv). This enforces access control, preventing processes from spawning outside their assigned partition or communicating with processes in unauthorized partitions.
• Syspage Structure Enhancements: Extended the syspage structures to include definitions for scheduler windows and partitions, and linked programs (syspage_prog_t) to their respective partitions. This provides a centralized configuration for the new partitioning features.

Changelog

• hal/aarch64/pmap.c
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
• hal/armv7a/pmap.c
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
• hal/armv7m/arch/pmap.h
  • Included syspage.h header.
  • Replaced u32 regions with const hal_syspage_part_t *hal in pmap_t structure.
• hal/armv7m/pmap.c
  • Included lib/lib.h header and defined MPU_BASE.
  • Modified pmap_common structure to remove kernelCodeRegion and add last_mpu_count.
  • Updated pmap_create to assign pmap->hal based on prog argument.
  • Rewrote pmap_switch to use hal_syspage_part_t for MPU configuration and optimized MPU region loading using assembly.
  • Modified pmap_isAllowed to check pmap->hal for access permissions and removed pmap_map2region.
  • Refactored _pmap_init to remove kernelCodeRegion and related logic, initializing pmap->hal and pmap_common.last_mpu_count.
• hal/armv7r/arch/pmap.h
  • Included syspage.h header.
  • Replaced u32 regions with const hal_syspage_part_t *hal in pmap_t structure.
• hal/armv7r/pmap.c
  • Modified pmap_common structure to remove kernelCodeRegion and add last_mpu_count and last_mpu_conf arrays.
  • Updated pmap_create to assign pmap->hal based on prog argument.
  • Rewrote pmap_switch to use hal_syspage_part_t for MPU configuration, disabling/enabling MPU and setting regions based on partition.
  • Modified pmap_isAllowed to check pmap->hal for access permissions and removed pmap_map2region.
  • Refactored _pmap_init to remove kernelCodeRegion and related logic, initializing pmap->hal, last_mpu_count, and last_mpu_conf.
• hal/armv8m/arch/pmap.h
  • Included syspage.h header.
  • Replaced u32 regions with const hal_syspage_part_t *hal in pmap_t structure.
• hal/armv8m/mcx/n94x/config.h
  • Included hal/types.h and include/syspage.h headers.
• hal/armv8m/pmap.c
  • Included lib/lib.h header.
  • Modified pmap_common structure to remove kernelCodeRegion and add last_mpu_count.
  • Updated pmap_create to assign pmap->hal based on prog argument.
  • Rewrote pmap_switch to use hal_syspage_part_t for MPU configuration and optimized MPU region loading using assembly.
  • Modified pmap_isAllowed to check pmap->hal for access permissions and removed pmap_map2region.
  • Refactored _pmap_init to remove kernelCodeRegion and related logic, initializing pmap->hal and pmap_common.last_mpu_count.
• hal/armv8r/pmap.c
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
• hal/ia32/pmap.c
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
• hal/pmap.h
  • Included syspage.h header.
  • Removed pmap_addMap function declaration.
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
• hal/riscv64/_init.S
  • Increased size of _hal_syspageCopied zero-initialized section from 0x600 to 0x700.
• hal/riscv64/pmap.c
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
• hal/sparcv8leon/pmap-nommu.c
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
  • Removed pmap_addMap function.
• hal/sparcv8leon/pmap.c
  • Updated pmap_create function signature to include const syspage_prog_t *prog argument.
• include/arch/aarch64/zynqmp/syspage.h
  • Added hal_syspage_part_t structure with a dummy member.
• include/arch/armv7a/imx6ull/syspage.h
  • Added hal_syspage_part_t structure with a dummy member.
• include/arch/armv7a/zynq7000/syspage.h
  • Added hal_syspage_part_t structure with a dummy member.
• include/arch/armv7m/imxrt/syspage.h
  • Refactored hal_syspage_t to move MPU configuration into a new hal_syspage_part_t structure and added mpuType to hal_syspage_t.
• include/arch/armv7m/stm32/syspage.h
  • Refactored hal_syspage_t to move MPU configuration into a new hal_syspage_part_t structure and added mpuType to hal_syspage_t.
• include/arch/armv7r/tda4vm/syspage.h
  • Refactored hal_syspage_t to move MPU configuration into a new hal_syspage_part_t structure and added mpuType to hal_syspage_t.
• include/arch/armv7r/zynqmp/syspage.h
  • Refactored hal_syspage_t to move MPU configuration into a new hal_syspage_part_t structure and added mpuType to hal_syspage_t.
• include/arch/armv8m/mcx/syspage.h
  • Refactored hal_syspage_t to move MPU configuration into a new hal_syspage_part_t structure and added mpuType to hal_syspage_t.
• include/arch/armv8m/nrf/syspage.h
  • Refactored hal_syspage_t to move MPU configuration into a new hal_syspage_part_t structure and added mpuType to hal_syspage_t.
• include/arch/armv8m/stm32/syspage.h
  • Refactored hal_syspage_t to move MPU configuration into a new hal_syspage_part_t structure and added mpuType to hal_syspage_t.
• include/arch/armv8r/mps3an536/syspage.h
  • Added hal_syspage_part_t structure with a dummy member.
• include/arch/ia32/syspage.h
  • Added hal_syspage_part_t structure with a dummy member.
• include/arch/riscv64/syspage.h
  • Added hal_syspage_part_t structure with a dummy member.
• include/arch/sparcv8leon/syspage.h
  • Added hal_syspage_part_t structure with a dummy member.
• include/syspage.h
  • Added pFlagSpawnAll and pFlagIPCAll enums.
  • Introduced syspage_sched_window_t structure for scheduler windows.
  • Introduced syspage_part_t structure for kernel partitions, including HAL-specific data.
  • Updated syspage_prog_t to include a pointer to its syspage_part_t.
  • Modified syspage_t to include linked lists for partitions and schedWindows.
  • Added declarations for syspage_schedulerWindowList and syspage_partitionList functions.
• main.c
  • Updated proc_start call for main_initthr to pass NULL for the new partition argument.
• perf/buffer-mem.c
  • Updated vm_pageFree and vm_pageAlloc calls to pass NULL for the new partition argument.
• proc/msg-nommu.c
  • Added msg_isAllowed static function to check IPC permissions based on process partitions.
  • Integrated msg_isAllowed checks into proc_send and proc_recv functions.
• proc/msg.c
  • Updated vm_pageAlloc and vm_pageFree calls to pass the source process's partition.
  • Added msg_isAllowed static function to check IPC permissions based on process partitions.
  • Integrated msg_isAllowed checks into proc_send and proc_recv functions.
• proc/process.c
  • Updated proc_start function signature to include syspage_part_t *partition argument.
  • Added partition field to the process_t structure during allocation.
  • Removed unused i variable in process_exec.
  • Modified pmap_create call in process_exec to pass spawn->prog.
  • Removed pmap_addMap calls for instruction and data maps in process_exec.
  • Added partition selection and access checks in proc_spawn.
  • Updated proc_start call in proc_spawn to pass the determined partition.
  • Updated vm_objectGet calls in proc_fileSpawn and proc_execve to pass the current process's partition.
  • Updated proc_start call in proc_vfork to pass the parent process's partition.
• proc/process.h
  • Added syspage_part_t *partition member to the _process_t structure.
  • Updated proc_start function signature to include syspage_part_t *partition argument.
• proc/threads.c
  • Defined NUM_PRIO and NO_WAKEUP macros.
  • Modified threads_common structure to use dynamic arrays for ready, sleeping, sleepMin, actWindow, and windowStart to support multiple scheduler windows.
  • Replaced MAX_PRIO macro with NUM_PRIO.
  • Updated _threads_updateWakeup to take a windowId argument and manage wakeup times per window.
  • Modified threads_timeintr to iterate through scheduler windows and update wakeup times for background windows.
  • Added proc_getSchedWindowId and proc_getReadyQueues functions to determine the correct scheduler window and ready queue for a process.
  • Updated _threads_schedule to manage scheduler windows, switch active windows, and select threads from appropriate ready queues.
  • Modified hal_timerSetWakeup call in _threads_schedule to use window-specific wakeup times.
  • Updated proc_threadCreate to use NUM_PRIO and add threads to partition-specific ready queues.
  • Adjusted _proc_lockGetPriority and _proc_threadGetLockPriority to use NUM_PRIO - 1U.
  • Modified _proc_threadSetPriority to update threads in partition-specific ready queues.
  • Updated proc_threadPriority to use NUM_PRIO.
  • Modified _proc_threadDequeue, _proc_threadEnqueue, and _proc_threadSleepAbs to use partition-specific sleeping red-black trees and update wakeup times.
  • Rewrote _proc_nextWakeup to consider all scheduler windows and their respective wakeup times.
  • Updated proc_threadsDump to iterate through all scheduler windows when dumping threads.
  • Refactored _threads_init to dynamically allocate and initialize scheduler queues, sleeping trees, and wakeup times for each scheduler window, and to initialize actWindow and windowStart for each CPU.
  • Updated proc_threadCreate call for idle threads to use NUM_PRIO - 1U.
  • Added a workaround for sparcv8leon to set up the SYSTICK timer.
• syscalls.c
  • Updated vm_objectGet call in syscalls_sys_mmap to pass the process's partition.
  • Modified syscalls_sys_mmap for NOMMU systems to use partition-specific shared maps for memory allocation.
  • Removed process_t *proc declaration in syscalls_sys_munmap and syscalls_sys_mprotect.
  • Modified syscalls_sys_munmap and syscalls_sys_mprotect for NOMMU systems to use shared maps based on syspage_map_t.
• syspage.c
  • Added syspage_schedulerWindowList and syspage_partitionList functions.
  • Updated syspage_init to handle relocation of syspage_prog_t's partition field.
  • Added relocation logic for syspage_part_t and syspage_sched_window_t linked lists.
• syspage.h
  • Added declarations for syspage_schedulerWindowList and syspage_partitionList functions.
• test/proc.c
  • Updated proc_start call in test_proc_exit to pass NULL for the new partition argument.
• test/vm.c
  • Updated vm_pageAlloc and vm_pageFree calls to pass NULL for the new partition argument.
• vm/amap.c
  • Updated amap_putanon to take syspage_part_t *part argument and pass it to vm_pageFree.
  • Updated amap_putanons to pass amap->partition to amap_putanon.
  • Updated amap_create function signature to include syspage_part_t *part argument.
  • Added partition field to the amap_t structure during creation.
  • Updated vm_pageAlloc and vm_pageFree calls in amap_page to pass amap->partition.
• vm/amap.h
  • Added syspage_part_t *partition member to the _amap_t structure.
  • Updated amap_create function signature to include syspage_part_t *part argument.
• vm/map.c
  • Updated _map_force function signature to include syspage_part_t *part argument.
  • Updated _vm_mmap to pass the process's partition to _map_force.
  • Added logic in vm_mapForce to retrieve the current process's partition and pass it to _map_force.
  • Updated amap_create call in _map_force to pass the partition.
  • Updated _map_force calls in vm_mprotect to pass the process's partition.
  • Updated vm_pageAlloc and vm_pageFree calls in vm_mapCreate and vm_mapDestroy to pass NULL for the partition argument.
  • Updated pmap_create calls in vm_mapCreate to pass NULL for the prog argument.
  • Updated _map_force calls in vm_mapCopy to pass the process's partition.
• vm/object.c
  • Updated vm_objectGet function signature to include syspage_part_t *part argument.
  • Added part field to the vm_object_t structure during creation.
  • Updated vm_pageFree calls in vm_objectPut to pass o->part.
  • Updated object_fetch function signature to take vm_object_t *o instead of oid_t oid.
  • Updated proc_open, vm_pageAlloc, vm_pageFree, proc_close, proc_read, vm_munmap calls in object_fetch to use o->oid and o->part.
  • Modified vm_objectPage to handle o == NULL by allocating a page using amap->partition if available.
  • Updated vm_pageFree calls in vm_objectPage to pass o->part.
  • Updated vm_pageAlloc call in vm_objectContiguous to pass the current process's partition.
  • Updated vm_pageFree call in vm_objectContiguous to pass part.
  • Added part field to the vm_object_t structure during contiguous object creation.
  • Updated vm_objectGet call in _object_init to pass NULL for the partition argument.
• vm/object.h
  • Added syspage_part_t *part member to the _vm_object_t structure.
  • Updated vm_objectGet function signature to include syspage_part_t *part argument.
• vm/page-nommu.c
  • Updated vm_pageAlloc and vm_pageFree function signatures to include syspage_part_t *partition argument.
• vm/page.c
  • Updated _page_alloc function signature to include syspage_part_t *part argument.
  • Added partition memory usage checks in _page_alloc.
  • Updated _page_alloc to increment part->usedMem.
  • Updated vm_pageAlloc function signature to include syspage_part_t *part argument and pass it to _page_alloc.
  • Updated vm_pageFree function signature to include syspage_part_t *part argument.
  • Added partition memory usage decrement and assertion in vm_pageFree.
  • Updated _page_alloc calls in _page_map, _page_sbrk, and _page_init to pass NULL for the partition argument.
• vm/page.h
  • Updated vm_pageAlloc and vm_pageFree function signatures to include syspage_part_t *part argument.
• vm/zone.c
  • Updated vm_pageAlloc and vm_pageFree calls in _vm_zoneCreate and _vm_zoneDestroy to pass NULL for the partition argument.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new memory partitioning and scheduling mechanism, primarily affecting ARMv7M, ARMv7R, and ARMv8M architectures. Key changes include modifying the pmap_t structure to link directly to a hal_syspage_part_t for MPU configuration, refactoring pmap_switch and pmap_isAllowed to utilize this new structure, and removing dynamic MPU region management functions. The syspage.h and architecture-specific syspage.h files are updated to define syspage_part_t and syspage_sched_window_t structures, allowing for per-partition memory and scheduling configurations. Process and thread management functions, such as proc_start, vm_objectGet, vm_pageAlloc, and vm_pageFree, are updated to accept a syspage_part_t argument, enabling resource tracking and access control based on partitions. Additionally, IPC functions (proc_send, proc_recv) now include msg_isAllowed checks based on partition flags, and the scheduler (proc/threads.c) is enhanced to support multiple scheduling windows and per-partition ready/sleeping queues. Review comments highlight critical security vulnerabilities in NOMMU systems where syscalls_sys_munmap and syscalls_sys_mprotect could allow unprivileged processes to modify or unmap kernel memory due to a lack of ownership verification. Another comment points out potential memory leaks in the _threads_init function if vm_kmalloc calls fail, as allocated resources are not properly freed.

[github-actions]

Unit Test Results

7 958 tests   7 292 ✅  40m 46s ⏱️
  649 suites    495 💤
    1 files      171 ❌

For more details on these failures, see this check.

Results for commit d070577.

♻️ This comment has been updated with latest results.

[adamgreloch]

This is quite a complex functionality. I'd add some more comments about the introduced scheduling scheme (explicit description of background+cyclical partitions) and explain a bit the wakeup manipulations

[ziemleszcz]

Why does this PR include commits from master (by adamgreloch and jmaksymowicz)?

[etiaro]

Why does this PR include commits from master (by adamgreloch and jmaksymowicz)?

Ideally, I would like to rebase feature/partitioning to master, but I don't have the write permissions.
I want to keep it up-to-date, especially with error handling updates, and avoid resolving conflicts back-and-forth.
I'm not sure how should I handle that.

EDIT: this time fixed by recreating the feature/partitioning branch, as it had no diverging commits yet

[adamgreloch]

/gemini review

[gemini-code-assist]

Warning

Gemini is experiencing higher than usual traffic and was unable to create the review. Please try again in a few hours by commenting /gemini review.

[adamgreloch]

/gemini review come on tin-man

[gemini-code-assist]

Code Review

This pull request introduces support for partitions and scheduling windows across multiple architectures, enabling resource isolation and time-partitioned execution. Key changes include updating memory management (vm_pageAlloc, vm_pageFree, vm_objectGet) to track usage per partition, modifying the scheduler to handle per-window ready queues and sleeping trees, and enforcing partition-based access controls for IPC and process spawning. Review feedback identifies critical risks of NULL pointer dereferences in the thread initialization logic if scheduling windows or partitions are not explicitly defined in the syspage.

[etiaro]

Changed base branch to avoid problems with branch protection while rebasing and CI errors spam.

At the same time, changed the implementation from by default creating partition-per-app to single default partition, as it seems like a cleaner for the future features.

Introduced Per-CPU scheduler configuration for improved elasticity

Removed NOMMU multimap allocation, as it was only partially implemented and broke with the new conception of default partition. Will be introduced in the future together with MMU NUMA support.