!syscalls: move signal handling to kernel

[etiaro]

Description

• keep actions for every signal and signal trampoline in process_t struct
  • allocate actions lazily on first change
• remove unused sigmask from process_t struct, rely on thread_t sigmask field
• add handler address as signal_trampoline stack parameter
• use POSIX signal codes and default behaviors
• replaced signalHandle syscall with signalAction
• handled vforked process signal termination
• remove signalPost syscall as duplicate to sys_tkill

Motivation and Context

Moving singal handlers to kernel enables it to know whether coredump should be run, by knowing whether a signal handler is installed for given signal.
Simplifies handling exceptions with problems like stackoverflow (JIRA: RTOS-1066, currently stackoverflow causes kernel exception during sigsegv handling even with default sigsegv action).
Allows improved POSIX compliability (eg. not interrupting sleep by ignored signals).

Resolves phoenix-rtos/phoenix-rtos-project#1367
Resolves phoenix-rtos/phoenix-rtos-project#1456
Resolves phoenix-rtos/phoenix-rtos-project#1464
Resolves phoenix-rtos/phoenix-rtos-project#1453
Resolves phoenix-rtos/phoenix-rtos-project#1432
Resolves phoenix-rtos/phoenix-rtos-project#1448
JIRA: RTOS-736

Depends-On: phoenix-rtos-tests:etiaro/signals

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Chore (refactoring, style fixes, git/CI config, submodule management, no code logic changes)

How Has This Been Tested?

• Already covered by automatic testing.
• New test added: libc: add more signal tests phoenix-rtos-tests#410
• Tested by hand on:
• qemu with ia32, aarch64 and riscv, with SMP between 1 and 10 cores
• nightly tests
• additional test crafted for detecting smp/race issues (can be found here)

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing linter checks and tests passed.
• My changes generate no new compilation warnings for any of the targets.

Special treatment

• This PR needs additional PRs to work (list the PRs, preferably in merge-order).
  • !signal: move signal handling to kernel libphoenix#420
• I will merge this PR by myself when appropriate.

[gemini-code-assist]

Summary of Changes

Hello @etiaro, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a major overhaul of the kernel's signal handling system, transitioning from a custom implementation to one that adheres closely to POSIX standards. The changes involve restructuring data types, centralizing signal configuration within the kernel's process and thread management, and updating the system call interface for signal manipulation. This refactoring aims to provide a more robust and standard-compliant signal delivery mechanism.

Highlights

• POSIX Signal Alignment: The signal handling mechanism has been significantly refactored to align with POSIX standards. This includes replacing custom signal enumerations with standard POSIX signal numbers (e.g., SIGHUP, SIGSEGV) and introducing comprehensive sigaction, siginfo_t, and sigset_t structures.
• Kernel-level Signal Management: Signal handling logic has been moved more deeply into the kernel. The process_t structure now stores an array of sigaction structures for per-signal configuration and a dedicated sigtrampoline address. Signal masks are now primarily managed per-thread, improving granularity.
• Updated Signal Delivery and Syscalls: The core signal delivery functions (threads_sigpost, _threads_checkSignal) have been rewritten to utilize the new sigaction structure, supporting flags like SA_NODEFER and SA_RESETHAND. The signalHandle syscall has been renamed to signalAction and updated to accept sigaction parameters. Signal mask manipulation now uses a new threads_updateSigmask helper.
• HAL Integration: The Hardware Abstraction Layer (HAL) function hal_cpuPushSignal has been modified to differentiate between the signal trampoline and the actual handler address, ensuring both are correctly passed to the signal context on the stack.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request refactors the signal handling mechanism, moving it into the kernel and aligning it more closely with POSIX standards. The changes are well-structured, replacing the old system with a more robust sigaction-based approach. I've identified areas for improvement to enhance correctness and robustness, specifically around input validation in the signal posting logic and a type mismatch in a new function's signature.

[nalajcie]

Please don't add me as a reviewer to draft PRs without any extra context (as per our coding guidelines - we're not reviewing drafts, they are work-in-progress until you explicitly mark it as ready-for-review).

As an exception I did a code review of the kernel part, I won't be looking at libphoenix part unless explicitly asked.

I'm missing a context regarding as to Why the change is needed.

[nalajcie]

As you've summarised - the signal handling is now cluttered and really not readable. Probably all of the actual signal handling should be done in _threads_checkSignal (or maybe some wrapper function for that) - all other code paths are "optimizations" which are either prone to a race condition or a little bit convoluted (I'm not saying that we shouldn't eg. handle SIGKILL as early as possible as an optimization).

You may want to look at other OSes regarding how signal handling is being implemented there (usually in more complex way but the basics - like deciding on action to perform on signal dequeue in the context of receiving thread - are universal).

Please - present the additional test cases You've implemented the code you're pushing here passes - eg. I don't see setting return code of the process killed by a signal anywhere.

[nalajcie]

probably more kosher (but harder to implement currently) would be to create 2 stack frames - with trampoline (calling sigreturn) below the handler - and jump directly to the handler

[etiaro]

Not sure if it's worth the trouble, would require some additional changes apart from stack modification (eg. setting arm lr register in or before hal_jmp to handler)

[julianuziemblo]

small things. I'm not an expert but for me it looks good.

[github-actions]

Unit Test Results (nightly)

     1 files   3 070 suites   6h 3m 50s ⏱️
26 559 tests 22 596 ✅ 3 963 💤 0 ❌
26 745 runs  22 743 ✅ 4 002 💤 0 ❌

Results for commit c01e4ee.

♻️ This comment has been updated with latest results.

[etiaro]

Waiting with rerun for master nightly to work properly.