Skip to content

qubes-vmm-xen: init at 4.19.0-5, qubes-seabios: init at 4.0.2#341429

Closed
SigmaSquadron wants to merge 3 commits into
NixOS:masterfrom
SigmaSquadron:qubes-vmm-xen
Closed

qubes-vmm-xen: init at 4.19.0-5, qubes-seabios: init at 4.0.2#341429
SigmaSquadron wants to merge 3 commits into
NixOS:masterfrom
SigmaSquadron:qubes-vmm-xen

Conversation

@SigmaSquadron

@SigmaSquadron SigmaSquadron commented Sep 12, 2024

Copy link
Copy Markdown
Contributor

Description of changes

The first (and hopefully the last) custom Xen to be built based on the generic Xen builder. It's just a standard build of Xen with a lot of patches.

Compliments #341215.
Depends on #345192, #345324 and #342692.
Fixes #340544.

Things done

  • Built on platform(s)
    • x86_64-linux
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review pr 341429". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • This package is a minor addition. Any release notes should go to the main Qubes PR by Yaroslav.
  • Fits CONTRIBUTING.md.

cc @CertainLach


Add a 👍 reaction to pull requests you find important.

@SigmaSquadron

SigmaSquadron commented Sep 12, 2024

Copy link
Copy Markdown
Contributor Author

TODO:

  • Make sure the patches are all the changes that must be made to the package. (and nothing is hiding on the RPM build spec)
  • Figure out nixpkgs-vet:

    If the path being referenced is internal and has multiple uses, consider passing the file as an explicit callPackage argument in pkgs/top-level/all-packages.nix.

    • I wonder how stupid would it be to do something like makeXenPackage. Some other time, perhaps. Let's just not put it in by-name for now.
  • Test the custom hypervisor with Qubes packages (Its alive!) #341215 and see if anything is fixed/broken when compared to vanilla Xen.
    • SeaBIOS needed patching.
  • Make sure meta is functional.

@SigmaSquadron SigmaSquadron changed the title qubes-vmm-xen: init at 4.19.0-3 WIP: qubes-vmm-xen: init at 4.19.0-3 Sep 12, 2024
@SigmaSquadron SigmaSquadron added 2.status: work-in-progress 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. labels Sep 12, 2024
@SigmaSquadron SigmaSquadron force-pushed the qubes-vmm-xen branch 5 times, most recently from 9acc181 to af27cd3 Compare September 12, 2024 23:49
@ofborg ofborg Bot added 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Sep 13, 2024
@CertainLach

CertainLach commented Sep 14, 2024

Copy link
Copy Markdown
Member

++ lib.lists.optional withSeaBIOS "--with-system-seabios=${seabios}/share/seabios"
Is wrong, it should point to seabios file, not to the directory with it:
2024-09-14 17:03:28.955+0000: libxl: libxl_utils.c:348:libxl_read_file_contents: /nix/store/v1h1jgzlpcvn8rsslzb1fsr9gwv3wbpd-seabios-1.16.3/share/seabios is not a plain file: No such file or directory

Changing it to
++ lib.lists.optional withSeaBIOS "--with-system-seabios=${seabios}/share/seabios/Csm16.bin"
Fixes that problem.

I'm not sure if that's qubes changes, or system seabios flag is broken in xen derivation itself.

Interestingly, in qubes, this directory is populated with bios{-256k,-coreboot,-csm,-microvm,}.bin files, I wonder how is correct.

EDIT: Nvm, I think current behavior is correct, I'll provide fixes and qubes-seabios package in my PR.

@CertainLach

Copy link
Copy Markdown
Member

qubes-vmm-xen.passthru.efi has value "boot/xen-4.19.0-3.efi", but the file is in fact called "boot/xen-4.19.0.efi"

In qubes-packages PR I fix that by using

--- a/pkgs/by-name/qu/qubes-vmm-xen/package.nix
+++ b/pkgs/by-name/qu/qubes-vmm-xen/package.nix
@@ -11,7 +11,9 @@
 let
   pname = "qubes-vmm-xen";
   branch = "4.19";
-  version = "4.19.0-3";
+  versionPatches = "4.19.0";
+  versionSuffix = "3";
+  version = "${versionPatches}-${versionSuffix}";
   latest = true;
 
   xenPatches = import ../../../applications/virtualization/xen/generic/patches.nix {
@@ -43,7 +45,7 @@ let
   python = python311;
 in
 
-callPackage
+(callPackage
   (import ../../../applications/virtualization/xen/generic/default.nix {
     inherit
       pname
@@ -108,3 +110,8 @@ callPackage
     }
     // genericDefinition
   )
+).overrideAttrs (oldAttrs: {
+  passthru = oldAttrs.passthru // {
+    efi = "boot/xen-${versionPatches}.efi"
+  };
+})

@SigmaSquadron

Copy link
Copy Markdown
Contributor Author

qubes-vmm-xen.passthru.efi has value "boot/xen-4.19.0-3.efi", but the file is in fact called "boot/xen-4.19.0.efi"

oops. fixed.

@SigmaSquadron SigmaSquadron force-pushed the qubes-vmm-xen branch 2 times, most recently from 06d5bea to 40ce3b9 Compare September 15, 2024 00:40
@CertainLach

Copy link
Copy Markdown
Member

I believe seabios comment is relevant after all.

In my qubes branch, I was only able to use Xen with this argument specified:
https://github.com/NixOS/nixpkgs/pull/341215/files#diff-22074e16d7355ea382a13e722d42183c6f0655b57f7ffbe15c562d0f7977fb69R438

Comment thread pkgs/applications/qubes/qubes-vmm-xen/default.nix Outdated
Comment thread pkgs/applications/qubes/qubes-vmm-xen/default.nix Outdated
@SigmaSquadron

This comment was marked as spam.

@SigmaSquadron

This comment was marked as spam.

@SigmaSquadron SigmaSquadron reopened this Sep 26, 2024
@github-actions github-actions Bot added the 6.topic: xen-project Issues and PRs related to the Xen Project Hypervisor. label Sep 26, 2024
@SigmaSquadron

Copy link
Copy Markdown
Contributor Author

The label works!

@SigmaSquadron SigmaSquadron mentioned this pull request Sep 28, 2024
13 tasks
@github-actions github-actions Bot removed 6.topic: kernel The Linux kernel 8.has: documentation This PR adds or changes documentation 8.has: changelog This PR adds or changes release notes 8.has: module (update) This PR changes an existing module in `nixos/` 6.topic: policy discussion Discuss policies to work in and around Nixpkgs 6.topic: vim Advanced text editor 6.topic: erlang General-purpose, concurrent, functional high-level programming language 6.topic: ocaml OCaml is a general-purpose, high-level, multi-paradigm programming language. 6.topic: fetch Fetchers (e.g. fetchgit, fetchsvn, ...) 6.topic: nodejs Node.js is a free, open-source, cross-platform JavaScript runtime environment 6.topic: lua Lua is a powerful, efficient, lightweight, embeddable scripting language. 6.topic: module system About "NixOS" module system internals 6.topic: vscode A free and versatile code editor that supports almost every major programming language. 6.topic: flakes The experimental Nix feature 6.topic: lib The Nixpkgs function library labels Oct 6, 2024
@SigmaSquadron

This comment was marked as spam.

@SigmaSquadron

Copy link
Copy Markdown
Contributor Author

nixpkgs-vet passes with the changes from #345192. Waiting for OfBorg's evaluation.

@SigmaSquadron

Copy link
Copy Markdown
Contributor Author

Cool, it works! This can be merged as soon as the by-name PR is ready.

@SigmaSquadron

Copy link
Copy Markdown
Contributor Author

@CertainLach make the necessary changes to the generic builder in your branch; doing them here before #342692 is merged would break Xen's eval.

Comment thread pkgs/by-name/qu/qubes-seabios/package.nix Outdated

@CertainLach CertainLach Oct 13, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Qubes, I also add overrideAttrs with installPhase to copy qubes-vmm-stubdom binaries here, any idea on how to perform this operation in a cleaner maner?

Maybe an extra postInstall builder argument?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

qubes-vmm-xen also needs a corresponding python module

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to pick CertainLach@9f6e6ce for python module.

CertainLach and others added 2 commits November 13, 2024 18:17
Co-authored-by: Fernando Rodrigues <alpha@sigmasquadron.net>
The first (and hopefully the last) custom Xen to be built based on the
generic Xen builder. It's just a standard build of Xen with a lot of
patches.

Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net>
Comment thread pkgs/top-level/python-packages.nix Outdated
Signed-off-by: Fernando Rodrigues <alpha@sigmasquadron.net>
@SigmaSquadron

SigmaSquadron commented Nov 14, 2024

Copy link
Copy Markdown
Contributor Author

yeeeeah, nope. stubdom-bin it is. i get it why you decided to just unpack the RPM, not to mention that the kernel team will be very unhappy if we sneak in another Linux build here.

@CertainLach

Copy link
Copy Markdown
Member

Stubdom also depends on some qubes packages, and I have found a nasty dependency loop here too.

@CertainLach

Copy link
Copy Markdown
Member

For quicker iteration on qubes, I have split its packages into its own flake for now, and updated qubes-vmm-xen to 4.19.1 here: https://github.com/CertainLach/nixos-qubes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Xen support might require some patches from QubesOS for better workstation compatibility.

2 participants