Introduce OQS_MEMOPT_BUILD to enable memory‑optimized implementations

[bhess]

Introduces a OQS_MEMOPT_BUILD build option, a CMake configuration option (default OFF) that enables selection of memory-optimized algorithm implementations where available.

Motivation

Upstream implementations (such as #2361) provide alternative implementations optimized for reduced memory usage. Without a dedicated build flag, there is no way to automatically select these. This PR adds the infrastructure to let upstreams declare memory-optimized variants in their META.yml files, which liboqs will then expose and prioritize.

How it works

How it works
Upstream library META.yml files declare a memory-optimized implementation by adding memory_optimized: true to an implementation entry alongside regular variants:

implementations:  
- name: memopt    
  memory_optimized: true  
...

When copy_from_upstream processes this:

Sorting: Memory-optimized implementations are moved to the front of the implementation list, so they appear first in the generated #if/#elif dispatch chain.

CMake gating: The OQS_ENABLE_<ALG>_memopt CMake option is wrapped in if(OQS_MEMOPT_BUILD)...endif(), so it can only be switched on when OQS_MEMOPT_BUILD=ON.

Fallback: If no memopt variant exists for an algorithm, nothing changes, and OQS_MEMOPT_BUILD has no effect. If one exists but OQS_MEMOPT_BUILD=OFF, the standard implementation is used.

The PR also modifies the CBOM generation to avoid 'bom-ref' collision if there are multiple generic implementations.

• Does this PR change the input/output behaviour of a cryptographic algorithm (i.e., does it change known answer test values)? (If so, a version bump will be required from x.y.z to x.(y+1).0.)
• Does this PR change the list of algorithms available -- either adding, removing, or renaming? Does this PR otherwise change an API? (If so, PRs in fully supported downstream projects dependent on these, i.e., oqs-provider will also need to be ready for review and merge by the time this is merged. Also, make sure to update the list of algorithms in the continuous benchmarking files: .github/workflows/kem-bench.yml and sig-bench.yml)

Documentation and descriptive text assisted with AI.

[coveralls]

coverage: 83.03%. remained the same
when pulling 099e189 on bhe-memopt-build
into 6b390dd on main.

[bhess]

Marking this ready for review. #2361 currently builds on top of this PR, making use of the memory_optimized flag and can be seen as a test case.

[xuganyu96]

The individual suggestions are all cosmetic/readability recommendations, though I have some concerns with how this feature is designed.

Based on prior discussion, my understanding is MEM_OPT is needed because MQOM has memory-optimized implementation. For just one signature scheme, introducing a global build flag seems like overkill. Is it easier instead, to include the memory-optimized implementation as "just another implementation" that can be enabled/disabled in the same way x86_64/aarch64-optimized builds are enabled/disabled?

[bhess]

Thanks for your review @xuganyu96.

Based on prior discussion, my understanding is MEM_OPT is needed because MQOM has memory-optimized implementation. For just one signature scheme, introducing a global build flag seems like overkill. Is it easier instead, to include the memory-optimized implementation as "just another implementation" that can be enabled/disabled in the same way x86_64/aarch64-optimized builds are enabled/disabled?

Valid point. I deliberately made this a more generic feature with respect to mlkem-native/mldsa-native, which will also introduce a memory‑optimized variant. This ensures that the MEMOPT feature is available there as well. I expect users on memory‑constrained devices to be particularly interested in such a configuration option.

[baentsch]

Thanks for your review @xuganyu96.

Based on prior discussion, my understanding is MEM_OPT is needed because MQOM has memory-optimized implementation. For just one signature scheme, introducing a global build flag seems like overkill. Is it easier instead, to include the memory-optimized implementation as "just another implementation" that can be enabled/disabled in the same way x86_64/aarch64-optimized builds are enabled/disabled?

Valid point. I deliberately made this a more generic feature with respect to mlkem-native/mldsa-native, which will also introduce a memory‑optimized variant. This ensures that the MEMOPT feature is available there as well. I expect users on memory‑constrained devices to be particularly interested in such a configuration option.

The thought voiced by @xuganyu96 was exactly the reason for my comment in a call a few weeks back why this isn't just (introduced as yet) another variant (of the one specific algorithm having this feature). The explanation by @bhess of course now explains why a more generic approach makes sense.

Can I suggest to introduce such new generic facilities via a new issue, though, to enable broader discussion/information dissemination? This would make clear the benefits and also allow us to communicate this upstream (to other algorithm providers to invite them to also make such options available) and downstream (to users to inform them about this optimization option -- and of course document which algorithms benefit from this). Otherwise this sounds a bit like an afterthought meant to benefit only those algorithms we happen to have more information about than others.

More generally, this was the thing that originally attracted me to OQS: The team's willingness to support the broader PQC algorithm base, the identical treatment of all algorithms. I feel the project deviating from this "impartiality" and would like to invite everyone to help keep this in the back of their minds doing new things.

[bhess]

Can I suggest to introduce such new generic facilities via a new issue, though, to enable broader discussion/information dissemination? This would make clear the benefits and also allow us to communicate this upstream (to other algorithm providers to invite them to also make such options available) and downstream (to users to inform them about this optimization option -- and of course document which algorithms benefit from this). Otherwise this sounds a bit like an afterthought meant to benefit only those algorithms we happen to have more information about than others.

Good point, and apologies for the oversight, I was focused on keeping the momentum and unblock #2361. I've opened #2371 for that broader discussion and ensure these facilities are properly documented for everyone.

[xuganyu96]

I am still confused about the relationship between the global MEMOPT flag and individual memory-optimized builds. Will all schemes switch to memory-optimized builds when the global flag is turned on? Will users have the option to choose non-memory-optimized build for one scheme and a memory-optimized build for another scheme in the same build?

Also, I assume that the relevant code changes will be applied to kem.c once memory-optimized ML-KEM from mlkem-native ships?

[bhess]

I am still confused about the relationship between the global MEMOPT flag and individual memory-optimized builds. Will all schemes switch to memory-optimized builds when the global flag is turned on? Will users have the option to choose non-memory-optimized build for one scheme and a memory-optimized build for another scheme in the same build?

Also, I assume that the relevant code changes will be applied to kem.c once memory-optimized ML-KEM from mlkem-native ships?

Here's the intended behavior: When the global OQS_MEMOPT_BUILD flag is ON, it sets the default for all schemes, selecting the memory-optimized variant wherever one is provided. I expect this is the primary use case: on memory-constrained platforms, users generally want all schemes to minimize their footprint by default.

However, users can still mix. For example, one could set the global flag to ON but explicitly set -DOQS_ENABLE_SIG_<scheme>_MEMOPT=OFF to use the standard build for a specific algorithm.

Regarding KEMs: yes, the same logic will apply. Once a memory-optimized ML-KEM is available in mlkem-native.

[baentsch]

LGTM, Thanks @bhess! Documentation is clear & testing is added. Only nits: 1) Worthwhile pointing to this facility in https://github.com/open-quantum-safe/liboqs/wiki/Contributing-guide (when landed) such as for upstreams to know/read about this facility? 2) New test takes 18mins, i.e., more than the targeted 15.

[bhess]

2. New test takes 18mins, i.e., more than the targeted 15.

I noticed the test always runs in PRs, even though it's technically part of the [full tests] suite where we allow up to 30 minutes. It seems the PR workflows always trigger "platform-tests" by default, whereas only the push tests actually check for the [full tests] string in the commit message.

Should we update the PR tests to also require this tag so we stay within the 15-minute target for standard runs?

[baentsch]

Should we update the PR tests to also require this tag so we stay within the 15-minute target for standard runs?

Conceptually an OK approach but this creates the risk that such tests don't get run at all then. They then at least should be triggered at merge -- albeit that may be too late if they'd discover a problem...

Just thinking out loud: What about auto-triggering [full tests] after a first positive review has been provided? That would take them off the critical path, not run them too often, nor too rarely.

[bhess]

Should we update the PR tests to also require this tag so we stay within the 15-minute target for standard runs?

Conceptually an OK approach but this creates the risk that such tests don't get run at all then. They then at least should be triggered at merge -- albeit that may be too late if they'd discover a problem...

Just thinking out loud: What about auto-triggering [full tests] after a first positive review has been provided? That would take them off the critical path, not run them too often, nor too rarely.

I've created an issue to track this: #2375

Given the approvals I'd go ahead merging this PR unless there are any objections. Will update https://github.com/open-quantum-safe/liboqs/wiki/Contributing-guide once merged.