Add MQOM to liboqs

[rben-dev]

NOTE: This PR is a clean reopening of closed PR #2361. Please refer to it for the details of the conversation and the related PRs (adding the memopt variants in liboqs, fixing Zephyr build, adding a way to import common files with copy_from_upstream).

This PR adds the MQOM signature scheme. MQOM is part of on-ramp Round 2 NIST candidates, In order to avoid too many variants that would overload the library, we only provide implementations for GF(16) MQOM fast and short (with 3 or 5 rounds), as this is the best choice for a speed and signature size balance.

Three implementation profiles are integrated to liboqs:

• AVX2: this makes use of AVX2 and AES-NI extensions, suited for modern amd64 platforms.
• default: this is a portable implementation suitable for most desktop-like platforms (e.g. arm64 Mac M1, etc.)
• memopt: this is a portable implementation with memory optimizations that are more suitable for platforms with stringent memory constraints (e.g. embedded platforms or targets with small stacks, etc.)

Here are some notable elements for this MQOM integration:

• MQOM uses Keccak, and we use the liboqs API for this.
• MQOM in security level 1 uses AES-128, and we use liboqs API for this. However, for security levels 3 and 5 Rijndael-256-256 is used, and hence we had to provide local implementations for this (see in the rijndael/ folder upstream).
• MQOM uses a large stack in the AVX2 and default implementations:
  • We had to deactivate the threaded signature tests in tests/test_sig.c. This is particularly useful for Mac OS platforms where the threads stacks is quite small.
  • We also had to adapt the tests/test_leaks.py file by providing --max-stackframe=20480000 to valgrind as discussed in issue Maximum stack usage for valgrind "leak test" #2360 : without this the CI tests fail with false positives. The strategy for this fix can be of course discussed. The fix is part of the PR.

Many elements of this PR have been produces with the copy_from_upstream.py automation (thanks for this).

Thanks in advance for considering the inclusion of MQOM in the library, as we believe this can be useful for exploring on-ramp candidates. Please do not hesitate if anything is wrong or missing!

• Does this PR change the input/output behaviour of a cryptographic algorithm (i.e., does it change known answer test values)? (If so, a version bump will be required from x.y.z to x.(y+1).0.)
• Does this PR change the list of algorithms available -- either adding, removing, or renaming? Does this PR otherwise change an API? (If so, PRs in fully supported downstream projects dependent on these, i.e., oqs-provider will also need to be ready for review and merge by the time this is merged. Also, make sure to update the list of algorithms in the continuous benchmarking files: .github/workflows/kem-bench.yml and sig-bench.yml)

[rben-dev]

@xuganyu96: as discussed in closed PR #2361, reopening a new PR for a clean import.
Only rebased on PR #2382 as it is needed for the common files factorization.

[xuganyu96]

@bhess I see that this pull request also contains diff's from your pull request. Since your code change is specifically made to unblock MQOM integration, would you be okay with merging all changes in this single pull request rather than two separate pull requests?

@rben-dev Thank you for the contribution!

[bhess]

LGTM

@bhess I see that this pull request also contains diff's from your pull request. Since your code change is specifically made to unblock MQOM integration, would you be okay with merging all changes in this single pull request rather than two separate pull requests?

Sure, happy to go with whatever is easiest.

[rben-dev]

Thanks for the reviews @xuganyu96 @bhess, and thanks for all the PRs helping this integration @bhess!

[dstebila]

@bhess Sorry to bother you, I just merged the strnup changes #2356 and was going to rebase this PR for a final CI run before merging, but there are cbom conflicts. Would you be able to take a look?

[bhess]

Re-approving after resolving CBOM conflict.