dump-ast example program hits assertion: `!PA.isArgIdent(Idx)' failed.

[seelabs]

I read about pasta on your blog post and decide to play with it. I am not familiar with the internals of libclang or llvm, so this issue may be user error.

I successfully built pasta and was able to run the demo programs. When I tried the dump-ast example on a toy example it worked as expected. However, when I tried it against my real project, it hits an assert. In particular, when I run it against this project: https://github.com/XRPLF/rippled using the following command line:

./dump-ast /home/swd/apps/clang-latest/bin/clang++ -DBOOST_ASIO_DISABLE_HANDLER_TYPE_REQUIREMENTS -DBOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT -DBOOST_BEAST_ALLOW_DEPRECATED -DBOOST_CONTAINER_FWD_BAD_DEQUE -DBOOST_COROUTINES_NO_DEPRECATION_WARNING -DBOOST_FILESYSTEM_DEPRECATED -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=\\\"/usr/bin/addr2line\\\" -DBOOST_STACKTRACE_USE_ADDR2LINE -DBOOST_STACKTRACE_USE_BACKTRACE -DBOOST_STACKTRACE_USE_NOOP -DCARES_STATICLIB -DDATE_HEADER_ONLY -DDEBUG -DENABLE_TESTS -DGIT_COMMIT_HASH=\\\"ed9064ce3f6de6da6b63417e59f3100c8e6c4c6d\\\" -DHAS_UNCAUGHT_EXCEPTIONS=1 -DOPENSSL_NO_SSL2 -D_DEBUG -I/home/swd/projs/ripple/mine/src/ed25519-donna -I/home/swd/projs/ripple/mine/src/secp256k1/include -I/home/swd/projs/ripple/mine/src -isystem /home/swd/.conan/data/boost/1.82.0/_/_/package/f81afbcfa21cf6994cf0e45c1b11229c01c767a7/include -isystem /home/swd/.conan/data/openssl/1.1.1u/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/libarchive/3.6.2/_/_/package/d9fea6983af2a77bcbe3f0b088cea5aef9308bd7/include -isystem /home/swd/.conan/data/lz4/1.9.3/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen_grpc -isystem /home/swd/.conan/data/grpc/1.50.1/_/_/package/d156c58fd512defbc1155e665684d7875e800ba9/include -isystem /home/swd/.conan/data/abseil/20220623.0/_/_/package/e84b6d643e7f97d33bf72700134618fbae8bbbf6/include -isystem /home/swd/.conan/data/c-ares/1.19.0/_/_/package/ed86cef4487c6b9ce327a03d24ced8142000e8b9/include -isystem /home/swd/.conan/data/re2/20230301/_/_/package/e84b6d643e7f97d33bf72700134618fbae8bbbf6/include -isystem /home/swd/.conan/data/zlib/1.2.13/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/protobuf/3.21.9/_/_/package/681b3b1dffc7e4dc6f5551cdb6e1fe56bdda3e85/include -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen -isystem /home/swd/projs/ripple/mine/cur_build/proto_gen/src/ripple/proto -isystem /home/swd/.conan/data/soci/4.0.3/_/_/package/dc82e3681e64ed36c447531c9e4883a5c616a774/include -isystem /home/swd/.conan/data/bzip2/1.0.8/_/_/package/5e357164f23f7c4e38ebe87433827878dbe1a74c/include -isystem /home/swd/.conan/data/libbacktrace/cci.20210118/_/_/package/06b9cd1c4dccbb163a99056eda646b0e168548fc/include -isystem /home/swd/.conan/data/sqlite3/3.42.0/_/_/package/ae0e146c4c82ac1e46c279d15c964c80fedbdf5f/include -isystem /home/swd/.conan/data/nudb/2.0.8/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include -isystem /home/swd/.conan/data/date/3.0.1/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include -m64 -stdlib=libstdc++ -Wall -Wdeprecated -Wextra -Wno-unused-parameter -g -std=c++20 -fPIE -frtti -Wnon-virtual-dtor -Wno-sign-compare -Wno-char-subscripts -Wno-format -Wno-unused-local-typedefs -fstack-protector -o CMakeFiles/rippled.dir/src/ripple/app/ledger/impl/LedgerReplay.cpp.o -c /home/swd/projs/ripple/mine/src/ripple/app/ledger/impl/LedgerReplay.cpp

It hits the following assert:

dump-ast: /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:8299: void HandleUnkownTypeAttrAsAnnotateTypeAttr({anonymous}::TypeProcessingState&, clang::QualType&, const clang::ParsedAttr&): Assertion `!PA.isArgIdent(Idx)' failed.
[1]    313418 IOT instruction (core dumped)  ./dump-ast /home/swd/apps/clang-latest/bin/clang++           -DCARES_STATICLI

I attempted to debug this. I built the debug version of the libraries and went into gdb and attempted to try to find the offending code so I could create a minimal repro, but unfortunately my unfamiliarity with llvm is an obstacle. Two things we may be able to use as staring points.

In SemaType.cpp:8299

p Str
$8 = {
  Data = 0x5555a3a89598 "__access__",
  Length = 10
}

So the offending attribute is called "access".

In ParseDeclCXX.cpp:2107

p PP.getSourceManager().getPresumedLoc(StartLoc,true)
$6 = {
  Filename = 0x5555a18575d0 "<pasta-input>",
  ID = {
    ID = 851074
  },
  Line = 1716508,
  Col = 1,
  IncludeLoc = {
    ID = 0
  }
}

What I'm trying to do here is find the name of the file that contains the offending code. Some searching pointed me to getPresumedLoc, but clearly <pasta-input> isn't useful to find the offending code. Perhaps you have a suggestion to find the code?

Also, I should mention I was only playing with pasta. Pasta obviously works and it's very possible I'm "holding it wrong". I'm not expecting help here, but thought I'd report the issue in case it was a possible bug in pasta (and will help trying to debug it after regular work hours if it's helpful to you).

[pgoodman]

<pasta-input> is a synthetic input file that clang's parser operates on. PASTA first runs an instrumented preprocessor on the input, and in the process of this, forms <pasta-input>, which is just a very large std::string in memory. Each unique token is rendered onto a different line of this input file, so that a source location's line number acts as a kind of side-channel for a token index. Previously, I had attempted to patch Clang to embed more information into SourceLocations, but this became problematic as the underlying integral representation of SourceLocations is often stored directly in AST nodes. In retrospect, I regret the choice to use line numbers as a side-channel, as SourceLocations are basically byte offsets, and so I could have just used the byte offset itself with a list of "end of token bytes offsets", and then some binary searching. Oh well.

Anyway, probably you aren't holding it wrong. The reality is just that PASTA does some sketchy things to achieve its goals. Here's how I would proceed:

Patch the following code into lib/Compile/Run.cpp, around line 353:

  auto fd = open("/tmp/source.cpp", O_TRUNC | O_CREAT | O_WRONLY, 0666);
  write(fd, ast->preprocessed_code.data(), ast->preprocessed_code.size());
  close(fd);

Then build and run, and attach /tmp/source.cpp here. Another thing that could help is seeing the backtrace.

[seelabs]

Thanks for the explanation and next steps. The "source.cpp" is too large to paste here, so I created a repo here: https://github.com/seelabs/pasta-debugging/tree/main. "source.cpp" is here: https://github.com/seelabs/pasta-debugging/blob/main/source.cpp and the gdb transcript is here: https://github.com/seelabs/pasta-debugging/blob/main/gdb.txt with both bt full and bt. I'll paste bt here since it's small:

#0  HandleUnkownTypeAttrAsAnnotateTypeAttr (State=..., CurType=..., PA=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:8299
#1  0x0000555559ed322d in processTypeAttrs (state=..., type=..., TAL=TAL_DeclName, attrs=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:8413
#2  0x0000555559ec608b in GetFullTypeForDeclarator (state=..., declSpecType=..., TInfo=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:5810
#3  0x0000555559ec6d41 in clang::Sema::GetTypeForDeclarator (this=0x5555a18ddbf0, D=..., S=0x55556d734890) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaType.cpp:5935
#4  0x000055555909d929 in clang::Sema::HandleDeclarator (this=0x5555a18ddbf0, S=0x55556d734890, D=..., TemplateParamLists=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaDecl.cpp:6304
#5  0x00005555591ed58d in clang::Sema::ActOnCXXMemberDeclarator (this=0x5555a18ddbf0, S=0x55556d734890, AS=clang::AS_public, D=..., TemplateParameterLists=..., BW=0x0, VS=..., InitStyle=clang::ICIS_NoInit) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Sema/SemaDeclCXX.cpp:3510
#6  0x00005555589d2f3c in clang::Parser::ParseCXXInlineMethodDef (this=0x55559f6c1e60, AS=clang::AS_public, AccessAttrs=..., D=..., TemplateInfo=..., VS=..., PureSpecLoc=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseCXXInlineMethods.cpp:42
#7  0x0000555558a18a29 in clang::Parser::ParseCXXClassMemberDeclaration (this=0x55559f6c1e60, AS=clang::AS_public, AccessAttrs=..., TemplateInfo=..., TemplateDiags=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2956
#8  0x0000555558a1ac52 in clang::Parser::ParseCXXClassMemberDeclarationWithPragmas (this=0x55559f6c1e60, AS=@0x7fffffff8598: clang::AS_public, AccessAttrs=..., TagType=clang::TST_struct, TagDecl=0x5555a64f2098) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3391
#9  0x0000555558a1baee in clang::Parser::ParseCXXMemberSpecification (this=0x55559f6c1e60, RecordLoc=..., AttrFixitLoc=..., Attrs=..., TagType=25, TagDecl=0x5555a64f2098) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3595
#10 0x0000555558a1506a in clang::Parser::ParseClassSpecifier (this=0x55559f6c1e60, TagTokKind=clang::tok::kw_struct, StartLoc=..., DS=..., TemplateInfo=..., AS=clang::AS_private, EnteringContext=true, DSC=clang::Parser::DeclSpecContext::DSC_class, Attributes=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2107
#11 0x00005555589ee713 in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_private, DSContext=clang::Parser::DeclSpecContext::DSC_class, LateAttrs=0x7fffffff9540, AllowImplicitTypename=clang::ImplicitTypenameContext::Yes) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDecl.cpp:4220
#12 0x00005555589cc6cb in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_private, DSC=clang::Parser::DeclSpecContext::DSC_class, LateAttrs=0x7fffffff9540) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/include/clang/Parse/Parser.h:2421
#13 0x0000555558a18209 in clang::Parser::ParseCXXClassMemberDeclaration (this=0x55559f6c1e60, AS=clang::AS_private, AccessAttrs=..., TemplateInfo=..., TemplateDiags=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2803
#14 0x0000555558a1ac52 in clang::Parser::ParseCXXClassMemberDeclarationWithPragmas (this=0x55559f6c1e60, AS=@0x7fffffffa8c8: clang::AS_private, AccessAttrs=..., TagType=clang::TST_class, TagDecl=0x5555a64eae48) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3391
#15 0x0000555558a1baee in clang::Parser::ParseCXXMemberSpecification (this=0x55559f6c1e60, RecordLoc=..., AttrFixitLoc=..., Attrs=..., TagType=26, TagDecl=0x5555a64eae48) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:3595
#16 0x0000555558a1506a in clang::Parser::ParseClassSpecifier (this=0x55559f6c1e60, TagTokKind=clang::tok::kw_class, StartLoc=..., DS=..., TemplateInfo=..., AS=clang::AS_none, EnteringContext=true, DSC=clang::Parser::DeclSpecContext::DSC_top_level, Attributes=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:2107
#17 0x00005555589ee713 in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_none, DSContext=clang::Parser::DeclSpecContext::DSC_top_level, LateAttrs=0x0, AllowImplicitTypename=clang::ImplicitTypenameContext::Yes) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDecl.cpp:4220
#18 0x00005555589cc6cb in clang::Parser::ParseDeclarationSpecifiers (this=0x55559f6c1e60, DS=..., TemplateInfo=..., AS=clang::AS_none, DSC=clang::Parser::DeclSpecContext::DSC_top_level, LateAttrs=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/include/clang/Parse/Parser.h:2421
#19 0x00005555589c11f8 in clang::Parser::ParseDeclOrFunctionDefInternal (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=..., AS=clang::AS_none) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:1119
#20 0x00005555589c18e6 in clang::Parser::ParseDeclarationOrFunctionDefinition (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=0x0, AS=clang::AS_none) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:1226
#21 0x00005555589c0c1f in clang::Parser::ParseExternalDeclaration (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:1041
#22 0x0000555558a0cfe8 in clang::Parser::ParseInnerNamespace (this=0x55559f6c1e60, InnerNSs=..., index=0, InlineLoc=..., attrs=..., Tracker=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:261
#23 0x0000555558a0cc6e in clang::Parser::ParseNamespace (this=0x55559f6c1e60, Context=clang::DeclaratorContext::File, DeclEnd=..., InlineLoc=...) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDeclCXX.cpp:238
#24 0x00005555589e4dad in clang::Parser::ParseDeclaration (this=0x55559f6c1e60, Context=clang::DeclaratorContext::File, DeclEnd=..., DeclAttrs=..., DeclSpecAttrs=..., DeclSpecStart=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/ParseDecl.cpp:1807
#25 0x00005555589c070a in clang::Parser::ParseExternalDeclaration (this=0x55559f6c1e60, Attrs=..., DeclSpecAttrs=..., DS=0x0) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:957
#26 0x00005555589bfa16 in clang::Parser::ParseTopLevelDecl (this=0x55559f6c1e60, Result=..., ImportState=@0x7fffffffc6c8: clang::Sema::ModuleImportState::NotACXX20Module) at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/Parse/Parser.cpp:743
#27 0x00005555562df830 in pasta::CompileJob::Run[abi:cxx11]() const (this=0x555561541be0) at /home/swd/projs/common/pasta/lib/Compile/Run.cpp:425
#28 0x0000555555ef7b59 in main (argc=81, argv=0x7fffffffcd38) at /home/swd/projs/common/pasta/bin/DumpAST/Main.cpp:135

I also looked at source.cpp around line 1716508. Here's what appears there:

struct
_Stop_state_ref
{
_Stop_state_ref
(
)
=
default
;
[
[
__gnu__
::
__access__
(
__none__
,
2
)
]
]

[seelabs]

@pgoodman I found a simple way to reproduce the error.

1. Create a file called "repo.cpp" with one line #include <stop_token>
2. run ./dump-ast g++ -std=c++20 repo.cpp This will hit the assert above
3. run ./dump-ast g++ repo.cpp This will work as expected (note: no `-std=c++20)

So the culprit is -std=c++20.

I'll note that the problem is not specific to stop_token. Including just optional will also exhibit this behavior, but a simple include like cstdint will run as expected. (I had tested using stop_token because it was the header implicated in the above source.cpp)

[pgoodman]

Okay this is a bug in one of our patches: HandleUnkownTypeAttrAsAnnotateTypeAttr.

We've made this patch with the intent of implicitly converting things like __attribute__((hello_world("a"))) into __attribute__((annotate("hello_world", "a"))), though there are clearly some corner cases that are triggering assertions. I don't have the spare hours right now to fix this bug in our patches. Can you try mitigating it with the following change: Change this true to false: https://github.com/trailofbits/pasta/blob/master/lib/Compile/Run.cpp#L242-L244

[seelabs]

Thanks! I made the suggested change and that successfully ran against the minimal repro. I then tried against my real codebase and got:

A clang diagnostic or uncompilable error was produced when trying to get an AST due to error: /usr/lib/gcc/x86_64-pc-linux-gnu/13.1.1/include/xmmintrin.h:136:19 use of undeclared identifier '__builtin_ia32_addss'

I don't know if that's related to disabling the "unknown attribute" patch or not. (I know you don't have time to dig into this now, I just thought I'd report it anyway).

[pgoodman]

The missing builtin looks to be a GCC-specific intrinsic that Clang doesn't seem to have. There are a number of others that are related, and could be harvested from this list. Fixing this involves adding entries into this file. This mirrors how Clang adds them, but we do it retroactively and in a slightly hacky way that generally gets the job done without having to patch Clang.

Could you try making the following additions to our BuiltinsX86.h file, and testing them out? If they work, please submit a PR :-)

TARGET_BUILTIN("__builtin_ia32_comieq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comineq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comilt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comile", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comigt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_comige", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomieq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomineq", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomilt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomile", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomigt", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_ucomige", "iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_addps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_subps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_mulps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_divps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_addss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_subss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_mulss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_divss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpeqps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpltps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpleps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpgtps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpgeps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpunordps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpneqps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnltps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnleps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpngtps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpngeps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpordps", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpeqss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpltss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpless", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpunordss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpneqss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnlts", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpnless", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cmpordss", "V4iV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_maxps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_maxss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_minps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_minss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_andps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_andnps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_orps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_xorps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movss", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movhlps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movlhps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_unpckhps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_unpcklps", "V4fV4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtpi2ps", "V4fV4fV2i", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtsi2ss", "V4fV4f, i", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtps2pi", "V2iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvtss2si", "iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvttps2pi", "V2iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_cvttss2si", "iV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rcpps", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rsqrtps", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_sqrtps", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rcpss", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_rsqrtss", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_sqrtss", "V4fV4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_shufps", "V4fV4fV4f, i", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movntps", "vf*V4f", "", "sse")
TARGET_BUILTIN("__builtin_ia32_movmskps", "iV4f", "", "sse")

[seelabs]

I seem to be getting the same error as before. I modified the file pasta/lib/Compile/BuiltinsX86.h by adding the code above, recompiled, and re-ran. I also did a "make clean" and recompiled to double check. (as a dummy-check, here's my edited file as a gist: https://gist.github.com/seelabs/9a63d38260caf1ddce7bf7cdf043f8ae)

[pgoodman]

Oh I'm a dummy. I shouldn't have wrapped all those builtin names with double quotes. Can you try with this?

TARGET_BUILTIN(__builtin_ia32_comieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnlts, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andnps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_orps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_xorps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movhlps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movlhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpckhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpcklps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtpi2ps, "V4fV4fV2i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtsi2ss, "V4fV4f, i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_shufps, "V4fV4fV4f, i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movntps, "vf*V4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movmskps, "iV4f", "", "sse")

[seelabs]

Still no go, but a different error:

Unknown builtin type letter!
UNREACHABLE executed at /home/swd/projs/common/cxx-common-debug/vcpkg/buildtrees/llvm-16/src/org-16.0.5-90a3d25b70.clean/clang/lib/AST/ASTContext.cpp:11250!

I went into gdb and looked at the offending Str parameter to DecodeTypeFromStr. It was: ", i". So it's choking on the ,. I don't have time to dig into this right now, but I'll try to look later and see if I can locate the offending code.

[seelabs]

From our dumped source.cpp, it's pointing to this as the culprit:

extern
__inline
__m128
__attribute__
(
(
__gnu_inline__
,
__always_inline__
,
__artificial__
)
)
_mm_cvtsi32_ss
(
__m128
__A
,
int
__B
)
{
return
(
__m128
)
__builtin_ia32_cvtsi2ss
(
(
__v4sf
)
__A
,
__B
)
;
}

The actual line it's pointing at is the one with __builtin_ia32_cvtsi2ss, but given the value of Str I'm guessing we should be looking at:

__m128
__A
,
int

[pgoodman]

Try this?

TARGET_BUILTIN(__builtin_ia32_comieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_comige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomieq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomineq, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomilt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomile, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomigt, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_ucomige, "iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_addss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_subss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_mulss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_divss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpgeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnltps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnleps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngtps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpngeps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordps, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpeqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpltss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpunordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpneqss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnlts, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpnless, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cmpordss, "V4iV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_maxss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_minss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_andnps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_orps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_xorps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movss, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movhlps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movlhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpckhps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_unpcklps, "V4fV4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtpi2ps, "V4fV4fV2i", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtsi2ss, "V4fV4fi", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvtss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttps2pi, "V2iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_cvttss2si, "iV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtps, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rcpss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_rsqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_sqrtss, "V4fV4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_shufps, "V4fV4fV4fi", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movntps, "vf*V4f", "", "sse")
TARGET_BUILTIN(__builtin_ia32_movmskps, "iV4f", "", "sse")

[seelabs]

No go, but a different error:

A clang diagnostic or uncompilable error was produced when trying to get an AST due to error: /usr/lib/gcc/x86_64-pc-linux-gnu/13.1.1/include/xmmintrin.h:624:19 use of undeclared identifier '__builtin_ia32_cvtsi642ss'

[pgoodman]

Getting closer :-) Seems like we're missing that intrinsic too.

[seelabs]

Just a quick update: I'm attempting to add the needed intrinsics. I'm having trouble finding the documentation I need for these missing functions (the gcc manual doesn't seem to have them) and also the correct strings for the types. I'll spend time after work to poke at this, but it may be a while before I'm able to make a patch (but hopefully I'll learn something along the way).

[pgoodman]

Oh I'm making a script for this! I'll be bulk grabbing them from CBMC, which has a lot of forward declarations. It turns out there is some other stuff in the Python API that has required fixing in order for me to make the script, so it's taking me a bit of time :-)