Clang segfaults when compiling boost::rtree for x86_64 (r14, r14b, r15 beta1)

[kkaefer]

I'm observing a Clang segfault when compiling boost::rtree (from the Geometry package).

  FAILED: /Users/kkaefer/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++  --target=x86_64-none-linux-android --gcc-toolchain=/Users/kkaefer/Library/Android/sdk/ndk-bundle/toolchains/x86_64-4.9/prebuilt/darwin-x86_64 --sysroot=/Users/kkaefer/Library/Android/sdk/ndk-bundle/platforms/android-21/arch-x86_64  -Dnative_lib_EXPORTS -isystem ../../../../libs/boost_1_63_0 -isystem /Users/kkaefer/Library/Android/sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/include -isystem /Users/kkaefer/Library/Android/sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/x86_64/include -isystem /Users/kkaefer/Library/Android/sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/include/backward -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security  -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security  -std=c++11 -frtti -fexceptions -O2 -DNDEBUG -O2 -DNDEBUG  -fPIC -MD -MT CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o -MF CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o.d -o CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o -c /Users/kkaefer/Code/android-boost-rtree/app/src/main/cpp/native-lib.cpp
  clang++: error: unable to execute command: Segmentation fault: 11
  clang++: error: clang frontend command failed due to signal (use -v to see invocation)
  Android clang version 3.8.275480  (based on LLVM 3.8.275480)
  Target: x86_64-none-linux-android
  Thread model: posix
  InstalledDir: /Users/kkaefer/Library/Android/sdk/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin
  clang++: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
  clang++: note: diagnostic msg:
  ********************

  PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
  Preprocessed source(s) and associated run script(s) are located at:
  clang++: note: diagnostic msg: /var/folders/d9/49s5f6ln6qx7myj13xw4gztc0000gn/T/native-lib-4dc9cf.cpp
  clang++: note: diagnostic msg: /var/folders/d9/49s5f6ln6qx7myj13xw4gztc0000gn/T/native-lib-4dc9cf.sh
  clang++: note: diagnostic msg:

  ********************

This crash only occurs for x86_64, and only in Release mode (-O2 and higher).

I tested r14, r14b, and r15beta1 and they all suffer from this issue, since they all use the same compiler build Android clang version 3.8.275480 (based on LLVM 3.8.275480). The last good version is r13b, which is using Android clang version 3.8.256229 (based on LLVM 3.8.256229).

A minimal example is at https://github.com/kkaefer/android-boost-rtree with the code that triggers the segfault being

  using Point = boost::geometry::model::point<double, 2, boost::geometry::cs::cartesian>;
  using RTree = boost::geometry::index::rtree<Point, boost::geometry::index::rstar<16, 4>>;
  RTree rtree;
  rtree.insert(Point(2, 3));

(https://github.com/kkaefer/android-boost-rtree/blob/master/app/src/main/cpp/native-lib.cpp#L8-L12)

[kkaefer]

Here's the crash in our CI build, using r15beta1: https://circleci.com/gh/mapbox/mapbox-gl-native/718

[kkaefer]

And here's the LLVM issue I filed a while ago: https://bugs.llvm.org/show_bug.cgi?id=32330

[enh]

thanks. probably always worth filing a bug here in addition to the llvm one so that @pirama-arumuga-nainar and @stephenhines can help chase down the llvm bug.

[pirama-arumuga-nainar]

I can verify that this happens in the newer compiler in AOSP as well. However, it does not reproduce in upstream ToT.

The backtrace for AOSP failure is below. We can try a debug build to get more info and then search for a possible upstream revision that fixes this.

#0 0x00000000022e4b93 in llvm::TargetLowering::makeLibCall(llvm::SelectionDAG&, llvm::RTLIB::Libcall, llvm::EVT, llvm::ArrayRefllvm::SDValue, bool, llvm::SDLoc const&, bool, bool) const ()
#1 0x00000000021f825f in ?? ()
#2 0x00000000021f5218 in ?? ()
#3 0x000000000221dd45 in ?? ()
#4 0x00000000022222ba in llvm::SelectionDAG::LegalizeTypes() ()
#5 0x00000000022d258e in llvm::SelectionDAGISel::CodeGenAndEmitDAG() ()
#6 0x00000000022d19b2 in llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) ()
#7 0x00000000022cfce7 in llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) ()
#8 0x0000000001e06eb1 in ?? ()
#9 0x0000000002502ba4 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) ()
#10 0x0000000002b0a163 in llvm::FPPassManager::runOnFunction(llvm::Function&) ()
#11 0x0000000002b0a363 in llvm::FPPassManager::runOnModule(llvm::Module&) ()
#12 0x0000000002b0a7bf in llvm::legacy::PassManagerImpl::run(llvm::Module&) ()
#13 0x0000000000dc3b6f in clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::__1::unique_ptr<llvm::raw_pwrite_stream, std::__1::default_deletellvm::raw_pwrite_stream >) ()
#14 0x0000000000f35154 in clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) ()
#15 0x0000000000ff1da6 in clang::ParseAST(clang::Sema&, bool, bool) ()
#16 0x0000000000f339a4 in clang::CodeGenAction::ExecuteAction() ()
#17 0x0000000000a638bd in clang::FrontendAction::Execute() ()
#18 0x0000000000a28f07 in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) ()
#19 0x00000000009e93dd in clang::ExecuteCompilerInvocation(clang::CompilerInstance*) ()
#20 0x00000000009df8d6 in cc1_main(llvm::ArrayRef<char const*>, char const*, void*) ()
#21 0x00000000009e726e in main ()

[pirama-arumuga-nainar]

Here's the backtrace from a debug clang. It's related to 128-bit long double but I don't see any changes to upstream that may be fixing this assert.

#4 0x0000000006ec0625 in llvm::DAGTypeLegalizer::GetSoftenedFloat (
this=0x7fffffff2258, Op=...)
at external/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h:408
#5 0x0000000006ead744 in llvm::DAGTypeLegalizer::SoftenFloatRes_FMUL (
this=0x7fffffff2258, N=0x24383bd0)
at external/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp:411
#6 0x0000000006ea92c9 in llvm::DAGTypeLegalizer::SoftenFloatResult (
this=0x7fffffff2258, N=0x24383bd0, ResNo=0)
at external/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp:91
#7 0x0000000006ef2311 in llvm::DAGTypeLegalizer::run (this=0x7fffffff2258)
at external/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp:241
#8 0x0000000006ef8f1e in llvm::SelectionDAG::LegalizeTypes (this=0x22d76790)
at external/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.cpp:1175
#9 0x00000000070c536e in llvm::SelectionDAGISel::CodeGenAndEmitDAG (
this=0x228932b0)
at external/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:773
#10 0x00000000070c3b67 in llvm::SelectionDAGISel::SelectBasicBlock (
this=0x228932b0, Begin=..., End=..., HadTailCall=@0x7fffffff4dff: false)
at external/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:683
#11 0x00000000070c368f in llvm::SelectionDAGISel::SelectAllBasicBlocks (
this=0x228932b0, Fn=...)
at external/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:1551
#12 0x00000000070c0863 in llvm::SelectionDAGISel::runOnMachineFunction (
this=0x228932b0, mf=...)
at external/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:504
#13 0x000000000625920e in (anonymous namespace)::X86DAGToDAGISel::runOnMachineFunction (this=0x228932b0, MF=...)
at external/llvm/lib/Target/X86/X86ISelDAGToDAG.cpp:175
#14 0x00000000076dd6fe in llvm::MachineFunctionPass::runOnFunction (
this=0x228932b0, F=...) at external/llvm/lib/CodeGen/MachineFunctionPass.cpp:62
#15 0x0000000008700cf3 in llvm::FPPassManager::runOnFunction (this=0x21c2b1c0,
F=...) at external/llvm/lib/IR/LegacyPassManager.cpp:1509
#16 0x0000000008701090 in llvm::FPPassManager::runOnModule (this=0x21c2b1c0, M=...)
at external/llvm/lib/IR/LegacyPassManager.cpp:1530
#17 0x0000000008701f3b in (anonymous namespace)::MPPassManager::runOnModule (
this=0x203c5d30, M=...) at external/llvm/lib/IR/LegacyPassManager.cpp:1586
#18 0x00000000087013b1 in llvm::legacy::PassManagerImpl::run (this=0x203afd70,
M=...) at external/llvm/lib/IR/LegacyPassManager.cpp:1689
#19 0x0000000008702c81 in llvm::legacy::PassManager::run (this=0x7fffffff6398,
M=...) at external/llvm/lib/IR/LegacyPassManager.cpp:1720
#20 0x00000000033304cd in (anonymous namespace)::EmitAssemblyHelper::EmitAssembly (
this=0x7fffffff6da8, Action=clang::Backend_EmitObj, OS=...)
at external/clang/lib/CodeGen/BackendUtil.cpp:716
#21 0x000000000332d4f0 in clang::EmitBackendOutput (Diags=..., CGOpts=...,
TOpts=..., LOpts=..., TDesc=..., M=0xad38cf0, Action=clang::Backend_EmitObj,
OS=...) at external/clang/lib/CodeGen/BackendUtil.cpp:787
#22 0x000000000374036b in clang::BackendConsumer::HandleTranslationUnit (
this=0xad386e0, C=...) at external/clang/lib/CodeGen/CodeGenAction.cpp:214
#23 0x0000000003a4318a in clang::ParseAST (S=..., PrintStats=false,
SkipFunctionBodies=false) at external/clang/lib/Parse/ParseAST.cpp:159
#24 0x000000000230e05b in clang::ASTFrontendAction::ExecuteAction (this=0xad1ca60)
at external/clang/lib/Frontend/FrontendAction.cpp:557
#25 0x000000000373c11e in clang::CodeGenAction::ExecuteAction (this=0xad1ca60)
at external/clang/lib/CodeGen/CodeGenAction.cpp:892
#26 0x000000000230d557 in clang::FrontendAction::Execute (this=0xad1ca60)
at external/clang/lib/Frontend/FrontendAction.cpp:458
#27 0x0000000002233058 in clang::CompilerInstance::ExecuteAction (this=0xad17c10,
Act=...) at external/clang/lib/Frontend/CompilerInstance.cpp:871
#28 0x000000000206eb6c in clang::ExecuteCompilerInvocation (Clang=0xad17c10)
at external/clang/lib/FrontendTool/ExecuteCompilerInvocation.cpp:249
#29 0x0000000002039dc2 in cc1_main (Argv=...,
Argv0=0x7fffffffd925 "/ssd1/pirama/aosp-llvm1/out/debug-install/linux-x86/clang-debug/bin/clang++.real", MainAddr=0x205a690 <GetExecutablePath(char const*, bool)>)
at external/clang/tools/driver/cc1_main.cpp:221
#30 0x000000000205eb20 in ExecuteCC1Tool (argv=..., Tool=...)
at external/clang/tools/driver/driver.cpp:299
#31 0x000000000205c2d7 in main (argc_=80, argv_=0x7fffffffd238)
at external/clang/tools/driver/driver.cpp:380

[pirama-arumuga-nainar]

(In the previosu comment, frame #4 triggers an assertion failure and I trimmed the frames #0-#3 that are related to the subsequent abort).

[pirama-arumuga-nainar]

To update, Chih-hung from our team figured out that r289043 as the change that fixes the segfault. It'll be included in the Clang in the stable r15. He's also trying to figure out if r289043 just masks the underlying issue, and if so, coming up with a proper fix.

[DanAlbert]

@pirama-arumuga-nainar: assign it back to me when we have a clang with the fix available and I'll pull it into the NDK.

[pirama-arumuga-nainar]

A fix by Chih-hung is pending upstream review: https://reviews.llvm.org/D32102

[DanAlbert]

We picked up this patch in the latest update. I've just merged that into r15 for beta 2.