Compilation failure when trying to add xtensa as custom patch on apple/llvm-project

[Santhosh-KS]

Hi,
I'm trying to apply the expressif/llvm-project's Xtensa support on the apple/llvm-project to make the Swift for tensorflow tool chain available for esp32. Here is the patch I have applied . My general idea is to build the ML/AI models using S4TF and optimize it to run it on the esp32 cam. For this purpose I'm trying to cross compile the toolchain by applying the xtensa patch on the apple/llvm-project on swift/tensorflow branch.

When I am trying to build the espressif/llvm-project fork with the set of toolchains with clang support suggested here, I am facing the following error.

Here is the build command I'm using

cmake -G Ninja -DCMAKE_CROSSCOMPILING=True -DCMAKE_INSTALL_PREFIX=/opt/my_esp32_clang  -DLLVM_DEFAULT_TARGET_TRIPLE=xtensa-esp32-elf-clang  -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=Xtensa --sysroot=/home/gl-245/esp_stuff/xtensa-esp32-elf-clang --target xtensa-esp32-elf-clang  -DCMAKE_BUILD_TYPE=Release  -DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi" --tool-chain=/home/gl-245/esp_stuff/xtensa-esp32-elf-clang ../llvm

Initially I got some errors on LLVM_EXTERNAL_VISIBILITY I fixed as follows by using LLVM_LIBRARY_VISIBILITY

diff --git a/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp b/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp
index 78c6e13534e6..44ac0b224161 100644
--- a/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp
+++ b/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp
@@ -53,7 +53,7 @@ static MCDisassembler *createXtensaDisassembler(const Target &T,
   return new XtensaDisassembler(STI, Ctx, true);
 }
 
-extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeXtensaDisassembler() {
+extern "C" LLVM_LIBRARY_VISIBILITY void LLVMInitializeXtensaDisassembler() {
   TargetRegistry::RegisterMCDisassembler(TheXtensaTarget,
                                          createXtensaDisassembler);
 }

Can you please confirm if this fix is correct?

Here is the error I'm facing.

[22/1515] Building CXX object lib/Target/Xtensa/Disassembler/CMakeFiles/LLVMXtensaDisassembler.dir/XtensaDisassembler.cpp.o
FAILED: lib/Target/Xtensa/Disassembler/CMakeFiles/LLVMXtensaDisassembler.dir/XtensaDisassembler.cpp.o 
/usr/bin/c++ -DGTEST_HAS_RTTI=0 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib/Target/Xtensa/Disassembler -I/home/gl-245/swift_stuff/cherry_pick_ex/my_llvm_fork/llvm-project/llvm/lib/Target/Xtensa/Disassembler -I/home/gl-245/swift_stuff/cherry_pick_ex/my_llvm_fork/llvm-project/llvm/lib/Target/Xtensa -Ilib/Target/Xtensa -I/usr/include/libxml2 -Iinclude -I/home/gl-245/swift_stuff/cherry_pick_ex/my_llvm_fork/llvm-project/llvm/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -fno-exceptions -fno-rtti -std=c++14 -MD -MT lib/Target/Xtensa/Disassembler/CMakeFiles/LLVMXtensaDisassembler.dir/XtensaDisassembler.cpp.o -MF lib/Target/Xtensa/Disassembler/CMakeFiles/LLVMXtensaDisassembler.dir/XtensaDisassembler.cpp.o.d -o lib/Target/Xtensa/Disassembler/CMakeFiles/LLVMXtensaDisassembler.dir/XtensaDisassembler.cpp.o -c /home/gl-245/swift_stuff/cherry_pick_ex/my_llvm_fork/llvm-project/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp
/home/gl-245/swift_stuff/cherry_pick_ex/my_llvm_fork/llvm-project/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp:44:16: error: ‘llvm::MCDisassembler::DecodeStatus {anonymous}::XtensaDisassembler::getInstruction(llvm::MCInst&, uint64_t&, llvm::ArrayRef<unsigned char>, uint64_t, llvm::raw_ostream&) const’ marked ‘override’, but does not override
   DecodeStatus getInstruction(MCInst &Instr, uint64_t &Size,
                ^~~~~~~~~~~~~~
/home/gl-245/swift_stuff/cherry_pick_ex/my_llvm_fork/llvm-project/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp: In function ‘llvm::MCDisassembler* createXtensaDisassembler(const llvm::Target&, const llvm::MCSubtargetInfo&, llvm::MCContext&)’:
/home/gl-245/swift_stuff/cherry_pick_ex/my_llvm_fork/llvm-project/llvm/lib/Target/Xtensa/Disassembler/XtensaDisassembler.cpp:53:47: error: invalid new-expression of abstract class type ‘{anonymous}::XtensaDisassembler’
   return new XtensaDisassembler(STI, Ctx, true);

I'm not sure what is the exact cause though. I looked at the other implementations of Disassembler I find the similar code. So I cannot pin point the exact reason for failure. But I'm guessing it is something related to the include which is happening

  #include "XtensaGenDisassemblerTables.inc"
 
 DecodeStatus XtensaDisassembler::getInstruction(MCInst &MI, uint64_t &Size,
                                                  ArrayRef<uint8_t> Bytes,
                                                  uint64_t Address,
                                                  raw_ostream &CS) const {

I'm not sure how this file is generated. But this inc file doesn't have any include guards. So I'm suspecting this could be causing this error.
Any inputs on how to fix this error will be helpful.

I'm not an expert in this cross compilation stuff. If this approach of creating a custom patch is not correct. Any inputs on the best practice/industry standard on how to do the same is appreciated.