Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: switch to LLVM 16 #3741

Merged
merged 2 commits into from
Sep 18, 2023
Merged

all: switch to LLVM 16 #3741

merged 2 commits into from
Sep 18, 2023

Conversation

aykevl
Copy link
Member

@aykevl aykevl commented May 22, 2023

This commit adds support for LLVM 16 and switches to it by default. That means three LLVM versions are supported at the same time: LLVM 14, 15, and 16.

Part of this work was based on a PR by QuLogic: #3649
But I also had parts of this already implemented in an old branch I already made for LLVM 16.

Here are some differences:

  • It switches to LLVM 16 by default.
  • It updates some things to also make it work with a self-built LLVM.
  • It fixes the CGo bug in a slightly different way, and also fixes another one not included in the original PR.
  • It does not keep compiler tests passing on older LLVM versions. I have found this to be quite burdensome and therefore don't generally do this - the smoke tests should hopefully catch most regressions.

@aykevl aykevl mentioned this pull request May 22, 2023
@aykevl
Copy link
Member Author

aykevl commented May 22, 2023

Sizediff fails (which is expected). I ran the tool locally and found that almost all drivers smoke tests are reduced in size (the average over all of them is a reduction of 0.55%).

sizediff
 before   after   diff
   8336    7048  -1288 -15.45%  '-zvl65536b' is not a recognized feature for this target (ignoring feature)
  98648   98136   -512  -0.52%  tinygo build -size short -o ./build/test.hex -target=wioterminal ./examples/rtl8720dn/mqttsub/
  66792   66448   -344  -0.52%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp2515/main.go
  60684   60364   -320  -0.53%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adt7410/main.go
  81360   81040   -320  -0.39%  tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/lora/lorawan/atcmd/
  82504   82200   -304  -0.37%  tinygo build -size short -o ./build/test.hex -target=wioterminal ./examples/rtl8720dn/webclient/
  14792   14508   -284  -1.92%  tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/hts221/main.go
  61620   61340   -280  -0.45%  tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/is31fl3731/main.go
  10636   10360   -276  -2.59%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/itsybitsy-m0/main.go
  60344   60080   -264  -0.44%  tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/i2csoft/adt7410/
   8124    7860   -264  -3.25%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ttp229/main.go
  10400   10140   -260  -2.50%  tinygo build -size short -o ./build/test.elf -target=wioterminal ./examples/axp192/m5stack-core2-blinky/
  28064   27808   -256  -0.91%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp180/main.go
  63196   62940   -256  -0.41%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp280/main.go
  10952   10696   -256  -2.34%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/tm1637/main.go
  71840   71584   -256  -0.36%  tinygo build -size short -o ./build/test.hex -target=wioterminal ./examples/rtl8720dn/webserver/
   8968    8712   -256  -2.85%  tinygo build -size short -o ./build/test.uf2 -target=circuitplay-express ./examples/makeybutton/main.go
 136916  136668   -248  -0.18%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/udpstation/main.go
   8968    8728   -240  -2.68%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/speed/main.go
  14148   13912   -236  -1.67%  tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/lps22hb/main.go
 136908  136676   -232  -0.17%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/ntpclient/main.go
  70324   70092   -232  -0.33%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/time/
  72068   71836   -232  -0.32%  tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/sdcard/console/
  64708   64484   -224  -0.35%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/flash/console/qspi
  56168   55952   -216  -0.38%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/shtc3/main.go
 137308  137092   -216  -0.16%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/webclient/main.go
   8484    8276   -208  -2.45%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bh1750/main.go
  24828   24620   -208  -0.84%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/espat/espstation/main.go
   2889    2681   -208  -7.20%  tinygo build -size short -o ./build/test.hex -target=arduino ./examples/servo
  63128   62928   -200  -0.32%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmi160/main.go
  68572   68372   -200  -0.29%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/flash/console/spi
  11024   10824   -200  -1.81%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/basic
  11100   10900   -200  -1.80%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/scroll
 267492  267292   -200  -0.07%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/slideshow
  56188   55988   -200  -0.36%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht3x/main.go
  10220   10024   -196  -1.92%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/basic
  10248   10052   -196  -1.91%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/scroll
  29212   29020   -192  -0.66%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/pyportal_boing
   9996    9808   -188  -1.88%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp3008/main.go
   6420    6232   -188  -2.93%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1306/i2c_128x32/main.go
  17368   17180   -188  -1.08%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/thermistor/main.go
  16816   16628   -188  -1.12%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/microphone/main.go
  68808   68624   -184  -0.27%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/main.go
  24820   24636   -184  -0.74%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/espat/espconsole/main.go
   8576    8392   -184  -2.15%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/hcsr04/main.go
   8896    8716   -180  -2.02%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/main.go
  11484   11304   -180  -1.57%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/buzzer/main.go
   6980    6800   -180  -2.58%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/simple/main.go
  11780   11604   -176  -1.49%  tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/apds9960/proximity/main.go
  74784   74608   -176  -0.24%  tinygo build -size short -o ./build/test.hex -target=p1am-100 ./examples/p1am/main.go
 137144  136968   -176  -0.13%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/wifinina/tcpclient/main.go
   7120    6944   -176  -2.47%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/ws2812
   7012    6836   -176  -2.51%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/simple/main.go
   8092    7920   -172  -2.13%  tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/shifter/main.go
  13364   13192   -172  -1.29%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis2mdl/main.go
  13372   13204   -168  -1.26%  tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/amg88xx
  67880   67712   -168  -0.25%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/i2c/main.go
  11084   10920   -164  -1.48%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mag3110/main.go
  10716   10556   -160  -1.49%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/hd44780i2c/main.go
  25020   24864   -156  -0.62%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/espat/esphub/main.go
   9536    9380   -156  -1.64%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/fourwire/main.go
   9732    9576   -156  -1.60%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ds18b20/main.go
   9432    9280   -152  -1.61%  tinygo build -size short -o ./build/test.hex -target=nucleo-l432kc ./examples/aht20/main.go
   9768    9624   -144  -1.47%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adxl345/main.go
  15852   15712   -140  -0.88%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl53l1x/main.go
  10708   10572   -136  -1.27%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017-multiple/main.go
   8636    8500   -136  -1.57%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mma8653/main.go
   8420    8284   -136  -1.62%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mpu6050/main.go
   9524    9388   -136  -1.43%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/speed/main.go
   9044    8908   -136  -1.50%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/xpt2046/main.go
   7708    7576   -132  -1.71%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/blinkm/main.go
  10272   10140   -132  -1.29%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017/main.go
  68292   68164   -128  -0.19%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/uart/main.go
  70588   70460   -128  -0.18%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/timer/
  11340   11212   -128  -1.13%  tinygo build -size short -o ./build/test.hex -target=pico ./examples/irremote/main.go
   9932    9804   -128  -1.29%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu6886/main.go
   5452    5326   -126  -2.31%  '-zvl65536b' is not a recognized feature for this target (ignoring feature)
  12256   12132   -124  -1.01%  tinygo build -size short -o ./build/test.hex -target=pico ./examples/pca9685/main.go
  13876   13752   -124  -0.89%  tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl6180x/main.go
   7464    7344   -120  -1.61%  tinygo build -size short -o ./build/test.hex -target=nucleo-f103rb ./examples/shiftregister/main.go
  11300   11180   -120  -1.06%  tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/uc8151/main.go
   9024    8908   -116  -1.29%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina260/main.go
  77180   77068   -112  -0.15%  tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/dht/main.go
  10416   10304   -112  -1.08%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/scd4x/main.go
  12620   12516   -104  -0.82%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/lsm6ds3/main.go
  25940   25844    -96  -0.37%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/lsm303agr/main.go
  12532   12436    -96  -0.77%  tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/pyportal_touchpaint/main.go
  32332   32236    -96  -0.30%  tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bme280/main.go
  12124   12032    -92  -0.76%  tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis3dh/main.go
  13076   12984    -92  -0.70%  tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/veml6070/main.go
  10284   10204    -80  -0.78%  tinygo build -size short -o ./build/test.hex -target=circuitplay-bluefruit ./examples/tone
   8516    8440    -76  -0.89%  tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/max72xx/main.go
   6112    6040    -72  -1.18%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setbuffer/main.go
   6420    6348    -72  -1.12%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7735/main.go
   7492    7420    -72  -0.96%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/clkout/
   5728    5660    -68  -1.19%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1331/main.go
   6316    6248    -68  -1.08%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7789/main.go
   6484    6416    -68  -1.05%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13/main.go
   6036    5968    -68  -1.13%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13x/main.go
   6312    6244    -68  -1.08%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd4in2/main.go
   7072    7008    -64  -0.90%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/gc9a01/main.go
   5124    5060    -64  -1.25%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setpixel/main.go
   5960    5896    -64  -1.07%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1306/spi_128x64/main.go
  70652   70588    -64  -0.09%  tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/alarm/
  12432   12372    -60  -0.48%  tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bmp388/main.go
  12152   12108    -44  -0.36%  tinygo build -size short -o ./build/test.hex -target=pico ./examples/qmi8658c/main.go
  21912   21880    -32  -0.15%  tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/time/main.go
  25860   25828    -32  -0.12%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/ssd1289/main.go
  21808   21784    -24  -0.11%  tinygo build -size short -o ./build/test.hex -target=microbit-v2 ./examples/microbitmatrix/main.go
   7724    7708    -16  -0.21%  tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/sram/main.go
  15896   15880    -16  -0.10%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hub75/main.go
    896     880    -16  -1.79%  tinygo build -size short -o ./build/test.hex -target=digispark ./examples/ws2812
  14572   14560    -12  -0.08%  tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/sx126x/lora_rxtx/
   7920    7912     -8  -0.10%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/at24cx/main.go
   4712    4708     -4  -0.08%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/easystepper/main.go
  21808   21808      0   0.00%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/microbitmatrix/main.go
   1565    1565      0   0.00%  tinygo build -size short -o ./build/test.hex -target=arduino   ./examples/ws2812
   5688    5712     24   0.42%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/customchar/main.go
  15636   15660     24   0.15%  tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/as560x/main.go
   5652    5756    104   1.84%  tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/text/main.go
3464810 3445632 -19178  -0.55%  sum

@deadprogram
Copy link
Member

So I assume this is for v0.29.0 correct?

@aykevl
Copy link
Member Author

aykevl commented May 22, 2023

Yes. I wanted to include it in v0.28 but then I probably should have made this PR a while ago, to catch regressions.

@dgryski
Copy link
Member

dgryski commented Jun 14, 2023

Should we merge this now early in the cycle?

Copy link
Member Author

aykevl commented Jun 17, 2023

Yes, I'd like to get this merged soon.

@deadprogram
Copy link
Member

@aykevl please rebase against latest dev

@aykevl
Copy link
Member Author

aykevl commented Jul 6, 2023

Rebased, ready for merge!

@deadprogram
Copy link
Member

@aykevl
Copy link
Member Author

aykevl commented Jul 6, 2023

That looks like an expected error. It has LLVM 16 installed, but checks out the dev branch (that uses LLVM 15) and then fails to compile.
This should only happen in the PR, and this error should be gone once merged into the dev branch.

@deadprogram
Copy link
Member

@aykevl in order to update the llvm docker image used for cached builds you also need to push this branch to the build-llvm-image branch.

See

# To update, make any needed changes to this file,
# then push to the "build-llvm-image" branch.

@deadprogram
Copy link
Member

I just force pushed this branch to build-llvm-image branch.

@aykevl
Copy link
Member Author

aykevl commented Jul 6, 2023

I also found another issue with CGo I need to investigate. It manifests as a test failure on Windows but that's just the symptom, the issue is bigger than that.

@deadprogram deadprogram modified the milestones: v0.29.0, v0.30.0 Jul 20, 2023
@QuLogic
Copy link
Contributor

QuLogic commented Aug 8, 2023

This appears to work well enough as a replacement of #3649 for me.

The CGo failure on CI appears to be just a path separator thing, testdata/... vs testdata\...; is there some other problem to fix?

@aykevl
Copy link
Member Author

aykevl commented Aug 8, 2023

The CGo failure on CI appears to be just a path separator thing, testdata/... vs testdata\...; is there some other problem to fix?

There shouldn't be a path in the identifier in the first place. That's the problem that I need to fix.

@QuLogic
Copy link
Contributor

QuLogic commented Aug 15, 2023

Ah, see #3869.

@deadprogram
Copy link
Member

@deadprogram
Copy link
Member

@aykevl please note merge conflict. Also seems like the build is not starting up as expected. Not sure why, but I bet if you rebase it will.

@deadprogram
Copy link
Member

@aykevl
Copy link
Member Author

aykevl commented Sep 18, 2023

Yeah that one is weird, it reports a different size for me locally. Perhaps I have an older branch of LLVM or something.

Anyway, it should now be fixed (and I have also fixed another minor CI issue).

This commit adds support for LLVM 16 and switches to it by default. That
means three LLVM versions are supported at the same time: LLVM 14, 15,
and 16.

This commit includes work by QuLogic:

  * Part of this work was based on a PR by QuLogic:
    #3649
    But I also had parts of this already implemented in an old branch I
    already made for LLVM 16.
  * QuLogic also provided a CGo fix here, which is also incorporated in
    this commit:
    #3869

The difference with the original PR by QuLogic is that this commit is
more complete:
  * It switches to LLVM 16 by default.
  * It updates some things to also make it work with a self-built LLVM.
  * It fixes the CGo bug in a slightly different way, and also fixes
    another one not included in the original PR.
  * It does not keep compiler tests passing on older LLVM versions. I
    have found this to be quite burdensome and therefore don't generally
    do this - the smoke tests should hopefully catch most regressions.
So that `go install` works on MacOS with Homebrew (and on Linux with an
up-to-date distro).
@deadprogram
Copy link
Member

Great work on this @QuLogic and @aykevl thank you very much! Now merging.

@deadprogram deadprogram merged commit 8698a7e into dev Sep 18, 2023
21 of 22 checks passed
@deadprogram deadprogram deleted the llvm16 branch September 18, 2023 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants