Skip to content
This repository has been archived by the owner on May 27, 2024. It is now read-only.

esp-wifi and esp-storage seem to interact very badly on ESP32S3 #5

Closed
dberlin opened this issue Mar 20, 2023 · 12 comments
Closed

esp-wifi and esp-storage seem to interact very badly on ESP32S3 #5

dberlin opened this issue Mar 20, 2023 · 12 comments

Comments

@dberlin
Copy link

dberlin commented Mar 20, 2023

Attempts to read flash storage while BLE is active seem to cause crashes.

I do not have wifi enabled, only BLE.

The following GIST demonstrates it :

https://gist.github.com/dberlin/acbc1a1dbb9d12dbf44ec6a1fc3ce56c

Note that it won't even get to the real read part. It will crash on the new() call since flashstorage internally reads flash.

If you remove the wifi init, it will work fine.
Otherwise this crashes consistently with the following backtrace:

Init!
DEBUG - ble init
DEBUG - BT controller compile version 80abacd
DEBUG - !!!! unimplemented srand 288
DEBUG - The btdm_controller_init was initialized

Exception occured 'IllegalInstruction'
Context
PC=0x4037df38 PS=0x00060735
0x4037df38 - ram_chip_i2c_writeReg
at ??:??
0x00060735 - PS_WOE
at ??:??
A0=0x8003589a A1=0x3fcd0af0 A2=0x40055bc0 A3=0x00000000 A4=0x02000000
0x8003589a - _rtc_fast_bss_end
at ??:??
0x3fcd0af0 - _stack_start_cpu0
at ??:??
0x40055bc0 - rom_rx_gain_force
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x02000000 - PS_WOE
at ??:??
A5=0x059e0966 A6=0x05000966 A7=0xffffffff A8=0x8037df38 A9=0x3fcd0ad0
0x059e0966 - PS_WOE
at ??:??
0x05000966 - PS_WOE
at ??:??
0xffffffff - _rtc_fast_bss_end
at ??:??
0x8037df38 - _rtc_fast_bss_end
at ??:??
0x3fcd0ad0 - _stack_start_cpu0
at ??:??
A10=0x00000000 A11=0x04920966 A12=0x6000e000 A13=0x00000900 A14=0x40000648
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x04920966 - PS_WOE
at ??:??
0x6000e000 - _rtc_slow_data_end
at ??:??
0x00000900 - VECTORS_SIZE
at ??:??
0x40000648 - uart_tx_one_char
at ??:??
A15=0x00060b20
0x00060b20 - PS_WOE
at ??:??
SAR=0000001c
EXCCAUSE=0x00000000 EXCVADDR=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
LBEG=0x4201767d LEND=0x420176aa LCOUNT=0x00000000
0x4201767d - esp_recover_efuse_data
at ??:??
0x420176aa - esp_recover_efuse_data
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
THREADPTR=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
SCOMPARE1=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
BR=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
ACCLO=0x00000000 ACCHI=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
M0=0x00000000 M1=0x00000000 M2=0x00000000 M3=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
F64R_LO=0x00000000 F64R_HI=0x00000000 F64S=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
FCR=0x00000000 FSR=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
F0=0x00000000 F1=0x00000000 F2=0x00000000 F3=0x00000000 F4=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
F5=0x00000000 F6=0x00000000 F7=0x00000000 F8=0x00000000 F9=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
F10=0x00000000 F11=0x00000000 F12=0x00000000 F13=0x00000000 F14=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
0x00000000 - RESERVE_RTC_SLOW
at ??:??
F15=0x00000000
0x00000000 - RESERVE_RTC_SLOW
at ??:??

0x40035939
0x40035939 - rom_rx_gain_force
at ??:??
0x42019138
0x42019138 - i2c_bbpll_set
at ??:??
0x42015cda
0x42015cda - rf_init
at ??:??
0x420160be
0x420160be - register_chipv7_phy
at ??:??
0x420031f7
0x420031f7 - _ZN8esp_wifi10initialize17ha88d903e976453c3E
at ??:??
0x42000ce7
0x42000ce7 - main
at ??:??
0x4200530e
0x4200530e - Reset
at ??:??
0x403795c6
0x403795c6 - ESP32Reset
at ??:??
0x40000000
0x40000000 - _external_ram_end
at ??:??
0x403cdb78
0x403cdb78 - _text_heap_end
at ??:??
@dberlin
Copy link
Author

dberlin commented Mar 20, 2023

Oh even more interesting.

If I make a Cargo.toml that has these settings:
[profile.dev.package.esp-wifi]
opt-level = 3

And compile it in dev mode, it works.

It will still crash in release mode even with no profile options
If i reset release mode to opt_level=1 it works.
If i set it at opt_level=2 or above it fails.

Maybe a critical section is being removed or something?
I won't have time to look at the assembly today, but hopefully this will enable easy comparison and tracking down.

@MabezDev
Copy link
Member

Could you try with "-Ctarget-feature=-loop"? in your rust flags?

@dberlin
Copy link
Author

dberlin commented Mar 20, 2023

This causes invalid assembly:
error: could not compile xtensa-lx-rt due to 7 previous errors
warning: build failed, waiting for other jobs to finish...
error: invalid operand for instruction
--> /home/dannyb/.cargo/registry/src/github.51.al-1ecc6299db9ec823/xtensa-lx-rt-0.15.0/src/exception/assembly_esp32.rs:128:9
|
128 | rsr a3, LBEG
| ^
|
note: instantiated into assembly here
--> :22:22
|
22 | rsr a3, LBEG
| ^

@MabezDev
Copy link
Member

Hmm, that's odd. Can you pass it in the flags via .cargo/config.toml instead? Perhaps the feature detection isn't working properly when passing via environment variable.

@dberlin
Copy link
Author

dberlin commented Mar 20, 2023 via email

@MabezDev
Copy link
Member

What is your cargo version? The feature detection relies on CARGO_ENCODED_RUSTFLAGS environment variable, maybe your cargo is too old? (Sorry, I'm sure which version CARGO_ENCODED_RUSTFLAGS was added.)

@MabezDev
Copy link
Member

Actually, it should panic in the build script: https://github.com/esp-rs/xtensa-lx-rt/blob/283ea94b4db16ebe41f836e622d5cbd0b00ccfc2/build.rs#L48 if it's not available. Hmm.

@MabezDev
Copy link
Member

FYI I can successfully build the examples-esp32s3 in the esp-wifi repo with the following flags:

rustflags = [
    "-C", "link-arg=-Tlinkall.x",
    "-C", "link-arg=-Trom_functions.x",

    "-C", "target-feature=-loop"
]

Could you try that?

@dberlin
Copy link
Author

dberlin commented Mar 21, 2023

Let me see what i can make work.
If not, I may just disassemble the code and rom and see if there is anything obvious

@MabezDev
Copy link
Member

It is an existing issue, which a couple of people have run into already, see esp-rs/rust#161 & esp-rs/rust#164, which is why I mentioned it.

Fortunately, we will hopefully have a fix in our LLVM Xtensa backend soon! Currently testing a fix internally :).

@dberlin
Copy link
Author

dberlin commented Mar 21, 2023

Thank you, that helps a lot - i got it to compile/work with -loop, and looked at the LLVM IR dumps. it's definitely the same bug being described in those threads.
So i'm gonna close this unless you want it open for some tracking reason!

@MabezDev
Copy link
Member

Sorry that you're running into that issue - hopefully it will be fixed soon! Thanks again for the report, and help with debugging :).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dberlin @MabezDev