Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Share unified UTF-8/16 parsing with BigInteger #95402

Merged
merged 6 commits into from
Feb 1, 2024

Conversation

huoyaoyuan
Copy link
Member

Follow up of #85978 . Should really be in same PR.

Copied latest CoreLib implementation and polyfill internal interfaces.
This allows sharing UTF unified parsing and formatting code.

Originally I tried this with heavy ifdefs and tricks like using TChar = System.Char;. Then I realized that all the internal members can be polyfilled in S.R.Numerics, with very little overhead for UTF-16. This also unblocks UTF-8 adoption for BigInteger.

Wanna check with @stephentoub for whether this design is desired.

@ghost ghost added the community-contribution Indicates that the PR has been added by a community member label Nov 29, 2023
@ghost
Copy link

ghost commented Nov 29, 2023

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

Follow up of #85978 . Should really be in same PR.

Copied latest CoreLib implementation and polyfill internal interfaces.
This allows sharing UTF unified parsing and formatting code.

Originally I tried this with heavy ifdefs and tricks like using TChar = System.Char;. Then I realized that all the internal members can be polyfilled in S.R.Numerics, with very little overhead for UTF-16. This also unblocks UTF-8 adoption for BigInteger.

Wanna check with @stephentoub for whether this design is desired.

Author: huoyaoyuan
Assignees: -
Labels:

area-System.Numerics, community-contribution

Milestone: -

@huoyaoyuan
Copy link
Member Author

Test failures look unrelated. No integer/biginteger related.

@tannergooding
Copy link
Member

CC. @vargaz, another LLVM OOM. It's not clear to me which line indicates the failing assembly so I can know what needs to be skipped until the final PR can get merged?

C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Failed to compile C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc -> C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.o [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>C:\Windows\System32\chcp.com 65001 1>nul  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>setlocal [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>set errorlevel=dummy  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>set errorlevel=  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : C:\helix\work\workitem\e\publish>"C:\helix\work\correlation\build\\wasi-sdk\\bin\clang.exe" "@C:\helix\work\correlation\build\microsoft.netcore.app.runtime.wasi-wasm\runtimes\wasi-wasm\native\src\wasi-default.rsp" "@C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\wasi-compile-bc.rsp" -c -o "C:\Users\ContainerAdministrator\AppData\Local\Temp\tmpa2qi3t.tmp" "C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc"  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : clang version 16.0.0 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error :  (in-process) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error :  "C:/helix/work/correlation/build/wasi-sdk/bin/clang.exe" -cc1 -triple wasm32-unknown-wasi -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name aot-instances.dll.bc -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility=hidden -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=C:/helix/work/workitem/e/publish -resource-dir C:/helix/work/correlation/build/wasi-sdk/lib/clang/16 -Oz -fdebug-compilation-dir=C:/helix/work/workitem/e/publish -ferror-limit 19 -fgnuc-version=4.2.1 -vectorize-slp -o "C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmpa2qi3t.tmp" -x ir "C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\aot-instances.dll.bc" [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : clang -cc1 version 16.0.0 based upon LLVM 16.0.0 default target wasm32-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : LLVM ERROR: out of memory [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Allocation failed [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : Stack dump: [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 0.	Program arguments: C:\\helix\\work\\correlation\\build\\\\wasi-sdk\\\\bin\\clang.exe @C:\\helix\\work\\correlation\\build\\microsoft.netcore.app.runtime.wasi-wasm\\runtimes\\wasi-wasm\\native\\src\\wasi-default.rsp @C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\wasi-compile-bc.rsp -c -o C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmpa2qi3t.tmp C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\aot-instances.dll.bc [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 1.	Code generation [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 2.	Running pass 'Function Pass Manager' on module 'C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc'. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(818,5): error : 3.	Running pass 'WebAssembly Instruction Selection' on function '@aot_instances_System_SZGenericArrayEnumerator_1_T_SINGLE__cctor' [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]

@vargaz
Copy link
Contributor

vargaz commented Feb 1, 2024

Those failures are happening on other PRs as well, so i think this is ok to merge.

@tannergooding tannergooding merged commit 9e302a5 into dotnet:main Feb 1, 2024
176 of 178 checks passed
@huoyaoyuan huoyaoyuan deleted the numerics-format-1-polifill branch February 2, 2024 02:32
@github-actions github-actions bot locked and limited conversation to collaborators Mar 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Numerics community-contribution Indicates that the PR has been added by a community member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants