h2o: fuzz target crashes on startup.

[Dor1s]

https://oss-fuzz-build-logs.storage.googleapis.com/index.html

Step #5: INFO: performing bad build checks for /workspace/out/address/h2o-fuzzer-url.
Step #5: INFO: performing bad build checks for /workspace/out/address/h2o-fuzzer-http1.
Step #5: INFO: performing bad build checks for /workspace/out/address/h2o-fuzzer-http2.
Step #5: Broken fuzz targets (1):
Step #5: h2o-fuzzer-http1:
Step #5: h2o-fuzzer-http1 has a crash input in its seed coprpus:
Step #5: Using seed corpus: h2o-fuzzer-http1_seed_corpus.zip
Step #5: /workspace/out/address/h2o-fuzzer-http1 -rss_limit_mb=2048 -timeout=25 -runs=0 /tmp/h2o-fuzzer-http1_corpus -close_fd_mask=3 -max_len=16384 -dict=http.dict < /dev/null
Step #5: Dictionary: 126 entries
Step #5: INFO: Seed: 2301698242
Step #5: INFO: Loaded 1 modules   (9426 inline 8-bit counters): 9426 [0xc0e350, 0xc10822), 
Step #5: INFO: Loaded 1 PC tables (9426 PCs): 9426 [0x933450,0x958170), 
Step #5: INFO:     1709 files found in /tmp/h2o-fuzzer-http1_corpus
Step #5: ALARM: working on the last Unit for 25 seconds
Step #5:        and the timeout value is 25 (use -timeout=N to change)
Step #5: MS: 0 ; base unit: 0000000000000000000000000000000000000000
Step #5: 
Step #5: 
Step #5: artifact_prefix='./'; Test unit written to ./timeout-da39a3ee5e6b4b0d3255bfef95601890afd80709
Step #5: Base64: 
Step #5: ==127== ERROR: libFuzzer: timeout after 25 seconds
Step #5:     #0 0x564d03 in __sanitizer_print_stack_trace /src/llvm/projects/compiler-rt/lib/asan/asan_stack.cc:38
Step #5:     #1 0x69d938 in fuzzer::Fuzzer::AlarmCallback() /src/libfuzzer/FuzzerLoop.cpp:292:7
Step #5:     #2 0x7f63c479a38f  (/lib/x86_64-linux-gnu/libpthread.so.0+0x1138f)
Step #5:     #3 0x7f63c40a8a12 in epoll_wait (/lib/x86_64-linux-gnu/libc.so.6+0x107a12)
Step #5:     #4 0x599909 in evloop_do_proceed /src/h2o/lib/common/socket/evloop/epoll.c.h:115:15
Step #5:     #5 0x599909 in h2o_evloop_run /src/h2o/lib/common/socket/evloop.c.h:599
Step #5:     #6 0x595d10 in LLVMFuzzerTestOneInput /src/h2o/fuzz/driver.cc:400:9
Step #5:     #7 0x6a1681 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:517:13
Step #5:     #8 0x6a4449 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:703:3
Step #5:     #9 0x6a5348 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:741:3
Step #5:     #10 0x684e4e in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:754:6
Step #5:     #11 0x677dcc in main /src/libfuzzer/FuzzerMain.cpp:20:10
Step #5:     #12 0x7f63c3fc182f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
Step #5:     #13 0x48a0c8 in _start (out/address/h2o-fuzzer-http1+0x48a0c8)
Step #5: 
Step #5: SUMMARY: libFuzzer: timeout
Step #5: ERROR: 33% of fuzz targets seem to be broken. See the list above for a detailed information.

//cc @jfoote

[Dor1s]

FYI, we've temporarily disabled that check, but it's still worth to fix any crashes on seed corpus. See the discussion in #1364 (comment)

[jfoote]

Thanks for the report @Dor1s. I have reproduced the issue; it appears to be an intermittent bug. This may be a race related to our use of threading in the test harness (which was a necessary evil). I will follow up here when we have isolated and patched the issue.

[Dor1s]

There is a startup timeout (even without any corpus):

Step #11: Broken fuzz targets (1):
Step #11: h2o-fuzzer-http1:
Step #11: BAD BUILD: /workspace/out/undefined/h2o-fuzzer-http1 seems to have either startup crash or exit:
Step #11: INFO: Seed: 3801703249
Step #11: INFO: Loaded 1 modules   (9235 inline 8-bit counters): 9235 [0x9d6510, 0x9d8923), 
Step #11: INFO: Loaded 1 PC tables (9235 PCs): 9235 [0x72f410,0x753540), 
Step #11: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
Step #11: ALARM: working on the last Unit for 1202 seconds
Step #11:        and the timeout value is 1200 (use -timeout=N to change)
Step #11: MS: 0 ; base unit: 0000000000000000000000000000000000000000
Step #11: 
Step #11: 
Step #11: artifact_prefix='./'; Test unit written to ./timeout-da39a3ee5e6b4b0d3255bfef95601890afd80709
Step #11: Base64: 
Step #11: ==87== ERROR: libFuzzer: timeout after 1202 seconds
Step #11:     #0 0x49ae63 in __sanitizer_print_stack_trace /src/llvm/projects/compiler-rt/lib/ubsan/ubsan_diag_standalone.cc:29
Step #11:     #1 0x510a31 in fuzzer::Fuzzer::AlarmCallback() /src/libfuzzer/FuzzerLoop.cpp:292:7
Step #11:     #2 0x7fa53bcb938f  (/lib/x86_64-linux-gnu/libpthread.so.0+0x1138f)
Step #11:     #3 0x7fa53b5c7a12 in epoll_wait (/lib/x86_64-linux-gnu/libc.so.6+0x107a12)
Step #11:     #4 0x49f914 in evloop_do_proceed /src/h2o/lib/common/socket/evloop/epoll.c.h:115:15
Step #11:     #5 0x49f914 in h2o_evloop_run /src/h2o/lib/common/socket/evloop.c.h:599
Step #11:     #6 0x49dd50 in LLVMFuzzerTestOneInput /src/h2o/fuzz/driver.cc:400:9
Step #11:     #7 0x511fac in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:517:13
Step #11:     #8 0x513178 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:703:3
Step #11:     #9 0x513641 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:741:3
Step #11:     #10 0x507d29 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:754:6
Step #11:     #11 0x503d08 in main /src/libfuzzer/FuzzerMain.cpp:20:10
Step #11:     #12 0x7fa53b4e082f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
Step #11:     #13 0x476048 in _start (out/undefined/h2o-fuzzer-http1+0x476048)
Step #11: 
Step #11: SUMMARY: libFuzzer: timeout
Step #11: ERROR: 33% of fuzz targets seem to be broken. See the list above for a detailed information.

[inferno-chromium]

Step #11: BAD BUILD: /workspace/out/undefined/h2o-fuzzer-http2 seems to have either startup crash or exit:
Step #11: WARNING: Failed to find function "__sanitizer_acquire_crash_state".
Step #11: INFO: Seed: 1319098975
Step #11: INFO: Loaded 1 modules (9235 inline 8-bit counters): 9235 [0x9d6590, 0x9d89a3),
Step #11: INFO: Loaded 1 PC tables (9235 PCs): 9235 [0x72f410,0x753540),
Step #11: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
Step #11: ALARM: working on the last Unit for 1202 seconds
Step #11: and the timeout value is 1200 (use -timeout=N to change)
Step #11: MS: 0 ; base unit: 0000000000000000000000000000000000000000
Step #11:
Step #11:
Step #11: artifact_prefix='./'; Test unit written to ./timeout-da39a3ee5e6b4b0d3255bfef95601890afd80709
Step #11: Base64:
Step #11: ==90== ERROR: libFuzzer: timeout after 1202 seconds
Step #11: #0 0x49aee3 in __sanitizer_print_stack_trace /src/llvm/projects/compiler-rt/lib/ubsan/ubsan_diag_standalone.cc:29
Step #11: #1 0x510b16 in fuzzer::Fuzzer::AlarmCallback() /src/libfuzzer/FuzzerLoop.cpp:300:7
Step #11: #2 0x7f60c348638f (/lib/x86_64-linux-gnu/libpthread.so.0+0x1138f)
Step #11: #3 0x7f60c2d94a12 in epoll_wait (/lib/x86_64-linux-gnu/libc.so.6+0x107a12)
Step #11: #4 0x49f994 in evloop_do_proceed /src/h2o/lib/common/socket/evloop/epoll.c.h:115:15
Step #11: #5 0x49f994 in h2o_evloop_run /src/h2o/lib/common/socket/evloop.c.h:599
Step #11: #6 0x49ddd0 in LLVMFuzzerTestOneInput /src/h2o/fuzz/driver.cc:400:9
Step #11: #7 0x51214c in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/libfuzzer/FuzzerLoop.cpp:528:13
Step #11: #8 0x513318 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > > > const&) /src/libfuzzer/FuzzerLoop.cpp:714:3
Step #11: #9 0x5137e1 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > > > const&) /src/libfuzzer/FuzzerLoop.cpp:752:3
Step #11: #10 0x507da9 in fuzzer::FuzzerDriver(int*, char***, int ()(unsigned char const, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:754:6
Step #11: #11 0x503d88 in main /src/libfuzzer/FuzzerMain.cpp:20:10
Step #11: #12 0x7f60c2cad82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
Step #11: #13 0x4760c8 in _start (out/undefined/h2o-fuzzer-http2+0x4760c8)
Step #11:
Step #11: SUMMARY: libFuzzer: timeout
Step #11: ERROR: 33% of fuzz targets seem to be broken. See the list above for a detailed information.
Finished Step #11
ERROR
ERROR: build step 11 "gcr.io/oss-fuzz-base/base-runner" failed: exit status 1

[jfoote]

Just a quick update: I reproduced the issue; the h2o team is planning to take a look this week.

[inferno-chromium]

No need to track here, you should see a bug filed with title ": Build failure" in monorail tracker.