infra: refine fuzztest work #8992

DavidKorczynski · 2022-11-16T17:13:39Z

Enable Fuzztest fuzzers for Tensorflow.

This depends on google/fuzztest#79 and eventually a PR on the tensorflow repo with the (to be refined) diff in this PR.

Signed-off-by: David Korczynski david@adalogics.com

This is draft. Signed-off-by: David Korczynski <david@adalogics.com>

This is needed for tensorflow. Signed-off-by: David Korczynski <david@adalogics.com>

DavidKorczynski · 2022-11-16T19:59:11Z

Compiling, running and extracting coverage of the tensorflow fuzztest fuzzers work:

Some refinement and cleanup is needed.

Cleanup is needed. Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>

Split between CFLAGS and CXXFLAGS. This is needed to make tensorflow work. Related: google/oss-fuzz#8992 Signed-off-by: David Korczynski <david@adalogics.com>

Split CFLAGS and CXXFLAGS into language-specific bazel constructs. This is needed to make Tensorflow work. Related: google/oss-fuzz#8992 Signed-off-by: David Korczynski <david@adalogics.com>

DavidKorczynski · 2022-11-17T09:56:57Z

CI is failing because of timeouts, however, in UBSAN we get past the point where the UBSAN FuzzTest fuzzers are build (showing the building works):

2022-11-16T23:47:38.9367669Z �[32m[565 / 565]�[0m checking cached actions
2022-11-16T23:47:38.9442114Z 
2022-11-16T23:47:38.9442589Z �[1A�[K�[32mINFO:�[0m Build completed successfully, 565 total actions
2022-11-16T23:47:38.9442820Z 
2022-11-16T23:47:38.9443069Z �[1A�[K�[32mINFO:�[0m Build completed successfully, 565 total actions
2022-11-16T23:47:38.9453814Z �[0m+ for fuzz_main_file in $FUZZ_TEST_BINARIES_OUT_PATHS
2022-11-16T23:47:38.9461793Z ++ bazel-out/k8-opt/bin/tensorflow/security/fuzzing/cc/status_fuzz --list_fuzz_tests
2022-11-16T23:47:38.9522466Z + FUZZ_TESTS=CC_FUZZING.FuzzTest
2022-11-16T23:47:38.9522995Z + cp bazel-out/k8-opt/bin/tensorflow/security/fuzzing/cc/status_fuzz /out/
2022-11-16T23:47:38.9704946Z ++ basename bazel-out/k8-opt/bin/tensorflow/security/fuzzing/cc/status_fuzz
2022-11-16T23:47:38.9721314Z + fuzz_basename=status_fuzz
2022-11-16T23:47:38.9721686Z + chmod -x /out/status_fuzz
2022-11-16T23:47:38.9739806Z + for fuzz_entrypoint in $FUZZ_TESTS
2022-11-16T23:47:38.9740158Z + TARGET_FUZZER=status_fuzz@CC_FUZZING.FuzzTest
2022-11-16T23:47:38.9740499Z + echo '#!/bin/sh
2022-11-16T23:47:38.9740767Z # LLVMFuzzerTestOneInput for fuzzer detection.
2022-11-16T23:47:38.9741036Z this_dir=$(dirname "$0")
2022-11-16T23:47:38.9741259Z chmod +x $this_dir/status_fuzz
2022-11-16T23:47:38.9741603Z status_fuzz --fuzz=CC_FUZZING.FuzzTest -- $@'
2022-11-16T23:47:38.9741900Z + chmod +x /out/status_fuzz@CC_FUZZING.FuzzTest
2022-11-16T23:47:38.9746754Z + for fuzz_main_file in $FUZZ_TEST_BINARIES_OUT_PATHS
2022-11-16T23:47:38.9750779Z ++ bazel-out/k8-opt/bin/tensorflow/security/fuzzing/cc/parseURI_fuzz --list_fuzz_tests
2022-11-16T23:47:38.9794784Z + FUZZ_TESTS=CC_FUZZING.FuzzTest
2022-11-16T23:47:38.9795308Z + cp bazel-out/k8-opt/bin/tensorflow/security/fuzzing/cc/parseURI_fuzz /out/
2022-11-16T23:47:38.9876186Z ++ basename bazel-out/k8-opt/bin/tensorflow/security/fuzzing/cc/parseURI_fuzz
2022-11-16T23:47:38.9884894Z + fuzz_basename=parseURI_fuzz
2022-11-16T23:47:38.9885275Z + chmod -x /out/parseURI_fuzz
2022-11-16T23:47:38.9894797Z + for fuzz_entrypoint in $FUZZ_TESTS
2022-11-16T23:47:38.9895099Z + TARGET_FUZZER=parseURI_fuzz@CC_FUZZING.FuzzTest
2022-11-16T23:47:38.9895451Z + echo '#!/bin/sh

Signed-off-by: David Korczynski <david@adalogics.com>

DavidKorczynski · 2022-11-22T21:58:25Z

@oliverchang I switched this up to focus on fuzztest as it now has a potential fix for #8997 (was missing $ in front of this_dir in the wrapper script). I can do the Tensorflow aspects, which will be only in the Tensorflow project folder afterwards when tensorflow/tensorflow#58646 is merged.

This is ready to go from my perspective.

oliverchang

Thanks! This mostly LGTM. I think just needs some more clarification in terms of documentation.

oliverchang · 2022-11-23T01:18:26Z

infra/base-images/base-builder/compile_fuzztests.sh


 # Bazel target names of the fuzz binaries.
-FUZZ_TEST_BINARIES=$(bazel query 'kind("cc_test", rdeps(..., @com_google_fuzztest//fuzztest:fuzztest_gtest_main))')
+#FUZZ_TEST_BINARIES=$(bazel query 'kind("cc_test", rdeps(..., @com_google_fuzztest//fuzztest:fuzztest_gtest_main))')


Can we just delete this commented line?

oliverchang · 2022-11-23T01:19:48Z

infra/base-images/base-builder/compile_fuzztests.sh

    chmod +x $OUT/$TARGET_FUZZER
  done
 done

 # synchronise coverage directory to bazel generated code.
-if [ "$SANITIZER" = "coverage" ]
+if [[ "$SANITIZER" = "coverage" && ${FUZZTEST_DO_SYNC:-"yes"} == "yes" ]]


Can you add more details on when FUZZTEST_DO_SYNC is needed in comments here?

oliverchang · 2022-11-23T01:20:00Z

infra/base-images/base-builder/compile_fuzztests.sh

                    -e 'bazel-testlogs')"
  for link in $project_folders; do
   if [[ -d "${PWD}"/$link/external  ]]
   then
     rsync -avLk "${RSYNC_FILTER_ARGS[@]}" "${PWD}"/$link/external "${REMAP_PATH}"
   fi
+   if [[ -d "${PWD}"/$link/k8-opt  ]]


Likewise here -- can you add more context in comments for future readers of this code?

oliverchang · 2022-11-23T01:20:56Z

infra/base-images/base-builder/compile_fuzztests.sh

@@ -15,24 +15,34 @@
 #
 ################################################################################

+set -x
+
+if [[ ${FUZZTEST_TARGET_FOLDER:-"unset"} == "unset" ]];


I think we need a bit more documentation on what this is needed for.

DavidKorczynski · 2022-11-24T23:23:53Z

@oliverchang updated the docs, let me know what you think!

Signed-off-by: David Korczynski <david@adalogics.com>

oliverchang · 2022-11-25T02:26:54Z

/gcbrun trial_build.py all --sanitizer address --fuzzing-engine libfuzzer

oliverchang · 2022-11-25T02:27:10Z

Thanks! Let's merge this on Monday to avoid any potential breakages over the weekend.

oliverchang · 2022-11-27T23:16:03Z

/gcbrun trial_build.py all --sanitizer address --fuzzing-engine libfuzzer

oliverchang · 2022-11-27T23:16:18Z

/gcbrun trial_build.py all --sanitizer address --fuzzing-engine libfuzzer

oliverchang · 2022-11-28T07:17:37Z

trial_build failures:

Step #1: bottleneck False
Step #1: numpy False
Step #1: skia False

Errors all look like spurious unrelated. issues.

DavidKorczynski · 2022-12-03T00:11:43Z

@oliverchang am still not seeing results in the dashboard, is this and google/clusterfuzz#2861 deployed?

oliverchang · 2022-12-05T05:39:57Z

Thanks for flagging @DavidKorczynski. We fixed a few more issues that came up in ClusterFuzz, and things should be moving along now. We should hopefully have things visible on the dashbboard in the next day or two.

Enable Fuzztest fuzzers for Tensorflow. This depends on google/fuzztest#79 and eventually a PR on the tensorflow repo with the (to be refined) diff in this PR. Signed-off-by: David Korczynski <david@adalogics.com> Signed-off-by: David Korczynski <david@adalogics.com> Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>

DavidKorczynski added 2 commits November 16, 2022 17:11

infra/tensorflow: make fuzztest work

a68b67e

This is draft. Signed-off-by: David Korczynski <david@adalogics.com>

extend compile_fuzztests.sh with more directories to rsync

c9f527f

This is needed for tensorflow. Signed-off-by: David Korczynski <david@adalogics.com>

DavidKorczynski added 2 commits November 16, 2022 11:59

Make coverage work too for fuzztests.

6033d17

Cleanup is needed. Signed-off-by: David Korczynski <david@adalogics.com>

cleanup

7ffc769

Signed-off-by: David Korczynski <david@adalogics.com>

DavidKorczynski mentioned this pull request Nov 16, 2022

bazel: extend oss-fuzz set up. google/fuzztest#79

Merged

make it based on tensorflow repo

4364b6b

Provide the correct directory in fuzztest wrapper scripts

23abcb1

Signed-off-by: David Korczynski <david@adalogics.com>

DavidKorczynski mentioned this pull request Nov 22, 2022

Make fuzztest fuzzer ready for OSS-Fuzz tensorflow/tensorflow#58646

Closed

cleanup tensorflow diff

7da1833

DavidKorczynski marked this pull request as ready for review November 22, 2022 21:55

DavidKorczynski changed the title ~~infra/tensorflow: make fuzztest work~~ infra: refine fuzztest work Nov 22, 2022

DavidKorczynski added 3 commits November 22, 2022 14:04

nullify tensorflow changes to make this PR only be about fuzztest

0e2dbf2

remove debug logic

48a6b25

nit

fd8b329

oliverchang approved these changes Nov 23, 2022

View reviewed changes

DavidKorczynski added 2 commits November 24, 2022 15:39

Add documentation for the fuzztest build script

bef4b41

Signed-off-by: David Korczynski <david@adalogics.com>

nit

f44cc28

oliverchang approved these changes Nov 25, 2022

View reviewed changes

Merge branch 'master' into tensorflow-fuzztest

3e30076

oliverchang merged commit 529a009 into master Nov 28, 2022

oliverchang deleted the tensorflow-fuzztest branch November 28, 2022 07:17

This was referenced Dec 19, 2022

Add FuzzTest support and connect it to Tensorflow project #9238

Open

tensorflow: fix fuzztest set up #9239

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

infra: refine fuzztest work #8992

infra: refine fuzztest work #8992

DavidKorczynski commented Nov 16, 2022 •

edited

Loading

DavidKorczynski commented Nov 16, 2022

DavidKorczynski commented Nov 17, 2022

DavidKorczynski commented Nov 22, 2022

oliverchang left a comment

oliverchang Nov 23, 2022

oliverchang Nov 23, 2022

oliverchang Nov 23, 2022

oliverchang Nov 23, 2022

DavidKorczynski commented Nov 24, 2022

oliverchang commented Nov 25, 2022

oliverchang commented Nov 25, 2022

oliverchang commented Nov 27, 2022

oliverchang commented Nov 27, 2022

oliverchang commented Nov 28, 2022

DavidKorczynski commented Dec 3, 2022

oliverchang commented Dec 5, 2022

infra: refine fuzztest work #8992

infra: refine fuzztest work #8992

Conversation

DavidKorczynski commented Nov 16, 2022 • edited Loading

DavidKorczynski commented Nov 16, 2022

DavidKorczynski commented Nov 17, 2022

DavidKorczynski commented Nov 22, 2022

oliverchang left a comment

Choose a reason for hiding this comment

oliverchang Nov 23, 2022

Choose a reason for hiding this comment

oliverchang Nov 23, 2022

Choose a reason for hiding this comment

oliverchang Nov 23, 2022

Choose a reason for hiding this comment

oliverchang Nov 23, 2022

Choose a reason for hiding this comment

DavidKorczynski commented Nov 24, 2022

oliverchang commented Nov 25, 2022

oliverchang commented Nov 25, 2022

oliverchang commented Nov 27, 2022

oliverchang commented Nov 27, 2022

oliverchang commented Nov 28, 2022

DavidKorczynski commented Dec 3, 2022

oliverchang commented Dec 5, 2022

DavidKorczynski commented Nov 16, 2022 •

edited

Loading