Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have a mode that crashes on all panics #8

Open
jakubadamw opened this issue Jul 21, 2020 · 2 comments
Open

Have a mode that crashes on all panics #8

jakubadamw opened this issue Jul 21, 2020 · 2 comments
Labels
enhancement New feature or request

Comments

@jakubadamw
Copy link
Owner

jakubadamw commented Jul 21, 2020

So with #6 and #7 merged in rutenspitz no longer crashes in a fuzz cycle on a panic when both the implementation and the model panic in accordance. This change in behaviour is fine, since the main purpose of this crate is to observe the equivalence of two models. But I think it may be useful to use it also to find panics on malformed input (well, a series of operations to a model). This is exactly what the url::Url fuzzer did, although it did in a rather convoluted way by using url::Url as the supposed doppelganger of url::Url itself.

Rather than doing that we should have a mode which turns off all comparisons, essentially always using

rutenspitz/lib/src/lib.rs

Lines 482 to 486 in ff2c1b2

pub fn execute(self, tested: &mut #tested) {
match &self {
#(#method_tests),*
}
}
for individual operation tests, without using (and requiring) a reference model at all.

This is perhaps somewhat out of scope given the name of this crate but nonetheless seems like a useful extension.

@jakubadamw jakubadamw added the enhancement New feature or request label Jul 21, 2020
@Shnatsel
Copy link

This sounds like a "don't compare, just fuzz" mode to me. Several people including myself attempted implementing that independently, see https://github.com/Eh2406/auto-fuzz-test

But that's just a PoC, so it didn't use any proc macros, just filling in values in a template.

@Shnatsel
Copy link

You know, a mode without comparison to a reference model, but also without considering panics as failures would indeed be very useful. For example, it would have found CVE-2018-1000810 if run on std::String

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants