Changed how fuzz tests are iterated to allow powerloss-fuzz testing

Instead of iterating over a number of seeds in the test itself, the
seeds are now permuted as a part of normal test defines.

This lets each seed take advantage of other test features, mainly the
ability to test powerlosses heuristically.

This is probably how it should have been done in the first place, but
the permutation tests can't do this since the number of permutations
changes as the size of the test input changes. The test define system
can't handle that very well.

The tradeoffs here are:

- We can't do cross-fuzz checks, such as the balance checks in the rbyd
  tests, though those really should be moved to benchmarks anyways.

- The large number of cheap fuzz permutations skews the total
  permutation count, though I'm not sure this matters.

  before: 3083 permutations (-Gnor)
  after: 409893 permutations (-Gnor)

scripts/bench.py

@@ -758,20 +758,22 @@ def find_ids(runner, bench_ids=[], **args):
     # find suite/case by id
     bench_ids_ = []
     for id in bench_ids:
+        # strip permutation
+        name, *_ = id.split(':', 1)
         bench_ids__ = []
         # resolve globs
-        if '*' in id:
+        if '*' in name:
             bench_ids__.extend(suite
                 for suite in expected_suite_perms.keys()
-                if fnmatch.fnmatch(suite, id))
+                if fnmatch.fnmatch(suite, name))
             bench_ids__.extend(case_
                 for case_ in expected_case_perms.keys()
-                if fnmatch.fnmatch(case_, id))
+                if fnmatch.fnmatch(case_, name))
         # literal suite
-        elif id in expected_suite_perms:
+        elif name in expected_suite_perms:
             bench_ids__.append(id)
         # literal case
-        elif id in expected_case_perms:
+        elif name in expected_case_perms:
             bench_ids__.append(id)
 
         # no suite/case found? error

scripts/test.py

@@ -767,20 +767,22 @@ def find_ids(runner, test_ids=[], **args):
     # find suite/case by id
     test_ids_ = []
     for id in test_ids:
+        # strip permutation
+        name, *_ = id.split(':', 1)
         test_ids__ = []
         # resolve globs
-        if '*' in id:
+        if '*' in name:
             test_ids__.extend(suite
                 for suite in expected_suite_perms.keys()
-                if fnmatch.fnmatch(suite, id))
+                if fnmatch.fnmatch(suite, name))
             test_ids__.extend(case_
                 for case_ in expected_case_perms.keys()
-                if fnmatch.fnmatch(case_, id))
+                if fnmatch.fnmatch(case_, name))
         # literal suite
-        elif id in expected_suite_perms:
+        elif name in expected_suite_perms:
             test_ids__.append(id)
         # literal case
-        elif id in expected_case_perms:
+        elif name in expected_case_perms:
             test_ids__.append(id)
 
         # no suite/case found? error