random doc tests are failing #345
Labels
Comments
|
After a bit of tinkering I came up with this code, which reproduces the issue: use std::env;
use std::path::Path;
use cargo::core::{compiler::CompileMode, Shell, Workspace};
use cargo::ops::{self, CompileOptions, FilterRule, LibRule};
use cargo::util::Config;
fn setup_environment() {
env::set_var(
"RUSTDOCFLAGS",
concat!(
" --persist-doctests target/doctests",
" -Z unstable-options"
),
);
}
fn main() {
cargo::core::enable_nightly_features();
setup_environment();
let manifest = Path::new("/media/hdd/home/projects/hls_m3u8_test/Cargo.toml");
let cwd = manifest.parent().unwrap().to_path_buf();
let home = cargo::util::homedir(&cwd).unwrap();
let mut config = Config::new(Shell::new(), cwd, home);
config
.configure(0u32, None, &None, false, false, false, &None, &[])
.unwrap();
let workspace = Workspace::new(&manifest, &config).unwrap();
let mut compile_options = CompileOptions::new(&config, CompileMode::Doctest).unwrap();
compile_options.filter = ops::CompileFilter::new(
LibRule::Default, // compile the library, so the unit tests can be run filtered
FilterRule::All, // compile the binaries, so the unit tests in binaries can be run filtered
FilterRule::All, // compile the tests, so the integration tests can be run filtered
FilterRule::none(), // specify --examples to unit test binaries filtered
FilterRule::none(), // specify --benches to unit test benchmarks filtered
); // also, specify --doc to run doc tests filtered
let test_options = ops::TestOptions {
compile_opts: compile_options,
no_run: false,
no_fail_fast: false,
};
dbg!(ops::run_tests(&workspace, &test_options, &[]).unwrap());
}removing the |
|
closing this as it does seem to be an issue related to I can reproduce the issue with |
|
Have you raised an issue on the compiler or cargo? I wouldn't want this to disappear into the ether 😄 |
|
I am working on it :) Edit: done |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In my PR (sile/hls_m3u8#44) tests, that work with
cargo testare failing randomly.I ran
cargo tarpaulinmultiple times and noticed that the number of failed tests changes between each run.first run (84 passed; 41 failed)
second run (86 passed; 39 failed)
third run (82 passed; 43 failed)
I produced the above with
cargo tarpaulin --run-types Doctests | grep FAILED >> third_run.txtThe full output with
--debugincluding stdout and stderr is over 100k lines, so I just pasted the "important" lines here https://pastebin.com/m7rY2ZmEThis does happen in CI too https://travis-ci.org/sile/hls_m3u8/jobs/648390858#L614
The text was updated successfully, but these errors were encountered: