more lintcheck todos

[matthiaskrgr]

Done:
[x] add a LINTCHECK_TOML env var or something like that https://rust-lang.zulipchat.com/#narrow/stream/257328-clippy/topic/Meeting.202021-02-09 (#6735)
[x] check how clippy ICEs are handled and collect the names of crates that clippy ICEs on (#6708)
[x] parallelize/check multiple crates at a time? (we are mostly interested in the lint results of the root crate and we can only check one root crate at a time... (#6764))
[x] better error handling, currently cargo-metadata stderr ends up in the log (for example: dtolnay/syn#967 , also: #6508) (avoided by #6744 )
[x] allow/deny lints via crates.toml file (details #6744 (comment) ) ( #6750 )
[x] auto-rerun lintcheck
right now we have to manually remove target/lintcheck/shared_target_dir after clippy changed in order to rerun lintcheck.
It would be nice if lintcheck could check the modification date of the clippy binaries and the latest lintcheck log and auto-rerun if clippy-binary is younger than the logs (#6780)
[x] display some short stats on how many new warnings of what lints were found after running ( #6800 )
[x] add a Config object that eats the clap config and holds stuff like the lintcheck output path and toml path etc (#6813)
[x] fix mode? (runs clippy --fix and checks if we can apply all suggestions) (#6848)
[x] move lintcheck into its own crate (out of clippy-dev) (#6884)
[x] set up a new repo with gha and run lintcheck every 24 hours(?) and commit the logs into the repo (WIP: https://github.com/flip1995/clippy-lintcheck / https://github.com/matthiaskrgr/clippy-lintcheck )

[x] add a small test to ci (1-2 crates) that makes sure that lintcheck works (test is added but not checked on ci due to time constraints)

To do:

[ ] check cratesources with many deps first in par_iter mode (see #6764 (comment) )
This will require some more refactoring because we first need to download all the crates before we can count the total number of deps

If you have any more ideas, feel free to dump them here. :)

[matthiaskrgr]

We can obtain a potential list of crates to check via cargo prefetch --top-downloads=400 --list

[matthiaskrgr]

I've seen a couple of cases where clippy would use absolute paths to sources when it checks some file that has been auto-generated via build.rs and put somewhere inside the target dir.
rust-lang/rust#82276
These might cause trouble in the log diff.

[matthiaskrgr]

At https://github.com/matthiaskrgr/clippy-lintcheck I currently run lintcheck with the default crates and the extended crates/toml every 12 hours.
Logs are pushed directly into the repo.
Still needs some observation to make sure everything actually works as intended. :)

Edit: nominating so I don't forget to mention this at the next clippy meeting.

[matthiaskrgr]

More todos discussed in the clippy meeting: https://rust-lang.zulipchat.com/#narrow/stream/257328-clippy/topic/Meeting.202021-04-06

1. ask infra about rust-lang resources
   edit: https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/moving.20clippy-lintcheck.20to.20rust-lang.20org

2. move lintcheck over to the rust-lang repo

3. remove the lintcheck logs from the clippy repo and add the file to .gitignore

4. make lintcheck download the latest log from the github repo into the clippy repo for diffing.