Format only modified files

[albertlarsan68]

As discussed on #105688, this makes x fmt only format modified files.

Fixes #105688

[rustbot]

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

[albertlarsan68]

r? @jyn514

[matthiaskrgr]

🤔 what if I change something in rustfmt.toml and want to apply this change to the repo, will it still be picked up?

[Mark-Simulacrum]

x.py fmt takes ~1.6 seconds (2.4 seconds with an empty disk cache) for me. Is it much slower for others?

I'm not sure optimizing a few seconds is worth doing -- most of the time, the expectation is that folks are enabling rustfmt on save in their editor, so running it isn't even necessary (this is the case for me), but even without that it should be quite fast to run it.

In any case, as a separate item, we need to make sure that we're running it against all files on changes to the rustfmt binary (and other such wide-spread changes, like @matthiaskrgr notes).

[matthiaskrgr]

Also, I think it is common to
git fetch upstream
git checkout workbranch
git rebase upstream/master
x.py fmt

to fix the format differences introduced by the merge/rebase, will rustfmt still recognize files that were changed but committed as modified?

[albertlarsan68]

🤔 what if I change something in rustfmt.toml and want to apply this change to the repo, will it still be picked up?

You can pass any path and the early culling is not applied.

---

x.py fmt takes ~1.6 seconds (2.4 seconds with an empty disk cache) for me. Is it much slower for others?

It takes me over 30 secs, but with this modification it now only takes a few seconds when no file changed.

I'm not sure optimizing a few seconds is worth doing -- most of the time, the expectation is that folks are enabling rustfmt on save in their editor, so running it isn't even necessary (this is the case for me), but even without that it should be quite fast to run it.

See #105688 for the motivation

In any case, as a separate item, we need to make sure that we're running it against all files on changes to the rustfmt binary (and other such wide-spread changes, like matthiaskrgr notes).

I still need to implement this part

---

will rustfmt still recognize files that were changed but committed as modified?

This seeks for files modified between the rust-lang/rust's master and the current tree.

[matthiaskrgr]

For the record x.py fmt (it already runs in parallel) only takes 6 seconds on a 2016 i5-7200U dual core here 🤔 (on the entire repo, but no files actually modified)

[jyn514]

6 seconds sounds about right; for me at least it's too slow to run format-on-save.

If we want to limit the impact, we can make it opt-in and run it only in the pre-push hook, but I would like to eventually turn it on globally or make it opt-out.

[Mark-Simulacrum]

for me at least it's too slow to run format-on-save.

What do you mean by this? I don't run x.py fmt to format on save, I run rustfmt directly. x.py fmt is too slow on save, but that's fine; you're not supposed to use it for that. (Or should pass it the current file).

If we want to limit the impact, we can make it opt-in and run it only in the pre-push hook, but I would like to eventually turn it on globally or make it opt-out.

I think making it opt-in is also good. Regardless, please do make sure that CI is not using this optimization, because I don't trust it to not cause us problems down the line.

[chenyukang]

x fmt will take about 8 seconds in my local, it would be nice to optimize to 1 second.

[chenyukang]

Benchmark 1: ./x fmt
  Time (mean ± σ):     12.246 s ±  1.504 s    [User: 61.289 s, System: 5.698 s]
  Range (min … max):   10.908 s … 13.830 s    4 runs

😒 , I work in WSL, it seems really slow, maybe need to buy a new device for format-on-save!

[albertlarsan68]

@chenyukang Are those benches made with this branch, or master?

[chenyukang]

@chenyukang Are those benches made with this branch, or master?

it's master.

[jyn514]

🤔 what if I change something in rustfmt.toml and want to apply this change to the repo, will it still be picked up?

You can pass any path and the early culling is not applied.

That's not quite enough - changes in rustfmt.toml will affect all files in the repository. Ideally the contents of that file would be part of the stamp so we know when they change. But I'm ok with needing to opt in with x fmt --all or something like that; seems ok if people are explicitly changing the settings.

[albertlarsan68]

I think currently you can do x fmt . and it will not apply the culling.

[albertlarsan68]

It doesn't correctly detect files right now, will work on that. @rustbot author

[the8472]

If ./x fmt is slow you might want to check if you have any unneeded generated directories (e.g. node_modules or obj) in the rust project.

I work in WSL, it seems really slow, maybe need to buy a new device for format-on-save!

If you're using wsl2 then make sure you have the rust project checked out in a directory mounted as the native linux filesystem (e.g. ext4) as part of the disk image. That should be much faster than the 9p mounts that are used to access the windows filesystem.

You also should check if you have any generated directories other than build or target in your rust project. Running docker CI sometimes generates obj and node_modules for example which increase the time that x spends in traversing the directory tree.

[jyn514]

@the8472 on a Windows 10 laptop with antivirus scanning excluded for the folder and no untracked files other than build/, x fmt takes 8 seconds to execute for me.

[the8472]

"no untracked files" can be misleading because x still dives into gitignored directories when looking for git roots.

[Mark-Simulacrum]

I wonder if switching from using ignore + adding ignored paths to the file listing, we should (when git is available) use git ls-files. That should much more efficiently skip ignored paths, I would expect. (It runs in ~6ms on a rust checkout for me).

That said, at least for me 80% of x.py fmt (which takes ~3 seconds) is in rustfmt itself, so it's not clear that significant improvements from which files we visit will be had. I wonder what the split there is like on non-Linux(?) platforms, maybe we spend much more time in directory traversal. Presumably rustfmt itself is about as fast anywhere, regardless of platform.

[jyn514]

"no untracked files" can be misleading because x still dives into gitignored directories when looking for git roots.

I used git status --ignored to make sure there weren't ignored files.

I wonder if switching from using ignore + adding ignored paths to the file listing, we should (when git is available) use git ls-files. That should much more efficiently skip ignored paths, I would expect. (It runs in ~6ms on a rust checkout for me).

👍 seems useful and simpler than the current approach

That said, at least for me 80% of x.py fmt (which takes ~3 seconds) is in rustfmt itself, so it's not clear that significant improvements from which files we visit will be had. I wonder what the split there is like on non-Linux(?) platforms, maybe we spend much more time in directory traversal. Presumably rustfmt itself is about as fast anywhere, regardless of platform.

How did you measure this? Happy to report what it's like on Windows if you know how.

[jyn514]

That said, at least for me 80% of x.py fmt (which takes ~3 seconds) is in rustfmt itself, so it's not clear that significant improvements from which files we visit will be had.

Wait hold on, this doesn't make a lot of sense to me. Presumably rustfmt runs in about the same time on files that need formatting and those that don't, so visiting less files in the first place should still be a big improvement even if rustfmt's runtime is the limiting factor.

[Mark-Simulacrum]

Sorry, I meant "how we visit files", not "which".

[Mark-Simulacrum]

How did you measure this? Happy to report what it's like on Windows if you know how.

I measured by trying to produce a flamegraph (with perf on Linux). Not sure what is available on windows like that.

[albertlarsan68]

@the8472 Here is a proof that hyperfine's user/system time is incorrect in this case:

When directly invoking python, the user/system time is the same tiny amount regardless of the work done.

When indirecting through powershell (which is what x does on windows), the user/system time is bigger than direct python, but the same regardless of the work done.

When subtracting the user/system time from the powershell times, we get roughly the same times as their direct counterparts.
So, the user/system times are measuring the powershell startup time and the search for the correct python executable.

Benchmark 1: python3.10 .\x.py
  Time (mean ± σ):      2.820 s ±  0.178 s    [User: 0.008 s, System: 0.037 s]
  Range (min … max):    2.671 s …  3.205 s    10 runs

Benchmark 2: python3.10 .\x.py fmt
  Time (mean ± σ):     16.022 s ±  1.016 s    [User: 0.013 s, System: 0.037 s]
  Range (min … max):   14.598 s … 17.467 s    10 runs

Benchmark 3: python3.10 .\x.py fmt .
  Time (mean ± σ):     67.320 s ± 15.144 s    [User: 0.014 s, System: 0.050 s]
  Range (min … max):   55.116 s … 100.146 s    10 runs

Benchmark 4: x
  Time (mean ± σ):     19.698 s ±  3.657 s    [User: 8.633 s, System: 5.378 s]
  Range (min … max):   16.174 s … 25.800 s    10 runs

Benchmark 5: x fmt
  Time (mean ± σ):     37.601 s ±  9.312 s    [User: 9.065 s, System: 5.418 s]
  Range (min … max):   28.874 s … 60.072 s    10 runs

Benchmark 6: x fmt .
  Time (mean ± σ):     88.858 s ± 18.543 s    [User: 9.158 s, System: 5.678 s]
  Range (min … max):   72.185 s … 131.717 s    10 runs

[the8472]

Which hyperfine version are you using? The user time accounting was changed in 1.15, released in september.

[albertlarsan68]

I do use hyperfine v1.15.0

[jyn514]

I think the discussion about benchmarking is important to have but not necessarily relevant to this PR. Formatting only modified files will benefit everyone, not just @albertlarsan68, and I think the impact of a bug is limited enough it's ok to land given that we don't have specific code we think is buggy.

@albertlarsan68 if you add that line in the changelog I mentioned in #105702 (review), I think we can land this :)

[albertlarsan68]

@rustbot ready
@jyn514 Has the changelog entry a good balance between information quantity and brevity?

[jyn514]

Looks great, thanks!

@bors r+ rollup (not tested in CI)

[bors]

📌 Commit 0b942a8 has been approved by jyn514

It is now in the queue for this repository.