make JSON error byte position start at top of file #42973

zackmdavis · 2017-06-29T20:40:01Z

The hi and lo offsets in a span are relative to a CodeMap, but this
doesn't seem to be terribly useful for tool consumers who don't have the
codemap, but might want the byte offset within an actual file?

I couldn't get @killercup's example to run, perhaps due to the limitations of the merely-stage-1 compiler that I built (error was libproc_macro-456500c7095d8fbe.so: cannot open shared object file: No such file or directory)??—but a dummy project confirms that the byte offsets have successfully been changed to be file-relative—

Before:

$ cargo run --message-format json
   Compiling byte_json v0.1.0 (file:///home/ubuntu/byte_json)
{"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `rah`","rendered":null,"spans":[{"byte_end":100,"byte_start":67,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/foo.rs","is_primary":true,"label":null,"line_end":5,"line_start":3,"suggested_replacement":null,"text":[{"highlight_end":11,"highlight_start":1,"text":"fn rah() {"},{"highlight_end":21,"highlight_start":1,"text":"    println!(\"rah!\")"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}}
{"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `alas`","rendered":null,"spans":[{"byte_end":137,"byte_start":102,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/bar.rs","is_primary":true,"label":null,"line_end":3,"line_start":1,"suggested_replacement":null,"text":[{"highlight_end":12,"highlight_start":1,"text":"fn alas() {"},{"highlight_end":22,"highlight_start":1,"text":"    println!(\"alas\");"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}}
{"features":[],"filenames":["/home/ubuntu/byte_json/target/debug/byte_json"],"fresh":false,"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","profile":{"debug_assertions":true,"debuginfo":2,"opt_level":"0","overflow_checks":true,"test":false},"reason":"compiler-artifact","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}}
    Finished dev [unoptimized + debuginfo] target(s) in 0.36 secs
     Running `target/debug/byte_json`
Hello, world!

After:

$ RUSTC=../rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc cargo run --message-format json
   Compiling byte_json v0.1.0 (file:///home/ubuntu/byte_json)
{"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `rah`","rendered":null,"spans":[{"byte_end":35,"byte_start":2,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/foo.rs","is_primary":true,"label":null,"line_end":5,"line_start":3,"suggested_replacement":null,"text":[{"highlight_end":11,"highlight_start":1,"text":"fn rah() {"},{"highlight_end":21,"highlight_start":1,"text":"    println!(\"rah!\")"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}}
{"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `alas`","rendered":null,"spans":[{"byte_end":35,"byte_start":0,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/bar.rs","is_primary":true,"label":null,"line_end":3,"line_start":1,"suggested_replacement":null,"text":[{"highlight_end":12,"highlight_start":1,"text":"fn alas() {"},{"highlight_end":22,"highlight_start":1,"text":"    println!(\"alas\");"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}}
{"features":[],"filenames":["/home/ubuntu/byte_json/target/debug/byte_json"],"fresh":false,"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","profile":{"debug_assertions":true,"debuginfo":2,"opt_level":"0","overflow_checks":true,"test":false},"reason":"compiler-artifact","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}}
    Finished dev [unoptimized + debuginfo] target(s) in 2.59 secs
     Running `target/debug/byte_json`
Hello, world!

Resolves #35164.

r? @jonathandturner

killercup · 2017-06-30T08:10:11Z

Nice! I'll build this branch and give it a try. I'd really like there to be a test for this as well :)

killercup · 2017-06-30T10:41:18Z

So just half an hour after compiling this branch I have good news: My example now outputs

Worked.
Worked.

🎉

shepmaster · 2017-06-30T20:37:56Z

Thank you @zackmdavis! We will have @jonathandturner or a equally qualified reviewer take a look soon.

sophiajt · 2017-07-01T00:14:24Z

@shepmaster - Since I haven't touched this area in a while (or done much with the compiler), may want to hand it to one of the compiler team folks.

Mark-Simulacrum · 2017-07-01T03:59:14Z

r? @jseyfried or @petrochenkov perhaps

jseyfried · 2017-07-05T18:38:16Z

LGTM, but r? @nrc

zackmdavis · 2017-07-05T19:20:22Z

(I wonder if it would be uncharitable for me to speculate that a diffusion of responsibility effect is making this PR subjectively feel more risky than it actually is? The current behavior, in which JSON errors can contain byte_start and byte_end values outside the byte range of the file, can't possibly be correct ...)

aidanhs · 2017-07-05T22:04:26Z

@zackmdavis diffusion of responsibility is one of the reasons the infra team goes through PRs to make sure none of them get left behind, pinging reviewers when appropriate.

Reviewers may cc each other for a whole host of reasons - perhaps someone familiar with the area will realise there are other related pieces that need fixing to make this fix even better.

In short - don't worry, we'll get there eventually!

carols10cents · 2017-07-10T12:39:27Z

friendly ping to keep this on your radar @nrc!

nrc · 2017-07-10T18:47:43Z

I approve of this change, however, we have a back compat guarantee and this is technically a breaking change. I see two paths forward - we leave the existing fields and add new ones for file_relative_byte_start, etc. Or we regard this as a bug fix and change these fields. Ideally we would do a Crater run of a PR with these fields removed, but I fear this will not catch any breakage. I would like to be re-assured that these fields weren't being used in their current form.

Nominated for discussion at the dev-tools meeting today.

Sorry I missed the review notification for this.

nrc · 2017-07-10T20:27:31Z

Discussed at the dev-tools meeting. We thought it would be fine to make this change, assuming that it is not widely used, and the only way to use the current offsets is to use unstable code (libsyntax). As far as we know, @killercup is the only user.

I would like to see a test please!

zackmdavis · 2017-07-13T00:57:09Z

I would like to see a test please!

Do we have an established way to test JSON error output? (Grepping around in the test/ directory doesn't look promising.) The comments-in-run-pass/-fail files" style isn't applicable, and a UI test (which I would assume would work given a compile-flags: --error-output json directive comment) seems unnecessarily brittle (equality of JSON objects is what matters, not byte-for-byte output).

nrc · 2017-07-13T03:15:59Z

@zackmdavis you could make a run-make test, that would let you specify the json error type and compare the results however you like. You might also be able to write a #[test] unit test.

zackmdavis · 2017-07-19T23:40:13Z

9ddda12 adds a run-make test.

(The test fails without this patch, with a reported byte start (respectively end) of 76 (respectively 82), reflecting the 52 bytes of main.rs coming earlier in the CodeMap than where the lint error occurs.)

nrc · 2017-07-20T22:31:35Z

Looks like tests are failing due to missing licenses:

[00:02:36] tidy error: /checkout/src/test/run-make/issue-35164/submodule/mod.rs: incorrect license
[00:02:36] tidy error: /checkout/src/test/run-make/issue-35164/main.rs: incorrect license

nrc · 2017-07-20T22:34:10Z

Looks good other than the licenses (r+)

zackmdavis · 2017-07-21T20:01:31Z

Force-pushed 6a58949, adding the licenses. I regret the error.

Mark-Simulacrum · 2017-07-21T21:06:48Z

@bors r=nrc

bors · 2017-07-21T21:06:49Z

📌 Commit 6a58949 has been approved by nrc

bors · 2017-07-21T23:46:01Z

🔒 Merge conflict

The `hi` and `lo` offsets in a span are relative to a `CodeMap`, but this doesn't seem to be terribly useful for tool consumers who don't have the codemap, but might want the byte offset within an actual file? Resolves rust-lang#35164.

This is in the matter of rust-lang#35164.

zackmdavis · 2017-07-21T23:51:51Z

(rebased)

Mark-Simulacrum · 2017-07-22T00:04:53Z

@bors r=nrc

bors · 2017-07-22T00:04:54Z

📌 Commit bb2b863 has been approved by nrc

bors · 2017-07-22T02:16:06Z

⌛ Testing commit bb2b863 with merge 68be86d...

@killercup

…of_file, r=nrc make JSON error byte position start at top of file The `hi` and `lo` offsets in a span are relative to a `CodeMap`, but this doesn't seem to be terribly useful for tool consumers who don't have the codemap, but might want the byte offset within an actual file? I couldn't get @killercup's [example](#35164 (comment)) to run, perhaps due to the limitations of the merely-stage-1 compiler that I built (error was `libproc_macro-456500c7095d8fbe.so: cannot open shared object file: No such file or directory`)??—but a dummy project confirms that the byte offsets have successfully been changed to be file-relative— **Before:** ``` $ cargo run --message-format json Compiling byte_json v0.1.0 (file:///home/ubuntu/byte_json) {"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `rah`","rendered":null,"spans":[{"byte_end":100,"byte_start":67,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/foo.rs","is_primary":true,"label":null,"line_end":5,"line_start":3,"suggested_replacement":null,"text":[{"highlight_end":11,"highlight_start":1,"text":"fn rah() {"},{"highlight_end":21,"highlight_start":1,"text":" println!(\"rah!\")"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}} {"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `alas`","rendered":null,"spans":[{"byte_end":137,"byte_start":102,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/bar.rs","is_primary":true,"label":null,"line_end":3,"line_start":1,"suggested_replacement":null,"text":[{"highlight_end":12,"highlight_start":1,"text":"fn alas() {"},{"highlight_end":22,"highlight_start":1,"text":" println!(\"alas\");"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}} {"features":[],"filenames":["/home/ubuntu/byte_json/target/debug/byte_json"],"fresh":false,"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","profile":{"debug_assertions":true,"debuginfo":2,"opt_level":"0","overflow_checks":true,"test":false},"reason":"compiler-artifact","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}} Finished dev [unoptimized + debuginfo] target(s) in 0.36 secs Running `target/debug/byte_json` Hello, world! ``` **After:** ``` $ RUSTC=../rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc cargo run --message-format json Compiling byte_json v0.1.0 (file:///home/ubuntu/byte_json) {"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `rah`","rendered":null,"spans":[{"byte_end":35,"byte_start":2,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/foo.rs","is_primary":true,"label":null,"line_end":5,"line_start":3,"suggested_replacement":null,"text":[{"highlight_end":11,"highlight_start":1,"text":"fn rah() {"},{"highlight_end":21,"highlight_start":1,"text":" println!(\"rah!\")"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}} {"message":{"children":[{"children":[],"code":null,"level":"note","message":"#[warn(dead_code)] on by default","rendered":null,"spans":[]}],"code":null,"level":"warning","message":"function is never used: `alas`","rendered":null,"spans":[{"byte_end":35,"byte_start":0,"column_end":2,"column_start":1,"expansion":null,"file_name":"src/bar.rs","is_primary":true,"label":null,"line_end":3,"line_start":1,"suggested_replacement":null,"text":[{"highlight_end":12,"highlight_start":1,"text":"fn alas() {"},{"highlight_end":22,"highlight_start":1,"text":" println!(\"alas\");"},{"highlight_end":2,"highlight_start":1,"text":"}"}]}]},"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","reason":"compiler-message","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}} {"features":[],"filenames":["/home/ubuntu/byte_json/target/debug/byte_json"],"fresh":false,"package_id":"byte_json 0.1.0 (path+file:///home/ubuntu/byte_json)","profile":{"debug_assertions":true,"debuginfo":2,"opt_level":"0","overflow_checks":true,"test":false},"reason":"compiler-artifact","target":{"crate_types":["bin"],"kind":["bin"],"name":"byte_json","src_path":"/home/ubuntu/byte_json/src/main.rs"}} Finished dev [unoptimized + debuginfo] target(s) in 2.59 secs Running `target/debug/byte_json` Hello, world! ``` Resolves #35164. r? @jonathandturner

bors · 2017-07-22T04:35:17Z

☀️ Test successful - status-appveyor, status-travis
Approved by: nrc
Pushing 68be86d to master...

Changelog: Version 1.21.0 (2017-10-12) ========================== Language -------- - [You can now use static references for literals.][43838] Example: ```rust fn main() { let x: &'static u32 = &0; } ``` - [Relaxed path syntax. Optional `::` before `<` is now allowed in all contexts.][43540] Example: ```rust my_macro!(Vec<i32>::new); // Always worked my_macro!(Vec::<i32>::new); // Now works ``` Compiler -------- - [Upgraded jemalloc to 4.5.0][43911] - [Enabled unwinding panics on Redox][43917] - [Now runs LLVM in parallel during translation phase.][43506] This should reduce peak memory usage. Libraries --------- - [Generate builtin impls for `Clone` for all arrays and tuples that are `T: Clone`][43690] - [`Stdin`, `Stdout`, and `Stderr` now implement `AsRawFd`.][43459] - [`Rc` and `Arc` now implement `From<&[T]> where T: Clone`, `From<str>`, `From<String>`, `From<Box<T>> where T: ?Sized`, and `From<Vec<T>>`.][42565] Stabilized APIs --------------- [`std::mem::discriminant`] Cargo ----- - [You can now call `cargo install` with multiple package names][cargo/4216] - [Cargo commands inside a virtual workspace will now implicitly pass `--all`][cargo/4335] - [Added a `[patch]` section to `Cargo.toml` to handle prepublication dependencies][cargo/4123] [RFC 1969] - [`include` & `exclude` fields in `Cargo.toml` now accept gitignore like patterns][cargo/4270] - [Added the `--all-targets` option][cargo/4400] - [Using required dependencies as a feature is now deprecated and emits a warning][cargo/4364] Misc ---- - [Cargo docs are moving][43916] to [doc.rust-lang.org/cargo](https://doc.rust-lang.org/cargo) - [The rustdoc book is now available][43863] at [doc.rust-lang.org/rustdoc](https://doc.rust-lang.org/rustdoc) - [Added a preview of RLS has been made available through rustup][44204] Install with `rustup component add rls-preview` - [`std::os` documentation for Unix, Linux, and Windows now appears on doc.rust-lang.org][43348] Previously only showed `std::os::unix`. Compatibility Notes ------------------- - [Changes in method matching against higher-ranked types][43880] This may cause breakage in subtyping corner cases. [A more in-depth explanation is available.][info/43880] - [rustc's JSON error output's byte position start at top of file.][42973] Was previously relative to the rustc's internal `CodeMap` struct which required the unstable library `libsyntax` to correctly use. - [`unused_results` lint no longer ignores booleans][43728] [42565]: rust-lang/rust#42565 [42973]: rust-lang/rust#42973 [43348]: rust-lang/rust#43348 [43459]: rust-lang/rust#43459 [43506]: rust-lang/rust#43506 [43540]: rust-lang/rust#43540 [43690]: rust-lang/rust#43690 [43728]: rust-lang/rust#43728 [43838]: rust-lang/rust#43838 [43863]: rust-lang/rust#43863 [43880]: rust-lang/rust#43880 [43911]: rust-lang/rust#43911 [43916]: rust-lang/rust#43916 [43917]: rust-lang/rust#43917 [44204]: rust-lang/rust#44204 [cargo/4123]: rust-lang/cargo#4123 [cargo/4216]: rust-lang/cargo#4216 [cargo/4270]: rust-lang/cargo#4270 [cargo/4335]: rust-lang/cargo#4335 [cargo/4364]: rust-lang/cargo#4364 [cargo/4400]: rust-lang/cargo#4400 [RFC 1969]: rust-lang/rfcs#1969 [info/43880]: rust-lang/rust#44224 (comment) [`std::mem::discriminant`]: https://doc.rust-lang.org/std/mem/fn.discriminant.html

rust-highfive assigned sophiajt Jun 29, 2017

shepmaster added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 30, 2017

Mark-Simulacrum assigned jseyfried and unassigned sophiajt Jul 1, 2017

rust-highfive assigned nrc and unassigned jseyfried Jul 5, 2017

nrc added I-nominated T-dev-tools Relevant to the dev-tools subteam, which will review and decide on the PR/issue. labels Jul 10, 2017

brson added the relnotes Marks issues that should be documented in the release notes of the next release. label Jul 10, 2017

nrc added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed I-nominated S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 10, 2017

zackmdavis force-pushed the json_byte_position_to_start_at_top_of_file branch from 9ddda12 to 6a58949 Compare July 21, 2017 20:00

zackmdavis added 2 commits July 21, 2017 16:50

make JSON error byte position start at top of file

6043ce9

The `hi` and `lo` offsets in a span are relative to a `CodeMap`, but this doesn't seem to be terribly useful for tool consumers who don't have the codemap, but might want the byte offset within an actual file? Resolves rust-lang#35164.

run-make test for JSON error byte start/end

bb2b863

This is in the matter of rust-lang#35164.

zackmdavis force-pushed the json_byte_position_to_start_at_top_of_file branch from 6a58949 to bb2b863 Compare July 21, 2017 23:51

bors merged commit bb2b863 into rust-lang:master Jul 22, 2017

zackmdavis deleted the json_byte_position_to_start_at_top_of_file branch January 13, 2018 07:43

killercup mentioned this pull request May 6, 2018

Error applying replacements from module files rust-lang/rustfix#92

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make JSON error byte position start at top of file #42973

make JSON error byte position start at top of file #42973

zackmdavis commented Jun 29, 2017

killercup commented Jun 30, 2017

killercup commented Jun 30, 2017

shepmaster commented Jun 30, 2017

sophiajt commented Jul 1, 2017 •

edited

Loading

Mark-Simulacrum commented Jul 1, 2017

jseyfried commented Jul 5, 2017

zackmdavis commented Jul 5, 2017

aidanhs commented Jul 5, 2017

carols10cents commented Jul 10, 2017

nrc commented Jul 10, 2017

nrc commented Jul 10, 2017

zackmdavis commented Jul 13, 2017

nrc commented Jul 13, 2017

zackmdavis commented Jul 19, 2017

nrc commented Jul 20, 2017

nrc commented Jul 20, 2017

zackmdavis commented Jul 21, 2017

Mark-Simulacrum commented Jul 21, 2017

bors commented Jul 21, 2017

bors commented Jul 21, 2017

zackmdavis commented Jul 21, 2017

Mark-Simulacrum commented Jul 22, 2017

bors commented Jul 22, 2017

bors commented Jul 22, 2017

bors commented Jul 22, 2017

make JSON error byte position start at top of file #42973

make JSON error byte position start at top of file #42973

Conversation

zackmdavis commented Jun 29, 2017

killercup commented Jun 30, 2017

killercup commented Jun 30, 2017

shepmaster commented Jun 30, 2017

sophiajt commented Jul 1, 2017 • edited Loading

Mark-Simulacrum commented Jul 1, 2017

jseyfried commented Jul 5, 2017

zackmdavis commented Jul 5, 2017

aidanhs commented Jul 5, 2017

carols10cents commented Jul 10, 2017

nrc commented Jul 10, 2017

nrc commented Jul 10, 2017

zackmdavis commented Jul 13, 2017

nrc commented Jul 13, 2017

zackmdavis commented Jul 19, 2017

nrc commented Jul 20, 2017

nrc commented Jul 20, 2017

zackmdavis commented Jul 21, 2017

Mark-Simulacrum commented Jul 21, 2017

bors commented Jul 21, 2017

bors commented Jul 21, 2017

zackmdavis commented Jul 21, 2017

Mark-Simulacrum commented Jul 22, 2017

bors commented Jul 22, 2017

bors commented Jul 22, 2017

bors commented Jul 22, 2017

sophiajt commented Jul 1, 2017 •

edited

Loading