[WIP] use Layout as a source data of classification related tasks

[vladima]

Early WIP
F# compiler uses Layout structure for pretty printing but ultimately outputs result as a string. This become a culprit in classification related scenarios since in order to colorize everything nicely we have to effectively recover original Layout. This PR introduces TaggedText annotations for text chunks in Layout. Later annotated text chunks can easily be converted from the Layout to Roslyn TaggedText.

Remaining items:

• - add glyphs to quick info. Possible approach is proposed #1 but I think we can do this without doing extra request for symbol.
• - make sure that formatting in final result matches current version (though differences might be ok if they improve the experience)
• - fix existing tests
• - clean-up API surface - currently PR changes shapes/signatures of existing types and methods which will be a breaking change for anyone who consumed previous version of the compiler. Existing API that deal with string should be reverted back and new API that use Layout should be added instead of replacing existing ones.

per @vasily-kirichenko' comment:

• - bugfixes

Quick info:

Signature help:

Completion details:

[dungpa]

Woohoo! I've been waiting for this PR for so long.

[vasily-kirichenko]

It's just fantastic 🥇 🥇 🥇

[vasily-kirichenko]

As you have not mentioned any bugs in the to do list, I feel free to report first ones :)

Here type should be highlighted as keyword:

The namespace should not be highlighted, bool keyword is not highlighted, ThreadPool type is not highlighted:

[saul]

This look absolutely fantastic!

Another issue is that the indentation seems to be lost:

[vasily-kirichenko]

@vladima do I understand right that this color information can be used for normal semantic code highlighting in the editor?

[vladima]

I'm not sure that I understand the question: compiler uses layouts to pretty print various kinds of messages but it is not used to format the source text itself. For classification in the editor one should use IEditorClassificationService which I think F# language service already does

[saul]

Another bug - type parameters include the apostrophe prefix in the colouring. The editor doesn't colour the apostrophe when doing syntax highlighting - I'm not sure which is the preferred option here.

Another positive is that FSI's output can now be colourised too:

[cloudRoutine]

@saul including the apostrophe is correct, the current highlighting in the editor is wrong

[vladima]

per F# spec:

typar :=
_ -- anonymous variable type
'ident -- type variable
^ident -- static head-type type variable

so I read it as ' is considered part of the name

[vladima]

status update: fixed indentation and classification of types inside xml doc comments

[saul]

Another small issue - I've noticed that "Full name: XXX" seems to be indented by a couple of spaces. This is not the case on master without your PR, so looks like a bug

[dsyme]

Wow, this is fantastic!

[dsyme]

• clean-up API surface - currently PR changes shapes/signatures of existing types and methods which will be a breaking change for anyone who consumed previous version of the compiler. Existing API that deal with string should be reverted back and new API that use Layout should be added instead of replacing existing ones.

This is OK as long as FSharp.Core is stable - for various reasons we have been making pretty regular breaking changes in the surface arera of FSharp.Compiler.Service, and bumping major version numbers on the nuget package when we do.

[dsyme]

If anyone wants to make some basic improvements like

• replacing struct/end by [<Struct>]
• eliding interface/end altogether
• eliding class/end altogether

etc that would also be great

[vladima]

@dotnet-bot test this please

[vasily-kirichenko]

Instance members are indented:

But static members sometimes are not:

(ok)

(wrong)

Actually I've found the only case where it's broken - Async type. The tooltip is huge, maybe it's the root cause.

[vasily-kirichenko]

Another catch - namespace and module keywords are not highlighted:

[vladima]

Colors for module and namespace keywords are fixed. As for indentation I do agree that it is inconsistent with other cases however I'd prefer to fix this in a separate PR. Reasons:

• technically this is not a regression since it matches existing behavior for types with hidden representation
• current PR is already quite big and adding more things to it will make its review more complicated.

// cc @KevinRansom and @dsyme for opinions.

[saul]

One last issue @vladima, see how the separators look in current master:

Note that there are no blank lines around the separator, and there is no separator at the end of the tooltip. See @vasily-kirichenko's first screenshot in his last comment:

[cartermp]

@vladima Agreed re: indentation - unless it's a minor change, adding more improvements to this PR will make it more difficult to review and get merged.

[cartermp]

@vasily-kirichenko I've noticed the Async tooltip exploding in size in #1973 as well. Perhaps there's some strange code path executed for Async where it's not truncated, because in most situations it cuts off with a ... about 1/3 of the way through listing all the members. I haven't been able to reproduce that kind of behavior for anything other than Async.

[vladima]

turned out fix is fairly simple:

[dsyme]

This looks good to me. Unless there are other concerns I think we should accept this

[KevinRansom]

@vladima

Thank you this is awesome work ...

Kevin