Simplify diagnostics story for custom ALC

[davidfowl]

Description

When issues like these come up, it's a game of wack-a-mole to figure out what objects from a custom ALC are rooted by objects in the default ALC. Can we do something low tech to make this a bit easier? Maybe an SOS command.

Regression?

No

cc @vitek-karas @janvorli

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke, @CoffeeFlux, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

When issues like these come up, it's a game of wack-a-mole to figure out what objects from a custom ALC are rooted by objects in the default ALC. Can we do something low tech to make this a bit easier? Maybe an SOS command.

Regression?

No

cc @vitek-karas @janvorli

Author:	davidfowl
Assignees:	-
Labels:	area-AssemblyLoader-coreclr, area-System.Runtime, untriaged
Milestone:	-

[jkotas]

Can we do something low tech to make this a bit easier? Maybe an SOS command.

The sos command for this is gcroot. Here is an example for how it can be used to investigate leaking ALCs #31377 (comment)

Leaking ALCs are managed memory leaks, so most tutorials and tricks for debugging managed memory leaks are applicable too.

[davidfowl]

Yea, I know but the repro I have is extremely unreliable, so maybe it's a bug. I think would be interesting to be able to identify which ALC an object came from using SOS.

[davidfowl]

Oh that thread helped. So LoaderAllocator is the magic I should be looking for.

[davidfowl]

Turns out event sources are really bad in this regard (probably all diagnostics really). It's not self contained enough to be unloadable.

[vitek-karas]

Just curious - do you think it happens to all event sources, or just those somebody is listening to? In theory we should be able to "do nothing" if nobody's listening... but maybe not.

Also - this should be fixable - we just need to make the references to event sources weak references (and handle the case where it goes away).

[davidfowl]

I hope we can solve it in general. EventSources basically register with etw or event pipe and end up getting finalized. Now it makes sense why I was seeing this repro be unreliable, at least when I was trying to narrow it down. I'll try to make a repro today

[janvorli]

@davidfowl are you aware of the following doc I have written on using and debugging unloadability? https://docs.microsoft.com/en-us/dotnet/standard/assembly/unloadability#debug-unloading-issues

[davidfowl]

Nope! That's super useful! Also I figured out this issue, it's pretty cool one 😄

[vitek-karas]

@davidfowl Can you please create a separate issue for the EventSource unloadability? (Since you have the data/samples)

[davidfowl]

Turns out EventSources weren't the problem. Well I should say, they get finalized eventually.

Here's the explanation dotnet/aspnetcore#31637 (comment)

TL;DR, async stack unwinding will root things (which makes sense). So extra care is needed when you execute async code in another load context. It's super easy to keep things alive.

[davidfowl]

We should add this one to the doc. Basically the equivalent of ExecuteAndUnload but the async version