Replace Autorest with azcore

[thomas11]

This PR introduces a new implementation of AzureClient that's based on azcore instead of the deprecated autorest. It's part of [Epic] Replace deprecated REST and auth packages in Azure Native #3576. The initial commit is by @mikhailshilkov, thanks!

The PR is rather large. I've attempted to keep logically meaningful individual commits, so reviewing commit by commit might be helpful.

The high-level view of the changes is:

• Implement a new AzureClient that uses azcore. The AzureClient interface remains unchanged.
  • azcore implements polling for long-running requests internally, using the Location header. That isn't always a qualified URL, as I've observed, so there's new code to make the Location qualified.
• azcore wants to authenticate via the new azidentity package which we're not using yet. We do already have a bridge, however, that gets tokens from the current auth stack and serves them via azidentity's interface. We pass that bridge into the new Azure client.
• Using the new client is a small change in provider.go. Since regressions of this change could be very impactful and blocking, there's an escape hatch PULUMI_USE_AUTOREST environment variable to revert to the old client.
• "Fix older bug in CanCreate" and "TokenCredential bridge: use correct environment" are fixes for already existing bugs that I've found.

Integration testing is essentially provided by the existing test suite since all tests now use the new Azure client.

Unit testing is still rudimentary (although already better than the old client's) and will be extended.

[github-actions]

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

[codecov]

Codecov Report

Attention: Patch coverage is 78.60082% with 52 lines in your changes missing coverage. Please review.

Project coverage is 58.97%. Comparing base (10f599c) to head (0968499).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
provider/pkg/azure/client_azcore.go	80.88%	22 Missing and 21 partials ⚠️
provider/pkg/provider/provider.go	56.25%	6 Missing and 1 partial ⚠️
provider/pkg/provider/auth.go	0.00%	1 Missing ⚠️
provider/pkg/provider/crud/crud.go	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3536      +/-   ##
==========================================
+ Coverage   58.39%   58.97%   +0.57%     
==========================================
  Files          67       68       +1     
  Lines        8418     8656     +238     
==========================================
+ Hits         4916     5105     +189     
- Misses       3054     3081      +27     
- Partials      448      470      +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mikhailshilkov]

This looks promising, also the code is quite easy to read! It's also nice how it's almost a drop-in replacement for the old client.

Some comments are below. I'm slightly concerned some of my findings don't result in test failures - it's hard to trust the test suite to catch all errors, it seems.

[mikhailshilkov]

On the rollout strategy, I suggest we should start with the opt-in strategy: use the old auth by default unless the flag is supplied. We can then seek opportunities for deliberate testing of the feature internally and externally with users that benefit from this change (what are those scenarios?)

[danielrbradley]

Overall looks great to me. Couple of minor questions inline.

[danielrbradley]

Not for this PR, but perhaps azureClient should be a pointer and should not be created until we've called configure? Is there some reason we need a default client implementation for use before we've configured the provider that you know of?

[thomas11]

The only reason I know is that azureClient would then be a pointer to an interface which is annoying to use: (*k.azureClient).Get.

[thomas11]

We can then seek opportunities for deliberate testing of the feature internally and externally with users that benefit from this change (what are those scenarios?)

It might be a hard sell as the benefits are all internal - get off an unsupported dependency, receive security updates and bug fixes, etc.

The next step, basing the auth stack on the new and supported azidentity, should close some user-visible bugs: #2432, #2734, maybe #957.

[mikhailshilkov]

It might be a hard sell as the benefits are all internal - get off an unsupported dependency, receive security updates and bug fixes, etc.

We'll have to keep the dependency for a while regardless of the default value. It there's little immediate user value (upside), we should probably be even more careful not to break anyone (downside).

The next step, basing the auth stack on the new and supported azidentity, should close some user-visible bugs: #2432, #2734, maybe #957.

Sounds like a great motivation to get somebody to try the new auth after those are shipped.

[thomas11]

It's weird that asyncStyle is used like a boolean flag here; the value doesn't matter. It's the same in the autorest client.