Remove Extraneous Caching of Resource Schemas

[bflad]

terraform-plugin-mux version

v0.10.0

Relevant provider source code

N/A

Terraform Configuration Files

N/A

Expected Behavior

terraform-plugin-mux should not cache resource schemas unless there is a really good reason as it inflates peak memory usage for providers. Currently all resource schemas are cached, which is problematic with larger providers (either in resource count and/or schema size). We are exploring whether Terraform itself can limit the amount of resource schema data being requested from providers to reduce overall memory utilization on both sides of the protocol. The resource schema caching in this Go module hinders any efforts in that regard though.

Actual Behavior

terraform-plugin-mux caches all resource schemas when creating a mux server. This caching is currently used with:

• Pre-loading the GetProviderSchema RPC response based on all the underlying providers on mux server creation, which is itself because the mux server creation has historically implemented validation logic such as duplicate resource type detection
• For protocol version 5 servers, verifies whether a PlanResourceChange RPC being sent to the underlying provider is a destroy plan, which is server capability based and the detection is based on fully decoding the ProposedNewState data using the actual schema

The pre-loaded GetProviderSchema response is unnecessary and can likely be wholly deferred until after server startup and the actual RPC is called. The mux server already has a separate, small cache map for duplicate checking and raising implementation error diagnostics. There may be additional protocol changes necessary if the protocol implements "limited" GetProviderSchema RPC requests though.

For the PlanResourceChange cache usage:

• For protocol version 6, the caching is unnecessary and it can likely be removed
• For protocol version 5, it may be possible to avoid the caching based on the server capability response of an underlying provider as a "quick" fix for some situations (mainly muxed framework providers), but that doesn't help terraform-plugin-sdk based providers which do not enable the server capability
• It may be further possible to either enhance terraform-plugin-go or write internal logic which only peeks ahead into the PlanResourceChange JSON/msgpack data for nullity, rather than trying to fully unmarshal the data with the real schema.

Steps to Reproduce

1. terraform validate / terraform plan / terraform apply

References

• [Bug]: Memory consumption has increased since v4.67.0 terraform-provider-aws#31722

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.