[BEAM-14505] Add Dataflow streaming pipeline update support to the Go SDK

[jrmccluskey]

Adds the ability to update streaming pipelines running on Dataflow through the --update flag. Includes adding the standard way to map old transforms to new ones, as described in https://cloud.google.com/dataflow/docs/guides/updating-a-pipeline

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[asf-ci]

Can one of the admins verify this patch?

[asf-ci]

Can one of the admins verify this patch?

[codecov]

Codecov Report

Merging #17747 (61316fa) into master (2039b39) will decrease coverage by 0.00%.
The diff coverage is 47.61%.

@@            Coverage Diff             @@
##           master   #17747      +/-   ##
==========================================
- Coverage   73.99%   73.99%   -0.01%     
==========================================
  Files         695      695              
  Lines       91826    91843      +17     
==========================================
+ Hits        67948    67958      +10     
- Misses      22632    22639       +7     
  Partials     1246     1246              

Flag	Coverage Δ
go	50.44% <47.61%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...o/pkg/beam/runners/dataflow/dataflowlib/execute.go	0.00% <0.00%> (ø)
...ks/go/pkg/beam/runners/dataflow/dataflowlib/job.go	21.55% <0.00%> (-0.55%)	⬇️
sdks/go/pkg/beam/runners/dataflow/dataflow.go	58.62% <100.00%> (+2.52%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2039b39...61316fa. Read the comment docs.

[github-actions]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @riteshghorse for label go.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[jrmccluskey]

R: @lostluck

[github-actions]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

[lostluck]

Jack, please update the PR description, it's incorrect: It's no longer a WIP and it does allow transform overrides (unless I missed my read).

[lostluck]

LGTM

I overall don't love the way we have these, as package variables, but that's unrelated to this PR. We could clean up the tests if we had a config struct, and had the flags all using the *Var versions pointing to that struct. The struct is the plumbed through instead of direct access to the flags. (I guess we already have such a struct for JobOptions too)

Then we aren't accessing the flags everywhere, and testing different configurations becomes setting fields in the struct, which doesn't have ordering risk, presently mitigated by re-setting all the flags.

But that's for a different PR.

[lostluck]

Jack, please update the PR description, it's incorrect: It's no longer a WIP and it does allow transform overrides (unless I missed my read).

Nvm. I did it, since there was nothing else for you to change blocking a merge.