Data type and conversions are stricter for BQ sinks

[damccorm]

From #17404, it seems like the timestamp conversion has been made stricter, such that the pipeline only accepts timestamps in the form “2022-05-09T18:04:59Z”.

 

Here are some formats that previously worked, but seem to no longer be accepted:

“2022-05-09 18:04:59”

“2022-05-09T18:04:59”

“2022-05-09T18:04:59.739-07:00”

 

Also from the same pull request, it seems to no longer accept integers in fields marked as floats

 

Unexpected value :0, type: class java.lang.Integer. Table field name: a, type: FLOAT64

 

Are these the desired behaviors? It’s different from the behavior I’ve observed when using the Streaming Inserts method.

Imported from Jira BEAM-14528. Original Jira may contain additional context.
Reported by: kkgcp.

[damccorm]

Unable to assign user @yirutang. If able, self-assign, otherwise tag @damccorm so that he can assign you. Because of GitHub's spam prevention system, your activity is required to enable assignment in this repo.

[kennknowles]

The integer issue is also filed as #21693 which was fixed by #17779. This seems like an important quick fix to accept standard date formats. @reuvenlax @yirutang

[kennknowles]

Is this a regression for existing pipelines, or is it a failure of the upgrade path to the new BQ APIs?

[reuvenlax]

@yirutang I believe your fixes for this are already merged, correct?

[reuvenlax]

AFAICT this was fixed by #17778, which has already been merged.