Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Status of testing Providers that were prepared on June 07, 2022 #24289

Closed
potiuk opened this issue Jun 7, 2022 · 30 comments
Closed

Status of testing Providers that were prepared on June 07, 2022 #24289

potiuk opened this issue Jun 7, 2022 · 30 comments
Labels
kind:meta High-level information important to the community testing status Status of testing releases

Comments

@potiuk
Copy link
Member

potiuk commented Jun 7, 2022

Body

I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider alibaba: 2.0.0rc2

Provider amazon: 4.0.0rc2

Provider apache.beam: 4.0.0rc2

Provider apache.cassandra: 3.0.0rc2

Provider apache.druid: 3.0.0rc2

Provider apache.hdfs: 3.0.0rc2

Provider apache.hive: 3.0.0rc2

Provider apache.kylin: 3.0.0rc2

Provider apache.pinot: 3.0.0rc2

Provider apache.spark: 3.0.0rc2

Provider cncf.kubernetes: 4.1.0rc2

Provider databricks: 3.0.0rc2

Provider dbt.cloud: 2.0.0rc2

Provider docker: 3.0.0rc2

Provider elasticsearch: 4.0.0rc2

Provider exasol: 3.0.0rc2

Provider github: 2.0.0rc2

Provider google: 8.0.0rc2

Provider hashicorp: 3.0.0rc2

Provider jdbc: 3.0.0rc2

Provider jenkins: 3.0.0rc2

Provider microsoft.azure: 4.0.0rc2

Provider oracle: 3.0.0rc2

Provider papermill: 3.0.0rc2

Provider presto: 3.0.0rc2

Provider qubole: 3.0.0rc2

Provider sftp: 3.0.0rc2

Provider snowflake: 3.0.0rc2

Provider sqlite: 3.0.0rc2

Provider ssh: 3.0.0rc2

The guidelines on how to test providers can be found in

Verify providers by contributors

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.
@potiuk potiuk added the kind:meta High-level information important to the community label Jun 7, 2022
@vincbeck
Copy link
Contributor

vincbeck commented Jun 7, 2022

All my changes are related to doc and/or sample dag. No testing needed

@potiuk
Copy link
Member Author

potiuk commented Jun 7, 2022

Cool. Bulk-updated them as "tested" :D

@raphaelauv
Copy link
Contributor

#23981 work

@potiuk
Copy link
Member Author

potiuk commented Jun 7, 2022

I also tested and updated all "my" changes

@raphaelauv
Copy link
Contributor

#20578 I have no regression ( not using a new connection or using args of the KPO ) if still using core airlfow kubernetes settings 👍

@potiuk
Copy link
Member Author

potiuk commented Jun 7, 2022

One comment. I am releasing rc2 in a moment (@josh-fell found that our RELEASE notes (part of packages) contained Airflow 2.1+ so I am re-relesing all providers with that one change included (but feel free to continue the testing - the only difference will be those descriptions.

@ishiis
Copy link
Contributor

ishiis commented Jun 7, 2022

#23724 #24000 is tested.

@Dark-Knight11
Copy link
Contributor

#23849 contains only refactoring of code. No testing needed

@pauldalewilliams
Copy link
Contributor

I've tested #24090 and can confirm it's working as expected.

@pankajastro
Copy link
Member

Tested #23839 working as expected.

@dstandish
Copy link
Contributor

@potiuk do you know why k8s is a major?

@jonathansp
Copy link
Contributor

I've tested #23622 and it works as expected.

@gilangardya
Copy link

I've tested #24165 (google, BigQueryInsertJobOperator) with composer-2.0.15-airflow-2.2.5, it works as expected.
These are the tests I did:

Job type Statement Details 7.0.0 2022.5.18+composer 8.0.0rc1
COPY N/A Copy BigQuery Table
EXTRACT N/A To CSV
LOAD N/A From CSV
QUERY N/A SELECT query
QUERY N/A WRITE_TRUNCATE with partitioned table
QUERY DDL CREATE TABLE AS SELECT
QUERY DML INSERT
QUERY DML MERGE
QUERY DCL GRANT
QUERY Procedural DECLARE
QUERY Debugging ASSERT
QUERY Other EXPORT DATA
QUERY Multi-statement DECLARE + INSERT + MERGE

@nhojjohn
Copy link
Contributor

nhojjohn commented Jun 7, 2022

I checked #23916 and noticed an problem with it. Feel free to revert it or consider the following PR which fixes the issue #24301

@SasanAhmadi
Copy link
Contributor

I've tested #22802 and it is working as expected.

@potiuk
Copy link
Member Author

potiuk commented Jun 7, 2022

I checked #23916 and noticed an problem with it. Feel free to revert it or consider the following PR which fixes the issue #24301

Cool. I will look to it - I am going release rc2 so I will merge it before.

@potiuk
Copy link
Member Author

potiuk commented Jun 7, 2022

@potiuk do you know why k8s is a major?

As discussed on slack - mistake :). Will release minor as rc2.

@gmcrocetti
Copy link
Contributor

gmcrocetti commented Jun 7, 2022

@alexkruc
Copy link
Contributor

alexkruc commented Jun 8, 2022

Verified the installation and that everything exists properly for my fixes:

@potiuk
Copy link
Member Author

potiuk commented Jun 8, 2022

Thanks @gmcrocetti - as I am releasing rc2 soon, I merged your fix :)

@kazanzhy
Copy link
Contributor

kazanzhy commented Jun 8, 2022

#23767 working as expected

@eskarimov
Copy link
Contributor

Tested #19736 by executing DatabricksRunNowDeferrableOperator and DatabricksSubmitRunDeferrableOperator, both work as expected.

@Taragolis
Copy link
Contributor

Tested #23976 working fine

@o-nikolas
Copy link
Contributor

My changes are testing related and not part of provider code shipped to customers. Though I verified those tests are still passing 👍

@potiuk
Copy link
Member Author

potiuk commented Jun 8, 2022

I re-released the RC2 after few fixes (thanks to those who noticed them) and also corrected a number of documentation issues (that turned to be a bit more complex due to AIP-47 induced changes) - thanks @josh-fell and @deeptansh for raising the issues.

Announcement is about to be sent to the devlist. There is no real need to re-test the already tested changes (I left them checked) as the changes were minimal when it comes to code.

But the not-checked issues also need some ❤️ :).

Summary of changes in RC2:

  • @dstandish - the "cncf.kubernetes' is now 4.1.0 not 5.0.0 (my mistake)
  • @nhojjohn - your fix Fix langauge override in papermill operator #24301 should be included in RC2 - worth double checking
  • @deeptansh - the links to example dags in the docs should now be stable and correct
  • A number of "system tests" (google and dbt) were moved while automating doc links - they were both missing "cloud" subdir (wich is needed for consistency
  • Amazon and Google which are in progress with AIP-47 have two links (examples + legacy)
  • I workarounded (proper fix will come) a Google provider BigQuery extract job failure

Have fun with testing (we release on Monday if all is good).

@josh-fell
Copy link
Contributor

All of mine are 👍

@nhojjohn
Copy link
Contributor

nhojjohn commented Jun 9, 2022

tested papermill operator with #24301 in rc2 and works now. Thanks!

@potiuk
Copy link
Member Author

potiuk commented Jun 11, 2022

Looking forward for more testing till Monday :).

@vincentkoc
Copy link
Contributor

Tests looking good

@potiuk
Copy link
Member Author

potiuk commented Jun 13, 2022

Thanks everyone!. Release is accepted. Proceeding with release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:meta High-level information important to the community testing status Status of testing releases
Projects
None yet
Development

No branches or pull requests