v0.13.5

Updates newly introduced scio-extra Voyager experimental API to v2

🏗️ Build improvements

• Update scalafmt-core to 3.7.15 (#5044) @scala-steward
• Update sbt, sbt-dependency-tree to 1.9.7 (#5043) @scala-steward

🌱 Dependency Updates

• Update voyager to 2.0.2 (#5030) @scala-steward
• Update jakarta.json-api to 2.1.3 (#5041) @scala-steward
• Update jedis to 4.4.6 (#5042) @scala-steward

Contributors to this release

@RustedBones and @scala-steward

v0.13.4

Includes Beam 2.51 support.

🚀 Enhancements

• Voyager support in Scio (#4996) @patrickwmcgee
• Support a transform() API for ScioContext (#5035) @clairemcginty
• Use readNextFilteredRowGroup instead of readNextRowGroup (#5025) @RustedBones
• Simplify ParquetBucketMetadata (#5024) @RustedBones
• Adds GrpcBatchDoFn (#4977) @senegalo

🐛 Bug Fixes

• Make macro print debug logs if bigquery.types.debug enabled (#5033) @RustedBones
• Pass avro reader schema for SMB operation (#5032) @RustedBones
• Remove TF model from the resource cache before closing (#5011) @jrglee
• Move log4j-over-slf4 to compile scope (#5018) @RustedBones
• Restrict implicit dynamic extensions for generic avro SCollection (#5021) @RustedBones
• Propagate ParquetOutputFormat options to ParquetWriter (#4980) @clairemcginty
• Route all logging to slf4j (#4981) @RustedBones

📗 Documentation

• fix some typos (#5039) @vuittont60
• Fix scaladoc links (#5022) @kellen
• Update dev docs (#5003) @RustedBones

🏗️ Build improvements

• Update sbt-mdoc to 2.4.0 (#5038) @scala-steward
• Set outputStrategy for run scope (#5019) @RustedBones
• Update sbt, sbt-dependency-tree to 1.9.6 (#5005) @scala-steward
• Update sbt-assembly to 2.1.3 (#5007) @scala-steward
• Update sbt-scalafix to 0.11.1 (#4997) @scala-steward
• Update sbt-jmh to 0.4.6 (#4992) @scala-steward
• Update sbt-scalafmt to 2.5.2 (#4989) @scala-steward
• Update sbt-bloop to 1.5.11 (#4982) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.41.0 (#4985) @scala-steward
• Update scalafmt-core to 3.7.14 (#4990) @scala-steward
• Update scalatest to 3.2.17 (#4991) @scala-steward
• Update sbt-scoverage to 2.0.9 (#4994) @scala-steward
• Bump actions/checkout from 3 to 4 (#4978) @dependabot

🌱 Dependency Updates

• Update beam to version 2.51 (#5023) @RustedBones
• Update elasticsearch-java to 8.10.4 (#5036) @scala-steward
• Update mysql-socket-factory-connector-j-8 to 1.14.1 (#5029) @scala-steward
• Update jedis to 4.4.5 (#5016) @scala-steward
• Update elasticsearch to v7.17.13 (#5008) @RustedBones
• Update magnolify-avro, magnolify-bigtable, ... to 0.6.3 (#5000) @scala-steward
• Update scala-compiler, scala-library, ... to 2.13.12 (#5001) @scala-steward
• Update cassandra-driver-core to 3.11.5 (#4984) @scala-steward
• Update scalactic to 3.2.17 (#4988) @scala-steward
• Update circe-core, circe-generic, ... to 0.14.6 (#4987) @scala-steward

Contributors to this release

@RustedBones, @clairemcginty, @dependabot, @dependabot[bot], @jrglee, @kellen, @patrickwmcgee, @scala-steward, @senegalo and @vuittont60

v0.13.3

Includes Beam 2.50.0 support.

🚀 Enhancements

• (fix #4970) Default to Parquet-SplittableDoFn if RunnerV2 is enabled (#4973) @clairemcginty

🐛 Bug Fixes

• Patch datum factory for specific data in IOs (#4975) @RustedBones
• Fix typeValidation in TransformOverride (#4967) @RustedBones
• Exclude logger implementation (#4969) @RustedBones

📗 Documentation

• Fix broken link (#4974) @saveriogzz
• Update jdbc doc to new API (#4954) @RustedBones

🌱 Dependency Updates

• Update beam to v2.50.0 (#4968) @RustedBones
• Update sbt, sbt-dependency-tree to 1.9.4 (#4965) @scala-steward
• Update sbt-avro to 3.4.3 (#4961) @scala-steward

Contributors to this release

@RustedBones, @clairemcginty, @saveriogzz and @scala-steward

v0.13.2

🚀 Enhancements

• Set typed table description when using scio BQ client (#4860) @RustedBones
• Added abstract JdbcDoFn to execute lookup (#4932) @shnapz

🐛 Bug Fixes

• Fix nonworking scalafix (#4931) @kellen
• Allow tuple coder extraction from wrapped RecordCoder (#4946) @RustedBones
• Don't re-use ReflectData in AvroCoder (#4951) @clairemcginty

🌱 Dependency Updates

• Update elasticsearch-java to 8.9.1 (#4956) @scala-steward
• Update cassandra-all to 3.11.16 (#4958) @scala-steward
• Update scalafmt-core to 3.7.12 (#4950) @scala-steward
• Update cassandra-driver-core to 3.11.4 (#4941) @scala-steward
• Update mysql-socket-factory-connector-j-8 to 1.13.1 (#4933) @scala-steward
• Update sbt, sbt-dependency-tree to 1.9.3 (#4935) @scala-steward

Full Changelog: v0.13.1...v0.13.2

v0.13.1

Includes Beam 2.49.0 support.

🚀 Enhancements

• Add scalafix rule for taps by @kellen in #4899
• scalafix by @RustedBones in #4904
• Package beam avro classes from extension lib in scio-repl by @RustedBones in #4929

📗 Documentation

• Enable MiMa, add migration guide, missing scalafix by @kellen in #4898
• Update doc with 0.13.1 release by @RustedBones in #4930

🌱 Dependency Updates

• Bump socco to 0.1.9 by @kellen in #4894
• Update elasticsearch-java to 8.8.2 by @scala-steward in #4900
• Update neo4j-java-driver to 4.4.12 by @scala-steward in #4901
• Update sbt, sbt-dependency-tree to 1.9.1 by @scala-steward in #4902
• Update scalafmt-core to 3.7.6 by @scala-steward in #4906
• Update sbt-bloop to 1.5.8 by @scala-steward in #4910
• Update scalafmt-core to 3.7.8 by @scala-steward in #4916
• Update sbt, sbt-dependency-tree to 1.9.2 by @scala-steward in #4915
• Update algebird-core, algebird-test to 0.13.10 by @scala-steward in #4914
• Bump JamesIves/github-pages-deploy-action from 4.4.2 to 4.4.3 by @dependabot in #4917
• Update mysql-socket-factory-connector-j-8 to 1.13.0 by @scala-steward in #4918
• Update sbt-paradox to 0.10.5 by @scala-steward in #4919
• Update sbt-tpolecat to 0.4.4 by @scala-steward in #4920
• Update scalafmt-core to 3.7.10 by @scala-steward in #4923
• Update sbt-bloop to 1.5.9 by @scala-steward in #4924
• Update sbt-mima-plugin to 1.1.3 by @scala-steward in #4925
• Update mysql-connector-java to 8.1.0 by @scala-steward in #4926
• Update beam-runners-core-construction-java, ... to 2.49.0 by @scala-steward in #4927

Full Changelog: v0.13.0...v0.13.1

v0.13.0

Includes Beam 2.48.0 support.

🔧 Breaking Changes

• Support is dropped for Elasticsearch 6.
• Elasticsearch 7 is migrated to new elastic search java client
• File Taps API changed to accept read params
• ⚠️ avro write default suffix is now ".avro" (previously was ""). Extension is not appended automatically anymore.

🚀 Enhancements

• Clean up Configuration initialization in scio-parquet (#4791) @clairemcginty
• Support EmptyMatchTreatment on TextIO (#4644) @turb
• Remove scio-elasticsearch v6 (#4802) @RustedBones
• Added optional header to saveAsDynamicTextFile (#4463) @ruchirsachdeva
• CoderTest changes for v0.13.x (#4806) @shnapz
• Remove deprecated avro specific coder macro (#4807) @RustedBones
• Relax avro record type bound (#4803) @RustedBones
• Mark Kryo coders nondeterminisitic (#4814) @kellen
• Provide module specific kryo coder registrar (#4753) @RustedBones
• Move TableRow API from core to GCP module (#4821) @RustedBones
• Add a better worded exception of unsupported SpecificRecord coder case (#4815) @shnapz
• Factorize scalactic and scalatest versions (#4836) @RustedBones
• Make ZSTD default compression for Parquet writes (#4726) @clairemcginty
• Added new JdbcIO read/write params to Scio (#4820) @shnapz
• Add prefix & suffix param for all IO APIs (#4809) @RustedBones
• Support EmptyMatchTreatment on JsonIO (#4818) @RustedBones
• Remove old skewed-join API (#4808) @RustedBones
• Support missing BQ write parameters (#4810) @RustedBones
• Use transform finder in JdbcIOTest (#4865) @RustedBones
• Cleanup context from fanout API (#4873) @RustedBones
• Move to avro extensions (#4883) @kellen
• Group grpc transforms (#4879) @RustedBones
• Safe coder unwrap (#4887) @RustedBones
• Add helper methods for Parquet ReadFiles transforms (#4801) @clairemcginty

🐛 Bug Fixes

• Fix site generation (#4891) @RustedBones
• Fix stray copypaste error (#4892) @kellen
• Fix spelling in LowPriorityCoderDerivation (#4844) @shnapz
• Add newline in .sbtopts (#4835) @RustedBones
• Correct shardNameTemplate in tests (#4817) @kellen
• Fix build badge (#4798) @RustedBones

📗 Documentation

• Fix GHA Build Documnetation step (#4833) @RustedBones
• Fix site settings (#4834) @RustedBones
• Add new sbt-site-paradox plugin (#4837) @RustedBones
• Documentation revamp (#4871) @kellen
• Update doc for 0.13 release (#4893) @RustedBones

🏗️ Build improvements

• Clean dependencies and add CI checks (#4732) @RustedBones
• Ignore checkerframework update (#4843) @RustedBones
• Move scalafix rules in proper package (#4855) @RustedBones
• Move scalafix rules in proper package (cont'd) (#4856) @RustedBones
• Sync apache commons lib with beam (#4863) @RustedBones
• Set io implementation as provided for smb and parquet (#4857) @RustedBones
• Leverage sbt-project-matrix to test all scalafix rules in isolation (#4874) @RustedBones
• Fix scio version for scalafix (#4875) @RustedBones
• Fix semanticdb (#4877) @RustedBones
• Drop java 8 in CI, set java 11 default, test java 17 (#4747) @RustedBones
• Fix dependency issues (#4876) @RustedBones

🌱 Dependency Updates

• Update scala-collection-compat to 2.10.0 (#4790) @scala-steward
• Update sbt-ci-release to 1.5.12 (#4796) @scala-steward
• Update elasticsearch-java to 8.7.1 (#4795) @scala-steward
• Update cassandra-all to 3.11.15 (#4799) @scala-steward
• Update elasticsearch-java to 8.8.0 (#4822) @scala-steward
• Update kryo-shaded to 4.0.3 (#4823) @scala-steward
• Update sbt-explicit-dependencies to 0.3.1 (#4824) @scala-steward
• Update sbt-ghpages to 0.8.0 (#4825) @scala-steward
• Update sbt-site to 1.5.0 (#4827) @scala-steward
• Update sbt, sbt-dependency-tree to 1.8.3 (#4830) @scala-steward
• Update scalatest to 3.2.16 (#4832) @scala-steward
• Update scalactic to 3.2.16 (#4831) @scala-steward
• Update commons-io to 2.12.0 (#4828) @scala-steward
• Update jedis to 4.4.1 (#4841) @scala-steward
• Update jakarta.json-api to 2.0.2 (#4839) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.40.16 (#4838) @scala-steward
• Update scalafmt-core to 3.7.4 (#4847) @scala-steward
• Update jakarta.json-api to 2.1.2 (#4845) @scala-steward
• Update sbt-scoverage to 2.0.8 (#4854) @scala-steward
• Update sbt to 1.9.0 (#4850) @scala-steward
• Update sbt-scalafix to 0.11.0 (#4853) @scala-steward
• Update flink to v1.16 (#4858) @RustedBones
• Update sbt-jmh to 0.4.5 (#4861) @scala-steward
• Update scala versions to 2.12.18 and 2.13.11 (#4864) @RustedBones
• Update jedis to 4.4.2 (#4870) @scala-steward
• Update sbt-header to 5.10.0 (#4867) @scala-steward
• Update elasticsearch-java to 8.8.1 (#4866) @scala-steward
• Update mysql-socket-factory to 1.11.2 (#4826) @scala-steward
• Update mysql-socket-factory-connector-j-8 to 1.12.0 (#4884) @scala-steward
• Update jedis to 4.4.3 (#4885) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.40.17 (#4889) @scala-steward
• Update scala-collection-compat to 2.11.0 (#4888) @scala-steward
• Update beam-runners-core-construction-java, ... to 2.48.0 (#4846) @scala-steward
• Bump JamesIves/github-pages-deploy-action from 4.4.1 to 4.4.2 (#4842) @dependabot

Contributors to this release

@RustedBones, @clairemcginty, @dependabot, @kellen, @ruchirsachdeva, @scala-steward, @turb, and @shnapz

Full Changelog: v0.12.8...v0.13.0

v0.11.15

This is a patch release for 0.11 adding opt-in logical type support for Parquet reads and writes.

🚀 Enhancements

• Add opt-in Logical type support to 0.11.x branch by @clairemcginty in #4792

Full Changelog: v0.11.14...v0.11.15

v0.12.8

🚀 Enhancements

• Parquet supports logical types on an opt-in basis (#4772, #4782) @clairemcginty
• Relax transform coder equality (#4780) @RustedBones
• Added more examples on Join types (#4769) @shnapz
• Add java sql Date and Time coder (#4781) @RustedBones
• Add deduplicate option to all AsyncLookupDoFn impl (#4775) @RustedBones

🐛 Bug Fixes

• Ensures guava callback to complete chained futures (#4778) @RustedBones
• Fix skewedJoin chill side with normal join instead of hashJoin (#4787) @RustedBones
• Set provided name on transforms with side outputs (#4779) @RustedBones

🌱 Dependency Updates

• Update mysql-connector-java to 8.0.33 (#4786) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.40.15 (#4785) @scala-steward
• Update mysql-socket-factory to 1.11.1 (#4776) @scala-steward
• Sync joda-time with beam (#4771) @RustedBones
• update elasticsearch7 to 7.17.9 (#4770) @RustedBones
• Update elasticsearch-java to 8.7.0 (#4760) @scala-steward
• Update scalafmt-core to 3.7.3 (#4758) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.40.14 (#4754) @scala-steward

Contributors to this release

@RustedBones, @clairemcginty, @scala-steward and @shnapz

v0.12.7

🚀 Enhancements

• Add support for DML statements to MockBigQuery (#4759) @fallonchen
• Allow F type param in tuple coders (#4767) @RustedBones

🐛 Bug Fixes

• Generate proper getCoderArguments on tuple coders (#4766) @RustedBones
• Fix avro coder regression on custom SpecificCoder (#4764) @RustedBones

🏗️ Build Improvements

• Remove add-to-project GHA due to SSO token issue (#4765) @RustedBones

🌱 Dependency Updates

• Update sbt-mima-plugin to 1.1.2 (#4761) @scala-steward

Contributors to this release

@RustedBones, @fallonchen and @scala-steward

v0.12.6

This release includes an upgrade to Beam 2.46.

🚀 Enhancements

• Add fanout for CMS hot key computation in skewed joins (#4752) @RustedBones
• Delay deprecation of old skewed API (#4755) @RustedBones
• Expose cache in scio-grpc (#4734) @RustedBones
• Add magnolify imports to docs for methods using Funnel (#4739) @kellen
• Distribute CMS hot key set as side input (#4724) @RustedBones
• Support Parquet in BQ ExtractOps (#4728) @clairemcginty
• Neo4j parametrized from existing SCollection (#4719) @sumitsu
• Make SeqExampleSCollectionOps API consistent (#4731) @clairemcginty
• Add skewed job methods with topN and top percent (#4718) @RustedBones
• Modernize scalafix rules (#4687) @RustedBones

🐛 Bug Fixes

• Don't forget to fan out (#4740) @kellen
• Don't re-use Configuration objects in Parquet SMB (#4727) @clairemcginty
• apply extendedErrorInfo input parameter (#4723) @farzad-sedghi

🏗️ Build Improvements

• Fork all tests. Default to java 11 (#4748) @RustedBones
• Factorize java options and propagate GCP settings (#4746) @RustedBones
• Add java 17 configuration for integration tests (#4745) @RustedBones
• Remove test matrix for integration test (#4737) @RustedBones
• Setup MiMa filter for planed breaking changes (#4733) @RustedBones

🌱 Dependency Updates

• Update beam to 2.46.0 (#4744) @RustedBones
• Update mysql-socket-factory to 1.11.0 (#4730) @scala-steward
• Update circe-core, circe-generic, ... to 0.14.5 (#4735) @scala-steward
• Bump actions/add-to-project from 0.4.0 to 0.4.1 (#4729) @dependabot
• Update scalafmt-core to 3.7.2 (#4720) @scala-steward
• Update elasticsearch-java to 8.6.2 (#4715) @scala-steward

Contributors to this release

@RustedBones, @clairemcginty, @dependabot, @dependabot[bot], @farzad-sedghi, @kellen, @scala-steward and @sumitsu