v0.3.0-beta3

Changes since v0.3.0-beta2

• Fix BigtableIO #497
• Support BigQuery table description #491
• JIterable serialization improvement #507
• Bump Avro to 1.8.1 #510
• Package object clean up #508
• Add logging for BigtableUtil#updateNumberOfBigtableNodes #498

List of breaking changes:

• https://github.com/spotify/scio/wiki/Apache-Beam

Known issues:

• Scala 2.12 build is not available pending some compiler bugs (SI-10232, SI-10233)
• NPE in DirectRunner (including local scio-repl) in streaming mode BEAM-1835

v0.3.0-beta2

Changes since v0.3.0-beta1

• Features and bug fixes from v0.2.13
• Bump Beam to 0.6.0 (release notes)
• Support Array[Byte] as Sparkey key-value. (#483)
• Fix Sparkey upload performance issue
• Fix missing coder issue in SCollection#applyTransform

List of breaking changes:

• https://github.com/spotify/scio/wiki/Apache-Beam

Known issues:

• Scala 2.12 build is not available yet but almost there

v0.2.13

"Ex luna scientia"

Notable changes:

• Fix groupBy performance issue with large groups
• Improve Avro serialization performance
• Fix closure serialization in SCollection#partition
• Fix a Taps bug when directory doesn't exist

v0.3.0-beta1

Changes since v0.3.0-alpha2

• Features and bug fixes from v0.2.12
• Graduate type safe BigQuery API from experimental #336
• Consolidate temp location handling, --temp is the recommended flag now
• Add Sparkey side input support #402
• Add convenience implicits to scio-bigtable
• Support Coder[T] in PubsubIO
• Support Algebird aggregator in fan out cases
• Many streaming API tweaks
• Add ScioResult#waitUntilDone method
• Bugfix: lots of documentation updates

List of breaking changes:

• https://github.com/spotify/scio/wiki/Apache-Beam

Known issues:

• Scala 2.12 build is not available yet but almost there

v0.2.12

"In extremo"

Notable changes:

• Side input performance fixes #453
• Make Args serializable and add toString
• Fix #466 MultiJoin transform names
• Fix #449. Warn on scala.App
• Add setup/teardown to JobTest to be more extensible

v0.3.0-alpha2

Changes since v0.3.0-alpha1

• Bump Beam to 0.5.0 (release notes)
• Features and bug fixes from v0.2.11
• Rolled back from Java 8 java.time to joda-time
• Use --gcpTempLocation instead of --stagingLocation when running on Dataflow service

List of breaking changes:

• https://github.com/spotify/scio/wiki/Apache-Beam

Known issues:

• Scala 2.12 build is not available yet
• You might need "junit" % "junit" % "4.12" in your project dependencies

v0.2.11

"Saltatio mortis"

Notable changes:

• Feature: add ProtoIO to TestDataManager
• Feature: allow to switch on/off BigQuery cache (#427)
• Feature: allow to explicitly pick legacy/standard syntax inside BigQuery query (#427) (doc)
• Feature: bump Algebird to version 0.12.4 (0.12.4 release notes)
• Bugfix: allow to change job name via scio context (#440)
• Bugfix: fix compilation on encrypted linux partitions (#439)
• Bugfix: support compressed text files in TextTap
• Bugfix: improve kantan.csv integration (#426)

v0.3.0-alpha1

List of breaking changes:

• https://github.com/spotify/scio/wiki/Apache-Beam

Known issues:

• BigQuery writes may not work BEAM-1258
• Scala 2.12 build is not available yet

v0.2.10

"De Mysteriis Dom Sathanas"

Notable changes:

• Support for named transforms (#357), see this FAQ for more
• Expose ScioContext#isTest publicly
• Fix TFRecordFile handling
• Fix ProcessContext reuse for side input/output
• Fix #394. Add doc for hotKeyThreshold scaling (#404)
• Fix #400 Start repl from sbt (#403)
• Add integration test for DistCache (#398)

v0.2.9

"Hoc tempore atque nunc et semper"

Notable changes:

• Upgrade Dataflow SDK to 1.9.0 (release notes)
• Add TensorFlow TFRecord file IO #382
• Improve DistCache API and test #386
• Add typed BigQuery taps #381
• Skip nulls in BigQueryTable.toTableRow #388
• BigQuery DateTime format fix #377