Skip to content

Make Scala compilation 26% faster #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: lucid-master
Choose a base branch
from
Open

Make Scala compilation 26% faster #97

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
321e0a9
Cleaned up the imports in src/main/scala
Jun 24, 2025
36088c3
Sidestep Zinc and use the compiler interface directly
Jun 30, 2025
1b35bc3
Do test discovery in ZincRunner
Jun 30, 2025
4b54a68
Ripped out the remaining Zinc-related code
Jun 30, 2025
531e308
Updated the documentation
Jul 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .bazelrc_shared
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ common --modify_execution_info=CppArchive=+supports-path-mapping
common --modify_execution_info=CppCompile=+supports-path-mapping
common --modify_execution_info=CppModuleMap=+supports-path-mapping

common --announce_rc
common --color=yes
common:rules --disk_cache=.bazel_cache
common:tests --disk_cache=../.bazel_cache
Expand Down Expand Up @@ -51,3 +50,5 @@ common --incompatible_strict_action_env
#common --noincompatible_enable_deprecated_label_apis

test --test_output=all

try-import %workspace%/user.bazelrc
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ hash2
target/

!bazel-*.rc
/user.bazelrc

# Ignore files generated by IDEs.
/.classpath
Expand Down
1 change: 1 addition & 0 deletions MODULE.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ annex.install(
"com.thesamet.scalapb:scalapb-runtime_2.13:{}".format(scalapb_version),
"net.sourceforge.argparse4j:argparse4j:0.8.1",
"org.jacoco:org.jacoco.core:0.7.5.201505241946",
"org.playframework:play-json_3:3.0.4",
"org.scala-lang:scala-compiler:{}".format(scala_2_13_version),
"org.scala-lang:scala-library:{}".format(scala_2_13_version),
"org.scala-lang:scala-reflect:{}".format(scala_2_13_version),
Expand Down
12 changes: 6 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,12 @@ vastly improve build times. However, to see these benefits, a project must first
tiny packages and make use of fine-grained dependencies. This is not always a realistic short-term
goal for large, monorepo Scala projects.

`lucidsoftware/rules_scala` allows for the optional use of Zinc incremental compilation to provide a
stepping stone for these projects as they migrate to Bazel. Although we've verified it to be correct
and determinisitc, we recommend leaving this disabled, as fine-grained and isolated targets are
more in-line with the [Bazel philosophy](https://bazel.build/basics/hermeticity).
`lucidsoftware/rules_scala` used to allow for the optional use of Zinc incremental compilation to
provide a stepping stone for these projects as they migrate to Bazel. Although we still reuse code
from Zinc and the compiler bridge, this ruleset no longer supports incremental compilation.
Mitigating nondeterminism issues required introducing an enormous amount of complexity to the
compilation worker, and we eventually discovered that supporting incremental compilation added
substantial overhead to compilation times.

`lucidsoftware/rules_scala` is written with maintainability and accessibility in mind. It aims to
facilitate the transition to Bazel, and to satisfy use cases throughout the Scala ecosystem.
Expand Down Expand Up @@ -40,7 +42,6 @@ straightforward.
* Errors on indirect and unused dependencies
* Buildozer suggestions for dependency errors
* [Optional Worker strategy](docs/scala.md#workers)
* [Optional Zinc-based stateful incremental compilation](docs/stateful.md#stateful-compilation)
* [Scalafmt](docs/scalafmt.md#scalafmt) integration
* Protobuf support with ScalaPB
* [scala_proto_library](docs/stardoc/scala_proto.md#scala_proto_library)
Expand All @@ -49,7 +50,6 @@ straightforward.
* [Customizable rules](docs/newdocs/phases.md#customizing-the-core-rules)
* [Multiple Scala versions in one build](docs/newdocs/scala_versions.md#specifying-the-scala-version-to-use), including Scala 3 (Dotty).
* [Optimal handling of macros and ijars](docs/newdocs/macros.md#macros-and-ijars)
* [Pass flags to Zinc compiler](docs/newdocs/zinc_flags.md)
* Modern implementation using Bazel's most idiomatic APIs

## Usage
Expand Down
224 changes: 222 additions & 2 deletions annex_install.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"__AUTOGENERATED_FILE_DO_NOT_MODIFY_THIS_FILE_MANUALLY": "THERE_IS_NO_DATA_ONLY_ZUUL",
"__INPUT_ARTIFACTS_HASH": 416553459,
"__RESOLVED_ARTIFACTS_HASH": -333989184,
"__INPUT_ARTIFACTS_HASH": -1426185539,
"__RESOLVED_ARTIFACTS_HASH": 696573469,
"conflict_resolution": {
"org.scala-sbt:io_2.13:1.10.1": "org.scala-sbt:io_2.13:1.10.4"
},
Expand Down Expand Up @@ -237,6 +237,41 @@
},
"version": "0.10.1"
},
"com.fasterxml.jackson.core:jackson-annotations": {
"shasums": {
"jar": "ea1b3a9ec60a353f4d7e78559c7140abb9bcf71b31456e9d57c8289f7cf397f1",
"sources": "e6f7f659bb4abd9be1fe2f259f95a2340744aff373eb34c134d6e2210bdc300d"
},
"version": "2.14.3"
},
"com.fasterxml.jackson.core:jackson-core": {
"shasums": {
"jar": "7ee2debad3c002e97b28b84d5f1b2044a38e780abb673948238f5bc656e2fe44",
"sources": "3ee18051ff11ca9e0fd203bb163f2322c09573c76b6c1b13e2bd1031fb6f962a"
},
"version": "2.14.3"
},
"com.fasterxml.jackson.core:jackson-databind": {
"shasums": {
"jar": "ef0694046dc86e8a601e69efef24cda88a85773aaf012f5b9b1167d6f2e7de45",
"sources": "f9f6c8dd8b543c02eefc5844d068d90ff070ab913115be5df704b99b1988264b"
},
"version": "2.14.3"
},
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8": {
"shasums": {
"jar": "174dff10d8b8a762354401346317be3844f31f807a68eb72ce2b287ca273f4a4",
"sources": "762f23914234ea46addf3cf7c6625c50a03a045c62887b038f61e58d3dc72e15"
},
"version": "2.14.3"
},
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310": {
"shasums": {
"jar": "0b1b6d0267f939fe1c2d3167d33176a0b9f8c9b38444787d281e846beddf9ad6",
"sources": "43ca94b09ba2f33407a8367d98600fa3027da371cb6c62cc2a274f00917e6450"
},
"version": "2.14.3"
},
"com.github.pathikrit:better-files_2.12": {
"shasums": {
"jar": "79b49bc134f06a6a091a962ec10ce3f1810403bccec7d99bf9928b7eb02e85c4",
Expand Down Expand Up @@ -671,6 +706,20 @@
},
"version": "5.0.1"
},
"org.playframework:play-functional_3": {
"shasums": {
"jar": "a0ff6d625cb7066a3e559148751e6d5822b76847008e544fceddac29a63ccf3e",
"sources": "130056318d90cadcbb1c8bd65c188963a9d20a302d71b168460d2c7deb54a972"
},
"version": "3.0.4"
},
"org.playframework:play-json_3": {
"shasums": {
"jar": "cccc992d104f5694798ee5391d29758ba59e374d1edc82a43cd9f3bdcc57764a",
"sources": "b8eec5e983e866af339aea1aca4d2104fcfa47e90daadfff99409f8509b804fb"
},
"version": "3.0.4"
},
"org.reactivestreams:reactive-streams": {
"shasums": {
"jar": "ef867702a614b96eb6c64fb65a8f5e14bdfcabbc1ae056f78a1643f7b79ca0eb",
Expand Down Expand Up @@ -1279,6 +1328,19 @@
"com.eed3si9n:sjson-new-core_2.13",
"org.scala-lang:scala-library"
],
"com.fasterxml.jackson.core:jackson-databind": [
"com.fasterxml.jackson.core:jackson-annotations",
"com.fasterxml.jackson.core:jackson-core"
],
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8": [
"com.fasterxml.jackson.core:jackson-core",
"com.fasterxml.jackson.core:jackson-databind"
],
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310": [
"com.fasterxml.jackson.core:jackson-annotations",
"com.fasterxml.jackson.core:jackson-core",
"com.fasterxml.jackson.core:jackson-databind"
],
"com.github.pathikrit:better-files_2.12": [
"org.scala-lang:scala-library"
],
Expand Down Expand Up @@ -1448,6 +1510,18 @@
"org.jline:jline-native",
"org.jline:jline-terminal"
],
"org.playframework:play-functional_3": [
"org.scala-lang:scala3-library_3"
],
"org.playframework:play-json_3": [
"com.fasterxml.jackson.core:jackson-annotations",
"com.fasterxml.jackson.core:jackson-core",
"com.fasterxml.jackson.core:jackson-databind",
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8",
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310",
"org.playframework:play-functional_3",
"org.scala-lang:scala3-library_3"
],
"org.scala-lang.modules:scala-collection-compat_2.13": [
"org.scala-lang:scala-library"
],
Expand Down Expand Up @@ -1940,6 +2014,61 @@
"com.eed3si9n:sjson-new-scalajson_2.13": [
"sjsonnew.support.scalajson.unsafe"
],
"com.fasterxml.jackson.core:jackson-annotations": [
"com.fasterxml.jackson.annotation"
],
"com.fasterxml.jackson.core:jackson-core": [
"com.fasterxml.jackson.core",
"com.fasterxml.jackson.core.async",
"com.fasterxml.jackson.core.base",
"com.fasterxml.jackson.core.exc",
"com.fasterxml.jackson.core.filter",
"com.fasterxml.jackson.core.format",
"com.fasterxml.jackson.core.io",
"com.fasterxml.jackson.core.io.doubleparser",
"com.fasterxml.jackson.core.io.schubfach",
"com.fasterxml.jackson.core.json",
"com.fasterxml.jackson.core.json.async",
"com.fasterxml.jackson.core.sym",
"com.fasterxml.jackson.core.type",
"com.fasterxml.jackson.core.util"
],
"com.fasterxml.jackson.core:jackson-databind": [
"com.fasterxml.jackson.databind",
"com.fasterxml.jackson.databind.annotation",
"com.fasterxml.jackson.databind.cfg",
"com.fasterxml.jackson.databind.deser",
"com.fasterxml.jackson.databind.deser.impl",
"com.fasterxml.jackson.databind.deser.std",
"com.fasterxml.jackson.databind.exc",
"com.fasterxml.jackson.databind.ext",
"com.fasterxml.jackson.databind.introspect",
"com.fasterxml.jackson.databind.jdk14",
"com.fasterxml.jackson.databind.json",
"com.fasterxml.jackson.databind.jsonFormatVisitors",
"com.fasterxml.jackson.databind.jsonschema",
"com.fasterxml.jackson.databind.jsontype",
"com.fasterxml.jackson.databind.jsontype.impl",
"com.fasterxml.jackson.databind.module",
"com.fasterxml.jackson.databind.node",
"com.fasterxml.jackson.databind.ser",
"com.fasterxml.jackson.databind.ser.impl",
"com.fasterxml.jackson.databind.ser.std",
"com.fasterxml.jackson.databind.type",
"com.fasterxml.jackson.databind.util",
"com.fasterxml.jackson.databind.util.internal"
],
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8": [
"com.fasterxml.jackson.datatype.jdk8"
],
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310": [
"com.fasterxml.jackson.datatype.jsr310",
"com.fasterxml.jackson.datatype.jsr310.deser",
"com.fasterxml.jackson.datatype.jsr310.deser.key",
"com.fasterxml.jackson.datatype.jsr310.ser",
"com.fasterxml.jackson.datatype.jsr310.ser.key",
"com.fasterxml.jackson.datatype.jsr310.util"
],
"com.github.pathikrit:better-files_2.12": [
"better.files"
],
Expand Down Expand Up @@ -2522,6 +2651,15 @@
"org.objectweb.asm.util",
"org.objectweb.asm.xml"
],
"org.playframework:play-functional_3": [
"play.api.libs.functional",
"play.api.libs.functional.syntax"
],
"org.playframework:play-json_3": [
"play.api.libs.json",
"play.api.libs.json.jackson",
"play.api.libs.json.util"
],
"org.reactivestreams:reactive-streams": [
"org.reactivestreams"
],
Expand Down Expand Up @@ -3149,6 +3287,16 @@
"com.eed3si9n:sjson-new-scalajson_2.12:jar:sources",
"com.eed3si9n:sjson-new-scalajson_2.13",
"com.eed3si9n:sjson-new-scalajson_2.13:jar:sources",
"com.fasterxml.jackson.core:jackson-annotations",
"com.fasterxml.jackson.core:jackson-annotations:jar:sources",
"com.fasterxml.jackson.core:jackson-core",
"com.fasterxml.jackson.core:jackson-core:jar:sources",
"com.fasterxml.jackson.core:jackson-databind",
"com.fasterxml.jackson.core:jackson-databind:jar:sources",
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8",
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:sources",
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310",
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:sources",
"com.github.pathikrit:better-files_2.12",
"com.github.pathikrit:better-files_2.12:jar:sources",
"com.google.code.findbugs:jsr305",
Expand Down Expand Up @@ -3272,6 +3420,10 @@
"org.jline:jline:jar:sources",
"org.ow2.asm:asm-debug-all",
"org.ow2.asm:asm-debug-all:jar:sources",
"org.playframework:play-functional_3",
"org.playframework:play-functional_3:jar:sources",
"org.playframework:play-json_3",
"org.playframework:play-json_3:jar:sources",
"org.reactivestreams:reactive-streams",
"org.reactivestreams:reactive-streams:jar:sources",
"org.scala-lang.modules:scala-asm",
Expand Down Expand Up @@ -3462,6 +3614,16 @@
"com.eed3si9n:sjson-new-scalajson_2.12:jar:sources",
"com.eed3si9n:sjson-new-scalajson_2.13",
"com.eed3si9n:sjson-new-scalajson_2.13:jar:sources",
"com.fasterxml.jackson.core:jackson-annotations",
"com.fasterxml.jackson.core:jackson-annotations:jar:sources",
"com.fasterxml.jackson.core:jackson-core",
"com.fasterxml.jackson.core:jackson-core:jar:sources",
"com.fasterxml.jackson.core:jackson-databind",
"com.fasterxml.jackson.core:jackson-databind:jar:sources",
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8",
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:sources",
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310",
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:sources",
"com.github.pathikrit:better-files_2.12",
"com.github.pathikrit:better-files_2.12:jar:sources",
"com.google.code.findbugs:jsr305",
Expand Down Expand Up @@ -3585,6 +3747,10 @@
"org.jline:jline:jar:sources",
"org.ow2.asm:asm-debug-all",
"org.ow2.asm:asm-debug-all:jar:sources",
"org.playframework:play-functional_3",
"org.playframework:play-functional_3:jar:sources",
"org.playframework:play-json_3",
"org.playframework:play-json_3:jar:sources",
"org.reactivestreams:reactive-streams",
"org.reactivestreams:reactive-streams:jar:sources",
"org.scala-lang.modules:scala-asm",
Expand Down Expand Up @@ -3775,6 +3941,16 @@
"com.eed3si9n:sjson-new-scalajson_2.12:jar:sources",
"com.eed3si9n:sjson-new-scalajson_2.13",
"com.eed3si9n:sjson-new-scalajson_2.13:jar:sources",
"com.fasterxml.jackson.core:jackson-annotations",
"com.fasterxml.jackson.core:jackson-annotations:jar:sources",
"com.fasterxml.jackson.core:jackson-core",
"com.fasterxml.jackson.core:jackson-core:jar:sources",
"com.fasterxml.jackson.core:jackson-databind",
"com.fasterxml.jackson.core:jackson-databind:jar:sources",
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8",
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:sources",
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310",
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:sources",
"com.github.pathikrit:better-files_2.12",
"com.github.pathikrit:better-files_2.12:jar:sources",
"com.google.code.findbugs:jsr305",
Expand Down Expand Up @@ -3898,6 +4074,10 @@
"org.jline:jline:jar:sources",
"org.ow2.asm:asm-debug-all",
"org.ow2.asm:asm-debug-all:jar:sources",
"org.playframework:play-functional_3",
"org.playframework:play-functional_3:jar:sources",
"org.playframework:play-json_3",
"org.playframework:play-json_3:jar:sources",
"org.reactivestreams:reactive-streams",
"org.reactivestreams:reactive-streams:jar:sources",
"org.scala-lang.modules:scala-asm",
Expand Down Expand Up @@ -4033,6 +4213,46 @@
"ch.qos.logback.classic.servlet.LogbackServletContainerInitializer"
]
},
"com.fasterxml.jackson.core:jackson-core": {
"com.fasterxml.jackson.core.JsonFactory": [
"com.fasterxml.jackson.core.JsonFactory"
]
},
"com.fasterxml.jackson.core:jackson-core:jar:sources": {
"com.fasterxml.jackson.core.JsonFactory": [
"com.fasterxml.jackson.core.JsonFactory"
]
},
"com.fasterxml.jackson.core:jackson-databind": {
"com.fasterxml.jackson.core.ObjectCodec": [
"com.fasterxml.jackson.databind.ObjectMapper"
]
},
"com.fasterxml.jackson.core:jackson-databind:jar:sources": {
"com.fasterxml.jackson.core.ObjectCodec": [
"com.fasterxml.jackson.databind.ObjectMapper"
]
},
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8": {
"com.fasterxml.jackson.databind.Module": [
"com.fasterxml.jackson.datatype.jdk8.Jdk8Module"
]
},
"com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:sources": {
"com.fasterxml.jackson.databind.Module": [
"com.fasterxml.jackson.datatype.jdk8.Jdk8Module"
]
},
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310": {
"com.fasterxml.jackson.databind.Module": [
"com.fasterxml.jackson.datatype.jsr310.JavaTimeModule"
]
},
"com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:sources": {
"com.fasterxml.jackson.databind.Module": [
"com.fasterxml.jackson.datatype.jsr310.JavaTimeModule"
]
},
"org.apache.logging.log4j:log4j-api": {
"org.apache.logging.log4j.util.PropertySource": [
"org.apache.logging.log4j.util.EnvironmentPropertySource",
Expand Down
Loading