trying to compile code after gradle.bat bloopInstall fails on Windows
princepurohit153 opened this issue · comments
I first ran gradle.bat bloopInstall
then ran bloop.bat compile on root project
but it getting error on compilation:
missing directory /.bloop/kafka/build/classes-empty-kafka.
code: https://github.com/apache/kafka
note this contains mix of scala and java
I believed that any .json file in .bloop directory can be treated as module to compile for bloop compile command. And hence passed kafka(root project) as argument.
Windows environment:
bloop about
bloop v1.5.4
Using Scala v2.12.15 and Zinc v1.6.0
Running on Java JDK v17.0.2 (C:\Users\Prince.jdks\openjdk-17.0.2)
-> Supports debugging user code, Java Debug Interface (JDI) is available.
Maintained by the Scala Center and the community.
Coursier version
cs.bat version
2.1.0-RC2
gradle.bat --version
Gradle 7.5.1
Build time: 2022-08-05 21:17:56 UTC
Revision: d1daa0cbf1a0103000b71484e1dbfe096e095918
Kotlin: 1.6.21
Groovy: 3.0.10
Ant: Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM: 17.0.2 (Oracle Corporation 17.0.2+8-86)
OS: Windows 10 10.0 amd64
Logs
PS C:\Users\Prince\Downloads\kafka> bloop.bat compile --verbose --cascade --project kafka
[D] Loading 86 projects from 'C:\Users\Prince\Downloads\kafka\.bloop'...
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\mirror-client.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\storage-api-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0110.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-25.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\jmh-benchmarks.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\streams.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0100-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0110-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-26.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\tools-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\generator-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\transforms-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-27-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\streams-scala.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\connect-api.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\runtime-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-20.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-28-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\file.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-23-test.json'
[D] Cache miss for scala instance org.scala-lang:scala-compiler:2.13.6.
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\com.google.protobuf\protobuf-java\3.7.0\dbb5e9230a91f2a6d011096c2b9c10a5a6e5f7f2\protobuf-java-3.7.0.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\net.java.dev.jna\jna\5.3.1\6eb9d07456c56b9c2560722e90382252f0f98405\jna-5.3.1.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\org.jline\jline\3.19.0\27edf6497c4fac20b63ca4cd8788581ca86cb83e\jline-3.19.0.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\org.scala-lang\scala-compiler\2.13.6\22973a59257545499e15c6e7f6332ce201c87a8\scala-compiler-2.13.6.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\org.scala-lang\scala-library\2.13.6\ed7a2f528c7389ea65746c22a01031613d98ab3d\scala-library-2.13.6.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\org.scala-lang\scala-reflect\2.13.6\bb523e56c63746a5752dece28fcd702c54fd3a11\scala-reflect-2.13.6.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\org.scala-sbt\compiler-bridge_2.13\1.3.5\3cddd365c477173648c2d9f312636d4552bce4de\compiler-bridge_2.13-1.3.5-sources.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\org.scala-sbt\compiler-interface\1.3.5\1f7379ff56b1795b523838cd06ff24272104c380\compiler-interface-1.3.5.jar
[D] => C:\Users\Prince\.gradle\caches\modules-2\files-2.1\org.scala-sbt\util-interface\1.3.0\edc7556e112da142bf7e9ead1440d024fd3314c4\util-interface-1.3.0.jar
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\log4j-appender-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-24.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\streams-examples-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0101-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0100.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\mirror.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\basic-auth-extension-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\trogdor.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\streams-examples.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\basic-auth-extension.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\kafka-examples.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\shell.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\transforms.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-30-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\streams-scala-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\test-utils-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\metadata-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-28.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-20-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\storage-api.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-21-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-25-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\connect.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\file-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\shell-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-11-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\raft-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\runtime.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\trogdor-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\mirror-client-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\generator.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\connect-api-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\server-common-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-31.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\core-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\test-utils.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\streams-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\raft.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0102-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\metadata.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\clients-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-24-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\server-common.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-22-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0101.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\json-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\storage-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-10.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\storage.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-26-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-30.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\clients.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-23.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-21.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-0102.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-31-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\tools.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\mirror-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\json.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-22.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\kafka.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\log4j-appender.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-10-test.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-11.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\core.json'
[D] Loading project from 'C:\Users\Prince\Downloads\kafka\.bloop\upgrade-system-tests-27.json'
[D] Missing analysis file for project 'jmh-benchmarks'
[D] Missing analysis file for project 'mirror-client'
[D] Missing analysis file for project 'upgrade-system-tests-0100-test'
[D] Missing analysis file for project 'upgrade-system-tests-25'
[D] Missing analysis file for project 'streams'
[D] Missing analysis file for project 'tools-test'
[D] Missing analysis file for project 'upgrade-system-tests-0110-test'
[D] Missing analysis file for project 'storage-api-test'
[D] Missing analysis file for project 'upgrade-system-tests-0110'
[D] Missing analysis file for project 'upgrade-system-tests-26'
[D] Missing analysis file for project 'upgrade-system-tests-27-test'
[D] Missing analysis file for project 'connect-api'
[D] Missing analysis file for project 'generator-test'
[D] Missing analysis file for project 'upgrade-system-tests-20'
[D] Missing analysis file for project 'file'
[D] Missing analysis file for project 'transforms-test'
[D] Missing analysis file for project 'upgrade-system-tests-23-test'
[D] Missing analysis file for project 'upgrade-system-tests-28-test'
[D] Missing analysis file for project 'runtime-test'
[D] Missing analysis file for project 'streams-scala'
[D] Missing analysis file for project 'upgrade-system-tests-24'
[D] Missing analysis file for project 'log4j-appender-test'
[D] Missing analysis file for project 'streams-examples-test'
[D] Missing analysis file for project 'upgrade-system-tests-0101-test'
[D] Missing analysis file for project 'basic-auth-extension-test'
[D] Missing analysis file for project 'mirror'
[D] Missing analysis file for project 'upgrade-system-tests-0100'
[D] Missing analysis file for project 'streams-examples'
[D] Missing analysis file for project 'basic-auth-extension'
[D] Missing analysis file for project 'trogdor'
[D] Missing analysis file for project 'shell'
[D] Missing analysis file for project 'kafka-examples'
[D] Missing analysis file for project 'upgrade-system-tests-30-test'
[D] Missing analysis file for project 'transforms'
[D] Missing analysis file for project 'test-utils-test'
[D] Missing analysis file for project 'upgrade-system-tests-28'
[D] Missing analysis file for project 'storage-api'
[D] Missing analysis file for project 'metadata-test'
[D] Missing analysis file for project 'upgrade-system-tests-20-test'
[D] Missing analysis file for project 'streams-scala-test'
[D] Missing analysis file for project 'connect'
[D] Missing analysis file for project 'upgrade-system-tests-25-test'
[D] Missing analysis file for project 'file-test'
[D] Missing analysis file for project 'upgrade-system-tests-11-test'
[D] Missing analysis file for project 'upgrade-system-tests-21-test'
[D] Missing analysis file for project 'raft-test'
[D] Missing analysis file for project 'mirror-client-test'
[D] Missing analysis file for project 'shell-test'
[D] Missing analysis file for project 'trogdor-test'
[D] Missing analysis file for project 'runtime'
[D] Missing analysis file for project 'upgrade-system-tests-31'
[D] Missing analysis file for project 'connect-api-test'
[D] Missing analysis file for project 'generator'
[D] Missing analysis file for project 'server-common-test'
[D] Missing analysis file for project 'test-utils'
[D] Missing analysis file for project 'upgrade-system-tests-0102-test'
[D] Missing analysis file for project 'raft'
[D] Missing analysis file for project 'metadata'
[D] Missing analysis file for project 'streams-test'
[D] Missing analysis file for project 'core-test'
[D] Missing analysis file for project 'upgrade-system-tests-24-test'
[D] Missing analysis file for project 'upgrade-system-tests-0101'
[D] Missing analysis file for project 'upgrade-system-tests-22-test'
[D] Missing analysis file for project 'server-common'
[D] Missing analysis file for project 'clients-test'
[D] Missing analysis file for project 'json-test'
[D] Missing analysis file for project 'upgrade-system-tests-10'
[D] Missing analysis file for project 'storage'
[D] Missing analysis file for project 'upgrade-system-tests-26-test'
[D] Missing analysis file for project 'storage-test'
[D] Missing analysis file for project 'clients'
[D] Missing analysis file for project 'upgrade-system-tests-30'
[D] Missing analysis file for project 'upgrade-system-tests-23'
[D] Missing analysis file for project 'upgrade-system-tests-21'
[D] Missing analysis file for project 'upgrade-system-tests-0102'
[D] Missing analysis file for project 'upgrade-system-tests-31-test'
[D] Missing analysis file for project 'upgrade-system-tests-22'
[D] Missing analysis file for project 'tools'
[D] Missing analysis file for project 'json'
[D] Missing analysis file for project 'mirror-test'
[D] Missing analysis file for project 'kafka'
[D] Missing analysis file for project 'upgrade-system-tests-11'
[D] Missing analysis file for project 'upgrade-system-tests-10-test'
[D] Missing analysis file for project 'log4j-appender'
[D] Missing analysis file for project 'upgrade-system-tests-27'
[D] Missing analysis file for project 'core'
[D] Computing sources and classpath hashes for kafka
[D] Scheduling compilation for kafka...
[D] Ignoring analysis for kafka, directory C:\Users\Prince\Downloads\kafka\.bloop\kafka\build\classes-empty-kafka is missing
[D] Creating a scala instance from Bloop's classloader...
[D] class scala.Predef$ detected in file:/C:/Users/Prince/AppData/Local/Coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.16/scala-library-2.12.16.jar
[D] interface scala.reflect.api.Trees detected in file:/C:/Users/Prince/AppData/Local/Coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.15/scala-reflect-2.12.15.jar
[D] class scala.tools.nsc.Main$ detected in file:/C:/Users/Prince/AppData/Local/Coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.12.15/scala-compiler-2.12.15.jar
[D] class scala.xml.Node detected in file:/C:/Users/Prince/AppData/Local/Coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.12/1.0.6/scala-xml_2.12-1.0.6.jar
[D] class jline.console.ConsoleReader detected in file:/C:/Users/Prince/AppData/Local/Coursier/cache/v1/https/repo1.maven.org/maven2/org/scala-sbt/jline/jline/2.14.7-sbt-a1b0ffbb8f64bb820f4f84a0c07a0c0964507493/jline-2.14.7-sbt-a1b0ffbb8f64bb820f4f84a0c07a0c0964507493.jar
[D] Created Bloop scala instance for 2.12.15
[D] Cache miss for scala instance org.scala-lang:scala-compiler:2.12.15.
[D] => C:\Users\Prince\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\modules\scala-xml_2.12\1.0.6\scala-xml_2.12-1.0.6.jar
[D] => C:\Users\Prince\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-compiler\2.12.15\scala-compiler-2.12.15.jar
[D] => C:\Users\Prince\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-library\2.12.16\scala-library-2.12.16.jar
[D] => C:\Users\Prince\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-lang\scala-reflect\2.12.15\scala-reflect-2.12.15.jar
[D] => C:\Users\Prince\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\org\scala-sbt\jline\jline\2.14.7-sbt-a1b0ffbb8f64bb820f4f84a0c07a0c0964507493\jline-2.14.7-sbt-a1b0ffbb8f64bb820f4f84a0c07a0c0964507493.jar
[D] Recording new last successful request for kafka associated with C:\Users\Prince\Downloads\kafka\.bloop\kafka\build\classes-empty-kafka
[D] Elapsed: 988.9646 ms
one other sub-module compiles fine but leads to different issues, mostly due to dependency of compilation on other modules.
do let me know if there is other way of compilation with bloop. Like with gradle we can just say build/compile it figures out dependencies of right order. Not sure with bloop it is manually done or some specific file mentions it?
bloop.bat compile --verbose --cascade --project clients
logs attached
client_compile_logs.txt
gradle-wrapper.properties mentioned version and I used different gradle version.
Switching to following gradle version seems to fix compilation part however now go to definition/file/ does not seem to work well.
C:\Users\Prince.gradle\wrapper\dists\gradle-7.4.2-all\
I am trying to use metals in vs code.
For now closing this issue, as it seems like gradle issue
The version of Gradle doesn't matter here. Kafka currently specifies 7.5.1 and that will work fine with Java 17
The issue here is down to source generation. Currently Bloop supports source generation but there is no way to automatically import that setup from the Gradle build as there's no real standard way to define source generation.
If you look here you can see how Kafka does it.
I imagine the reason Bloop is now working for you is that you have compiled the clients project using Gradle which has triggered the running of the source generators. Hence the missing files from your compilation (e.g. ApiVersionsResponseData) are now present in clients/src/generated/java/org/apache/kafka/common/message).
I can't really think of an easy way to auto-import this kind of source generator. You can import the project into Bloop/Metals as you have done and whenever you need to regenerate the source files (because you have changed the message json files) then you can re-run ./gradlew clients:processMessages from the command line. It's not ideal but currently it's the easiest way to work with source generators in Gradle.
Thanks a lot for decrypting this info.