Unrecognized field "urls" when attempting to use openjdk:11.0-nanoserver #2215

phxnsharp · 2020-01-07T16:04:00Z

Environment:

Jib 1.8.0
Maven 3.3.9 (as deployed with Netbeans 11.0)
Windows 10

Description of the issue:
Attempting to use a windows build of openjdk docker hub base image fails with the error:
Unrecognized field "urls" (class com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate), not marked as ignorable (3 known properties: "size", "digest", "mediaType"])

Expected behavior:
Project builds and runs correctly on Docker Desktop for Windows setup to run Windows containers.

Steps to reproduce:
Using spring-petclinic example because it makes it easy to reproduce. Not the project I am really attempting to work with.

git clone https://github.com/spring-projects/spring-petclinic.git
cd spring-petclinic
mvn package
Edit pom.xml and add jib plugin per information below
mvn -e jib:buildTar <-- This will work
Uncomment any of the Windows base images
mvn -e jib:buildTar <-- This will fail with the error below

jib-maven-plugin Configuration:

            <plugin>
              <groupId>com.google.cloud.tools</groupId>
              <artifactId>jib-maven-plugin</artifactId>
              <version>1.8.0</version>
              <configuration>
                <to>
                  <image>docker.io/nsharp/test_app</image>
                </to>
                <from>
                   <image>openjdk@sha256:221f36c282ea61f4121e1c6380a76bed8e3757e3aeb29c6fbf869bf832638454</image> <!-- latest linux/amd64 -->
                   <!-- <image>openjdk@sha256:399788bfd0f26e9bcbd45d21d0b7f0409392e51e7d9eae03d293fb89f214a145</image> --> <!-- latest windows/amd64 -->
                   <!-- <image>openjdk@sha256:0e1fe0b1652ba6047a7f0be5a6c66f63ecc5bbb0a36eab52b829bc74f02e58d6</image> --> <!-- 11.0-windowsservercore -->
                   <!-- <image>openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7</image> --> <!-- 11.0-nanoserver -->
                </from>
              </configuration>
            </plugin>

Log output:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.779 s
[INFO] Finished at: 2020-01-07T10:55:38-05:00
[INFO] Final Memory: 28M/97M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.8.0:buildTar (default-cli) on project spring-petclinic: Unrecognized field "urls" (class com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate), not marked as ignorable (3 known properties: "size", "digest", "mediaType"])
[ERROR] at [Source: (String)"{
[ERROR] "schemaVersion": 2,
[ERROR] "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
[ERROR] "config": {
[ERROR] "mediaType": "application/vnd.docker.container.image.v1+json",
[ERROR] "size": 3200,
[ERROR] "digest": "sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c"
[ERROR] },
[ERROR] "layers": [
[ERROR] {
[ERROR] "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
[ERROR] "size": 101106116,
[ERROR] "digest": "sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05"[truncated 2082 chars]; line: 14, column: 19] (through reference chain: com.google.cloud.tools.jib.image.json.V22ManifestTemplate["layers"]->java.util.ArrayList[0]->com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate["urls"])
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.8.0:buildTar (default-cli) on project spring-petclinic: Unrecognized field "urls" (class com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate), not marked as ignorable (3 known properties: "size", "digest", "mediaType"])
 at [Source: (String)"{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 3200,
      "digest": "sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
         "size": 101106116,
         "digest": "sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05"[truncated 2082 chars]; line: 14, column: 19] (through reference chain: com.google.cloud.tools.jib.image.json.V22ManifestTemplate["layers"]->java.util.ArrayList[0]->com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate["urls"])
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Unrecognized field "urls" (class com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate), not marked as ignorable (3 known properties: "size", "digest", "mediaType"])
 at [Source: (String)"{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 3200,
      "digest": "sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
         "size": 101106116,
         "digest": "sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05"[truncated 2082 chars]; line: 14, column: 19] (through reference chain: com.google.cloud.tools.jib.image.json.V22ManifestTemplate["layers"]->java.util.ArrayList[0]->com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate["urls"])
        at com.google.cloud.tools.jib.maven.BuildTarMojo.execute(BuildTarMojo.java:127)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
        ... 20 more
Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "urls" (class com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate), not marked as ignorable (3 known properties: "size", "digest", "mediaType"])
 at [Source: (String)"{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 3200,
      "digest": "sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
         "size": 101106116,
         "digest": "sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05"[truncated 2082 chars]; line: 14, column: 19] (through reference chain: com.google.cloud.tools.jib.image.json.V22ManifestTemplate["layers"]->java.util.ArrayList[0]->com.google.cloud.tools.jib.image.json.BuildableManifestTemplate$ContentDescriptorTemplate["urls"])
        at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
        at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:823)
        at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1153)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1589)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1567)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:286)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
        at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4014)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3005)
        at com.google.cloud.tools.jib.json.JsonTemplateMapper.readJson(JsonTemplateMapper.java:118)
        at com.google.cloud.tools.jib.registry.ManifestPuller.getManifestTemplateFromJson(ManifestPuller.java:149)
        at com.google.cloud.tools.jib.registry.ManifestPuller.handleResponse(ManifestPuller.java:99)
        at com.google.cloud.tools.jib.registry.ManifestPuller.handleResponse(ManifestPuller.java:45)
        at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:141)
        at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:115)
        at com.google.cloud.tools.jib.registry.RegistryClient.callRegistryEndpoint(RegistryClient.java:473)
        at com.google.cloud.tools.jib.registry.RegistryClient.pullManifest(RegistryClient.java:303)
        at com.google.cloud.tools.jib.registry.RegistryClient.pullManifest(RegistryClient.java:307)
        at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.pullBaseImage(PullBaseImageStep.java:205)
        at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call(PullBaseImageStep.java:165)
        at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call(PullBaseImageStep.java:59)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Additional Information:

The text was updated successfully, but these errors were encountered:

chanseokoh · 2020-01-07T17:01:26Z

Thanks for reporting this.

The OCI spec lists urls and annotations as optional properties. We should not fail if they are present. I'll fix this.

chanseokoh · 2020-01-07T17:28:24Z

It's an easy fix to ignore the optional properties.

However, I've noticed an issue in practice with the following images.

<image>openjdk@sha256:399788bfd0f26e9bcbd45d21d0b7f0409392e51e7d9eae03d293fb89f214a145</image> <!-- latest windows/amd64 -->
<image>openjdk@sha256:0e1fe0b1652ba6047a7f0be5a6c66f63ecc5bbb0a36eab52b829bc74f02e58d6</image> <!-- 11.0-windowsservercore -->
<image>openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7</image> <!-- 11.0-nanoserver -->

For example, the Docker Hub manifest for openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654 f3a4d7 lists the first layer as

$ docker manifest inspect openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654                                                                                                                           
f3a4d7                                                                                                                                                                                             
{                                                                                                                                                                                                  
        ...
        "layers": [                             
                {                               
                        "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
                        "size": 101106116,                                                       
                        "digest": "sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05b063446508f2ba15d510f2",
                        "urls": [               
                                "https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05b063446508f2ba15d510f2"
                        ]                       
                },

Docker Hub doesn't seem to actually host the blob 1951f408509ba9ddcf240ef5d838c72c5596f97a05b063446508f2ba15d510f2. So when Jib asks for it, the registry says it doesn't have it. I think the blob can be found in the MicroSoft registry given with the urls field.

The OCI spec says

urls array of strings

This OPTIONAL property specifies a list of URIs from which this object MAY be downloaded. Each entry MUST conform to RFC 3986. Entries SHOULD use the http and https schemes, as defined in RFC 7230.

The field is "optional" and the blob "MAY be downloaded", so ~~I think Docker Hub should just host the blob in itself~~ (UPDATE: I think these blobs cannot be uploaded to Docker Hub due to legal reasons.) But it would be convenient for Jib users if Jib tried the URLs in the urls array if the registry returns 404.

For now, I'll just apply the easy fix to ignore the urls field and leave this issue open for trying the given URLs.

chanseokoh · 2020-01-07T18:46:26Z

So when Jib asks for it, the registry says it doesn't have it.

Very weird. An hour ago, Docker Hub returned 404 for the layers with urls. Now, it's returning 401 for them. ~~I wonder if they are actively changing these Windows repositories or the general policy right now.~~

chanseokoh · 2020-01-07T20:25:14Z

#2217 is merged and will prevent Jib from crashing when there is a urls field. However, as mentioned, the openjdk Windows images above may not still work. The Docker Hub doesn't seem to return the blobs described in the manifest JSON. (And now it's a bit perplexing that they return 401 all of a sudden instead of 404.) ~~I hope they just host the blobs on Docker Hub.~~ (UPDATE: these missing layers are Windows layers that cannot be uploaded to Docker Hub probably for legal reasons.)

Jib could do further and try to pull from one of the urls, but this will be a bit involved to fix.

Does docker pull openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7 work on Windows (after cleaning the local Docker cache)?

phxnsharp · 2020-01-07T20:38:22Z

Does docker pull openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7 work on Windows (after cleaning the local Docker cache)?

PS Z:\> docker images -a
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

PS Z:\> docker pull openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7
sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7: Pulling from library/openjdk
1951f408509b: Pulling fs layer
163d55b77f49: Pulling fs layer
c888c8cbd618: Pulling fs layer
b25e433628b6: Pulling fs layer
5cdf60404a15: Pulling fs layer
9ea884077a3c: Pulling fs layer
370f7c37c71a: Pulling fs layer
50a6db676733: Pulling fs layer
5cdf60404a15: Waiting
9ea884077a3c: Waiting
b25e433628b6: Waiting
370f7c37c71a: Waiting
ccfbb191b269: Pulling fs layer
88f092a0c3bd: Pulling fs layer
50a6db676733: Waiting
ccfbb191b269: Waiting
88f092a0c3bd: Waiting
c888c8cbd618: Download complete
163d55b77f49: Verifying Checksum
163d55b77f49: Download complete
5cdf60404a15: Verifying Checksum
5cdf60404a15: Download complete
b25e433628b6: Verifying Checksum
b25e433628b6: Download complete
9ea884077a3c: Verifying Checksum
9ea884077a3c: Download complete
370f7c37c71a: Verifying Checksum
370f7c37c71a: Download complete
ccfbb191b269: Verifying Checksum
ccfbb191b269: Download complete
88f092a0c3bd: Verifying Checksum
88f092a0c3bd: Download complete
1951f408509b: Verifying Checksum
1951f408509b: Download complete
50a6db676733: Verifying Checksum
50a6db676733: Download complete
1951f408509b: Pull complete
163d55b77f49: Pull complete
c888c8cbd618: Pull complete
b25e433628b6: Pull complete
5cdf60404a15: Pull complete
9ea884077a3c: Pull complete
370f7c37c71a: Pull complete
50a6db676733: Pull complete
ccfbb191b269: Pull complete
88f092a0c3bd: Pull complete
Digest: sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7
Status: Downloaded newer image for openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7
docker.io/library/openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7

PS Z:\> docker images -a 
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
openjdk             <none>              8c9b0ced9024        3 weeks ago         563MB

chanseokoh · 2020-01-07T21:04:21Z

Thanks for the update. Unfortunately, this is quite involved to support in Jib.

Note to self:

If the Jib cache is empty, Docker Hub returns 404 (and fail) in the first run. Interestingly, subsequent runs return 401. Worth understanding why.

The media type for the blob hosted on a different server is "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip".

"layers": [
   {
      "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
      "size": 101106116,
      "digest": "sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05b063446508f2ba15d510f2",
      "urls": [
         "https://mcr.microsoft.com/v2/windows/nanoserver/blobs/sha256:1951f408509ba9ddcf240ef5d838c72c5596f97a05b063446508f2ba15d510f2"
      ]
   },
   {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 945,
      "digest": "sha256:163d55b77f49371136083ba8066ddbec4afad6e1f4fbba77fa4ffebc99a8098a"
   },

About the "foreign" layer media type, the Docker spec says

Layers of type application/vnd.docker.image.rootfs.foreign.diff.tar.gzip may be pulled from a remote location but they should never be pushed.

~~OTOH, the OCI spec doesn't seem to have a "foreign" layer media type.~~ Looks like the OCI spec equivalent to the Docker "foreign" layer type is the "non-distributable" type (can be gzip, tar, etc.).

chanseokoh · 2020-01-07T21:38:03Z

Project builds and runs correctly on Docker Desktop for Windows setup to run Windows containers.

@phxnsharp I'm curious. Do you really intend to run a Windows image on prod (e.g., deploy it to some Windows server-based cloud platform)?

phxnsharp · 2020-01-07T21:43:08Z

Do you really intend to run a Windows image on prod (e.g., deploy it to some Windows server-based cloud platform)?

Yes, but mostly in-house, not in the cloud.

chanseokoh · 2020-01-07T22:07:50Z

@phxnsharp um, actually, even if pulling foreign layers were supported, I'm afraid it's very possible that Windows images built by Jib could be broken anyway. Building Windows images has never been supported. We explicitly use AbsoluteUnixPath with forward slashes / everywhere. We just never attempted building Windows images.

phxnsharp · 2020-01-07T22:20:22Z

I'm afraid it's very possible that Windows images built by Jib could be broken anyway

Understood. We are currently just exploring options and are not in production yet. FWIW, A simple Dockerfile like this works for me using Docker's CLI:

FROM openjdk:11.0-nanoserver
ARG JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

It may very well be we will run into other pathing issues as we make this more complex. But for now, it is working.

Thanks so much for your support!

chanseokoh · 2020-01-07T23:57:57Z

There's a way to test building a Windows image with Jib. After you do docker pull openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7 (perhaps give it a different name for convenience), you can instruct Jib to use the image in the Docker daemon cache: <from><image>docker://openjdk@sha256:3f8305d1ec68251f3b2f22f6dfe00571c11e2d367e2de4c47524ebd654f3a4d7</image></from>. (However, using an image in Docker is very slow and inefficient.)

I'm personally curious to see if Jib-built Windows images will work. If so, it's an incentive for us to fix this issue. Even if it doesn't work, it'd be very useful for us to know if we should do a lot of work to support building Windows images in the future.

phxnsharp · 2020-01-08T13:23:02Z

There's a way to test building a Windows image with Jib.

Unfortunately it fails to build, whether I use the full sha256 string or the image ID. The error I get appears to be an unrelated issue:

[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.8.0:buildTar (default-cli) on project spring-petclinic: Unexpected character ('s' (code 115)): was expecting double-quote to start field name
[ERROR] at [Source: (String)"{size:562543460,imageId:sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c,diffIds:["sha256:2bfcd98d458eed97d6009c2df8db4290a0386f7cea68b82318b673b3d9e4a22b","sha256:ad252f9d36373a522cb79cd4664a09f740490d2f9a1ab254720909d279361a34","sha256:36a7d28bb29df0868143be6ab565824b9f71e61ec412d7b2f9cb85c362cbfb40","sha256:b67145a5a9d6091dcd57ca0e3e6f49309d9da4a5e559b453a6e4a09b34000991","sha256:9be96598d7dea222c68ffadd28956fa08296e57b075da0604c6d79c644ba7e2b","sha256:ea1a6e163d4ff36ce"[truncated 346 chars]; line: 1, column: 3]
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:1.8.0:buildTar (default-cli) on project spring-petclinic: Unexpected character ('s' (code 115)): was expecting double-quote to start field name
 at [Source: (String)"{size:562543460,imageId:sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c,diffIds:["sha256:2bfcd98d458eed97d6009c2df8db4290a0386f7cea68b82318b673b3d9e4a22b","sha256:ad252f9d36373a522cb79cd4664a09f740490d2f9a1ab254720909d279361a34","sha256:36a7d28bb29df0868143be6ab565824b9f71e61ec412d7b2f9cb85c362cbfb40","sha256:b67145a5a9d6091dcd57ca0e3e6f49309d9da4a5e559b453a6e4a09b34000991","sha256:9be96598d7dea222c68ffadd28956fa08296e57b075da0604c6d79c644ba7e2b","sha256:ea1a6e163d4ff36ce"[truncated 346 chars]; line: 1, column: 3]
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Unexpected character ('s' (code 115)): was expecting double-quote to start field name
 at [Source: (String)"{size:562543460,imageId:sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c,diffIds:["sha256:2bfcd98d458eed97d6009c2df8db4290a0386f7cea68b82318b673b3d9e4a22b","sha256:ad252f9d36373a522cb79cd4664a09f740490d2f9a1ab254720909d279361a34","sha256:36a7d28bb29df0868143be6ab565824b9f71e61ec412d7b2f9cb85c362cbfb40","sha256:b67145a5a9d6091dcd57ca0e3e6f49309d9da4a5e559b453a6e4a09b34000991","sha256:9be96598d7dea222c68ffadd28956fa08296e57b075da0604c6d79c644ba7e2b","sha256:ea1a6e163d4ff36ce"[truncated 346 chars]; line: 1, column: 3]
        at com.google.cloud.tools.jib.maven.BuildTarMojo.execute(BuildTarMojo.java:127)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
        ... 20 more
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('s' (code 115)): was expecting double-quote to start field name
 at [Source: (String)"{size:562543460,imageId:sha256:8c9b0ced90247b0e0d29ffb16d0e16480cec7e34d3c360340f076373d8f1c38c,diffIds:["sha256:2bfcd98d458eed97d6009c2df8db4290a0386f7cea68b82318b673b3d9e4a22b","sha256:ad252f9d36373a522cb79cd4664a09f740490d2f9a1ab254720909d279361a34","sha256:36a7d28bb29df0868143be6ab565824b9f71e61ec412d7b2f9cb85c362cbfb40","sha256:b67145a5a9d6091dcd57ca0e3e6f49309d9da4a5e559b453a6e4a09b34000991","sha256:9be96598d7dea222c68ffadd28956fa08296e57b075da0604c6d79c644ba7e2b","sha256:ea1a6e163d4ff36ce"[truncated 346 chars]; line: 1, column: 3]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:693)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:591)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddName(ReaderBasedJsonParser.java:1767)
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:692)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4014)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3005)
        at com.google.cloud.tools.jib.json.JsonTemplateMapper.readJson(JsonTemplateMapper.java:118)
        at com.google.cloud.tools.jib.docker.DockerClient.parseInspectResults(DockerClient.java:147)
        at com.google.cloud.tools.jib.docker.DockerClient.inspect(DockerClient.java:278)
        at com.google.cloud.tools.jib.builder.steps.LocalBaseImageSteps.lambda$retrieveDockerDaemonLayersStep$0(LocalBaseImageSteps.java:108)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

augi · 2020-02-07T15:35:17Z

Hello,

I'm here just to express that I would like also use jib with Windows Containers (we are using them in production for like 2 years).

I'm using Gradle and we are also getting the 404 error:

404 Not Found
{"errors":[{"code":"MANIFEST_UNKNOWN","message":"The named manifest is not known to the registry.","detail":{"manifest":"openjdk"}}]}

Or this error if trying to use my own base image:

404 Not Found
{"errors":[{"code":"BLOB_UNKNOWN","message":"blob unknown to registry","detail":{"blobSum":"sha256:3889bb8d808bbae6fa5a33e07093e65c31371bcf9e4c38c21be6b9af52

Maybe it's related to #1568 (handles foreign/non-distributable layers).

And I'm also getting the same Unexpected character ('s' (code 115)) error when trying to use an image from Docker daemon (instead of from registry).

Execution failed for task ':server:jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Unexpected character ('s' (code 115)): was expecting double-quote to start field name

Feel free to contact me if I could help with this.

chanseokoh · 2020-02-07T15:44:14Z

Hi @augi,

Yeah, #2215 is now a duplicate of #1568. Fixing #1568 will probably resolve your 404 issues. However, we've never considered building Windows containers (#2215 (comment)) and cannot guarantee anything. I noticed you already thumped-up to #2215 (comment). Have you tested using a windows base image from your local Docker engine and checked if your Windows container built by Jib works? I'm really curious.

augi · 2020-02-07T15:49:36Z

Yes, I've tested using the image from cache:

from {
        image = 'docker://openjdk:11.0.1-windowsservercore-ltsc2016'
    }

Unfortunately, I got the same error as @phxnsharp :

Execution failed for task ':server:jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Unexpected character ('s' (code 115)): was expecting double-quote to start field name
   at [Source: (String)"{size:11421955405,imageId:sha256:68204dc2fc12499d321ef43f00c439bbf2300b280d14737a92841d4b22ddd58c,diffIds:["sha256:f358be10862ccbc329638b9e10b3d497dd7cd28b0e8c7931b4a545c88d7f7cd6","sha256:ffce47ae4ffd0b88677730d9949223ae0f4de9c7b14fd7f23112a1724381dac8","sha256:9fb133e2afa1efc256864786d3fe44c896e171aebb52255e44bc5a6e38024ece","sha256:54f1b480fa3dca5845816d23b22bdb748d0fa22217ae9387121683dfe403da5a","sha256:e2e665092bb91d64c1fab2d3e6b5a80f3a52388badd5df17704f57405c81ad03","sha256:f47af1187470c38"[truncated 422 chars]; line: 1, column: 3]

augi · 2020-02-07T16:25:37Z

I've reported this error separately as #2270. It's not related to Windows Containers, it fails also for Linux Containers on Windows.

As I wrote, I'm willing to fix this particular issue.

chanseokoh · 2020-02-25T22:14:33Z

@phxnsharp we've released Jib 2.1.0, which fixes the bug that prevented loading local Docker images on Windows (#2270). At least you can now load local Docker images on Windows. However, as mentioned in #2270, building Windows containers is basically not supported.

chanseokoh · 2020-02-25T22:14:57Z

I'll close this as a dup of #1568.

chanseokoh self-assigned this Jan 7, 2020

chanseokoh added this to the v2.0.0 milestone Jan 7, 2020

chanseokoh mentioned this issue Jan 7, 2020

Do not fail when "urls" field is present in manifest JSON #2217

Merged

chanseokoh modified the milestones: v2.0.0, v2.1.0 Jan 7, 2020

chanseokoh removed their assignment Jan 7, 2020

chanseokoh mentioned this issue Jan 7, 2020

Possible 401 Unauthorized when base image is referred to by a digest and manifest is cached #2220

Closed

chanseokoh removed this from the v2.1.0 milestone Jan 28, 2020

augi mentioned this issue Feb 7, 2020

docker inspect output parsing fails on Windows #2270

Closed

chanseokoh mentioned this issue Feb 10, 2020

Support URL references to layers #1568

Closed

augi mentioned this issue Feb 12, 2020

Unrecognized field "LayerSources" error when processing an image in Windows Containers #2288

Open

chanseokoh closed this as completed Feb 25, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unrecognized field "urls" when attempting to use openjdk:11.0-nanoserver #2215

Unrecognized field "urls" when attempting to use openjdk:11.0-nanoserver #2215

phxnsharp commented Jan 7, 2020

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

phxnsharp commented Jan 7, 2020

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020

phxnsharp commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020

phxnsharp commented Jan 7, 2020

chanseokoh commented Jan 7, 2020 •

edited

Loading

phxnsharp commented Jan 8, 2020

augi commented Feb 7, 2020 •

edited

Loading

chanseokoh commented Feb 7, 2020 •

edited

Loading

augi commented Feb 7, 2020 •

edited

Loading

augi commented Feb 7, 2020

chanseokoh commented Feb 25, 2020

chanseokoh commented Feb 25, 2020

Unrecognized field "urls" when attempting to use openjdk:11.0-nanoserver #2215

Unrecognized field "urls" when attempting to use openjdk:11.0-nanoserver #2215

Comments

phxnsharp commented Jan 7, 2020

chanseokoh commented Jan 7, 2020 • edited Loading

chanseokoh commented Jan 7, 2020 • edited Loading

chanseokoh commented Jan 7, 2020 • edited Loading

chanseokoh commented Jan 7, 2020 • edited Loading

phxnsharp commented Jan 7, 2020

chanseokoh commented Jan 7, 2020 • edited Loading

chanseokoh commented Jan 7, 2020

phxnsharp commented Jan 7, 2020 • edited Loading

chanseokoh commented Jan 7, 2020

phxnsharp commented Jan 7, 2020

chanseokoh commented Jan 7, 2020 • edited Loading

phxnsharp commented Jan 8, 2020

augi commented Feb 7, 2020 • edited Loading

chanseokoh commented Feb 7, 2020 • edited Loading

augi commented Feb 7, 2020 • edited Loading

augi commented Feb 7, 2020

chanseokoh commented Feb 25, 2020

chanseokoh commented Feb 25, 2020

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

phxnsharp commented Jan 7, 2020 •

edited

Loading

chanseokoh commented Jan 7, 2020 •

edited

Loading

augi commented Feb 7, 2020 •

edited

Loading

chanseokoh commented Feb 7, 2020 •

edited

Loading

augi commented Feb 7, 2020 •

edited

Loading