Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JenkinsPipeline triggering Two Builds in Jenkins #282

Closed
disposab1e opened this issue Feb 27, 2019 · 9 comments
Closed

JenkinsPipeline triggering Two Builds in Jenkins #282

disposab1e opened this issue Feb 27, 2019 · 9 comments

Comments

@disposab1e
Copy link

Jenkins: 2.138.1
Jenkins Sync Plugin: 1.0.31
OpenShift: 3.11.82

After installation of an JenkinsPipeline Build Config via APB in OpenShift 2 builds in Jenkins are triggered in 95%.

This was the same with OpenShift 3.9.40 and it never disappeared including the 1.0.30 release of this plugin.

In OpenShift only one Build is to see, but in Jenkins 95% of all cases 2 builds are triggered. It was a little bit better in 3.9.40 (approx. 60% of all cases.

I'm happy to help to gather more information when I know what information is needed.

@gabemontero
Copy link

@disposab1e
From the namespace running the builds

  • oc get bc -o yaml
  • oc get builds -o yaml
    From the namespace running jenkins
  • oc logs <jenkins pod name>

@disposab1e
Copy link
Author

I send it from one build....

I cannot send everything from the logs... I try to collect what might be useful. I you want I can grep for special infos.

oc get bc schemas -o yaml

apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  creationTimestamp: 2019-02-27T15:00:17Z
  name: schemas
  namespace: jenkins-ci
  resourceVersion: "62112"
  selfLink: /apis/build.openshift.io/v1/namespaces/jenkins-ci/buildconfigs/schemas
  uid: 651ef08e-3aa0-11e9-9b7d-00505627371a
spec:
  nodeSelector:
    node-role.kubernetes.io/my-node: ci
  output: {}
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    git:
      ref: master
      uri: ssh://xxx@10.254.254.34/schemas.git
    sourceSecret:
      name: ssh-secret-bitbucket
    type: Git
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfilePath: build/Buildfile
    type: JenkinsPipeline
  triggers:
  - type: ConfigChange
status:
  lastVersion: 1

oc get build schemas-1 -o yaml

apiVersion: build.openshift.io/v1
kind: Build
metadata:
  annotations:
    openshift.io/build-config.name: schemas
    openshift.io/build.number: "1"
    openshift.io/jenkins-build-uri: http://jenkins-ci.cloudapps.example.com/job/-jenkins-ci-schemas/2/
    openshift.io/jenkins-console-log-url: http://jenkins-ci.cloudapps.example.com/job/-jenkins-ci-schemas/2/console
    openshift.io/jenkins-log-url: http://jenkins-ci.cloudapps.example.com/job/-jenkins-ci-schemas/2/consoleText
    openshift.io/jenkins-status-json: '{ cannot send these details }'
  creationTimestamp: 2019-02-27T15:00:17Z
  labels:
    buildconfig: schemas
    openshift.io/build-config.name: schemas
    openshift.io/build.start-policy: Serial
  name: schemas-1
  namespace: -jenkins-ci
  ownerReferences:
  - apiVersion: build.openshift.io/v1
    controller: true
    kind: BuildConfig
    name: schemas
    uid: 651ef08e-3aa0-11e9-9b7d-00505627371a
  resourceVersion: "63347"
  selfLink: /apis/build.openshift.io/v1/namespaces/-jenkins-ci/builds/schemas-1
  uid: 652365f1-3aa0-11e9-9b7d-00505627371a
spec:
  nodeSelector:
    node-role.kubernetes.io/: ci
  output: {}
  postCommit: {}
  resources: {}
  serviceAccount: builder
  source:
    git:
      ref: master
      uri: ssh://xxx@10.254.254.34/schemas.git
    sourceSecret:
      name: -ssh-secret-bitbucket
    type: Git
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfilePath: build/Buildfile
    type: JenkinsPipeline
  triggeredBy:
  - message: Build configuration change
status:
  completionTimestamp: 2019-02-27T15:08:29Z
  config:
    kind: BuildConfig
    name: schemas
    namespace: -jenkins-ci
  output: {}
  phase: Complete
  startTimestamp: 2019-02-27T15:06:02Z
  

oc logs jenkins-1-pnjn8

INFO: Waiting for Pod to be scheduled (0/100): slave-oraclejdk-7dx5s
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.BuildWatcher addEventToJenkinsJobRun
INFO: skipping watch event for build schemas-1 no job at this time
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.CredentialsUtils upsertCredential
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.PipelineJobListener buildConfigProjectForJob
INFO: Found BuildConfigProjectProperty for namespace: jenkins-ci name: schemas
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.PipelineJobListener upsertWorkflowJob
INFO: Upsert WorkflowJob jenkins-ci-schemas to BuildConfig: jenkins-ci/schemas in OpenShift
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.PipelineJobListener upsertBuildConfigForJob
INFO: Finding BuildConfig for namespace: jenkins-ci name: schemas
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.PipelineJobListener upsertBuildConfigForJob
INFO: Able to find BuildConfig for namespace: jenkins-ci name: schemas
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.BuildConfigWatcher$3 call
INFO: Created job jenkins-ci-schemas from BuildConfig NamespaceName{jenkins-ci:schemas} with revision: 62111
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.BuildWatcher flushBuildsWithNoBCList
INFO: triggering job run for previously skipped build schemas-1
Feb 27, 2019 3:00:17 PM io.fabric8.jenkins.openshiftsync.CredentialsUtils upsertCredential
Feb 27, 2019 3:00:18 PM io.fabric8.jenkins.openshiftsync.BuildWatcher flushBuildsWithNoBCList
INFO: triggering job run for previously skipped build schemas-1
Feb 27, 2019 3:00:18 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener onStarted
INFO: starting polling build job/jenkins-ci-schemas/1/
Feb 27, 2019 3:00:18 PM io.fabric8.jenkins.openshiftsync.CredentialsUtils upsertCredential
Feb 27, 2019 3:00:18 PM jenkins.model.RunIdMigrator migrate
INFO: Migrating build records in /var/lib/jenkins/jobs/jenkins-ci-schemas/builds
Feb 27, 2019 3:00:18 PM io.fabric8.jenkins.openshiftsync.BuildConfigWatcher$3 call
INFO: Updated job jenkins-ci-schemas from BuildConfig NamespaceName{jenkins-ci:schemas} with revision: 62112
Feb 27, 2019 3:00:18 PM io.fabric8.jenkins.openshiftsync.CredentialsUtils upsertCredential
Feb 27, 2019 3:00:20 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
INFO: Accepted JNLP4-connect connection #43 from /10.128.2.1:59654
Feb 27, 2019 3:00:20 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: pollRun
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at hudson.ExtensionList.get(ExtensionList.java:187)
at io.jenkins.blueocean.rest.factory.organization.OrganizationFactory.getInstance(OrganizationFactory.java:94)
at io.jenkins.blueocean.rest.factory.BlueRunFactory.getRun(BlueRunFactory.java:33)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollRun(BuildSyncRunListener.java:216)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollLoop(BuildSyncRunListener.java:202)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener$1.doRun(BuildSyncRunListener.java:159)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

TONS OF PREVIOUS MESSAGES

Feb 27, 2019 3:01:37 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Excess workload after pending Kubernetes agents: 1
Feb 27, 2019 3:01:37 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Template: Kubernetes Pod Template
Feb 27, 2019 3:01:37 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Feb 27, 2019 3:01:37 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning Kubernetes Pod Template from openshift with 1 executors. Remaining excess workload: -0.298
Feb 27, 2019 3:01:40 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: pollRun
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at hudson.ExtensionList.get(ExtensionList.java:187)
at io.jenkins.blueocean.rest.factory.organization.OrganizationFactory.getInstance(OrganizationFactory.java:94)
at io.jenkins.blueocean.rest.factory.BlueRunFactory.getRun(BlueRunFactory.java:33)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollRun(BuildSyncRunListener.java:216)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollLoop(BuildSyncRunListener.java:202)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener$1.doRun(BuildSyncRunListener.java:159)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

TONS OF PREVIOUS MESSAGES

Feb 27, 2019 3:01:47 PM hudson.slaves.NodeProvisioner$2 run
INFO: Kubernetes Pod Template provisioning successfully completed. We have now 3 computer(s)
Feb 27, 2019 3:01:47 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Feb 27, 2019 3:01:47 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Created Pod: slave-oraclejdk-rbd8t in namespace jenkins-ci
Feb 27, 2019 3:01:47 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for Pod to be scheduled (0/100): slave-oraclejdk-rbd8t
Feb 27, 2019 3:01:50 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
INFO: Accepted JNLP4-connect connection #44 from /10.128.2.1:41346
Feb 27, 2019 3:01:50 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: pollRun
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at hudson.ExtensionList.get(ExtensionList.java:187)
at io.jenkins.blueocean.rest.factory.organization.OrganizationFactory.getInstance(OrganizationFactory.java:94)
at io.jenkins.blueocean.rest.factory.BlueRunFactory.getRun(BlueRunFactory.java:33)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollRun(BuildSyncRunListener.java:216)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollLoop(BuildSyncRunListener.java:202)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener$1.doRun(BuildSyncRunListener.java:159)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Feb 27, 2019 3:03:48 PM io.fabric8.jenkins.openshiftsync.CredentialsUtils upsertCredential
INFO: Updated credential jenkins-ci-ssh-secret-bitbucket from Secret NamespaceName{jenkins-ci:ssh-secret-bitbucket} with revision: 54373
Feb 27, 2019 3:03:48 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Feb 27, 2019 3:03:48 PM io.fabric8.jenkins.openshiftsync.BuildWatcher reconcileRunsAndBuilds

TONS OF PREVIOUS MESSAGES

Feb 27, 2019 3:05:10 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: pollRun
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at hudson.ExtensionList.get(ExtensionList.java:187)
at io.jenkins.blueocean.rest.factory.organization.OrganizationFactory.getInstance(OrganizationFactory.java:94)
at io.jenkins.blueocean.rest.factory.BlueRunFactory.getRun(BlueRunFactory.java:33)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollRun(BuildSyncRunListener.java:216)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollLoop(BuildSyncRunListener.java:202)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener$1.doRun(BuildSyncRunListener.java:159)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Feb 27, 2019 3:05:10 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for agent slave-oraclejdk-7dx5s
Feb 27, 2019 3:05:10 PM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
WARNING: Computer.threadPoolForRemoting [#313] for slave-oraclejdk-7dx5s terminated
java.nio.channels.ClosedChannelException
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209)
at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314)
at hudson.remoting.Channel.close(Channel.java:1450)
at hudson.remoting.Channel.close(Channel.java:1403)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:821)
at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:105)
at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:737)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Feb 27, 2019 3:05:11 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Feb 27, 2019 3:05:11 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminated Kubernetes instance for agent jenkins-ci/slave-oraclejdk-7dx5s
Feb 27, 2019 3:05:11 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Disconnected computer slave-oraclejdk-7dx5s
Terminated Kubernetes instance for agent jenkins-ci/slave-oraclejdk-7dx5s
Feb 27, 2019 3:05:11 PM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
INFO: jenkins-ci-mod-config-resources #2 completed: SUCCESS
Feb 27, 2019 3:05:11 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: pollRun
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at hudson.ExtensionList.get(ExtensionList.java:187)
at io.jenkins.blueocean.rest.factory.organization.OrganizationFactory.getInstance(OrganizationFactory.java:94)
at io.jenkins.blueocean.rest.factory.BlueRunFactory.getRun(BlueRunFactory.java:33)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollRun(BuildSyncRunListener.java:216)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.onCompleted(BuildSyncRunListener.java:171)
at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:211)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:807)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$1100(WorkflowRun.java:143)
at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:1220)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1437)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:417)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

TONS OF PREVIOUS MESSAGES

Feb 27, 2019 3:06:02 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for agent slave-oraclejdk-rbd8t
Feb 27, 2019 3:06:02 PM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
WARNING: Computer.threadPoolForRemoting [#332] for slave-oraclejdk-rbd8t terminated
java.nio.channels.ClosedChannelException
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209)
at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314)
at hudson.remoting.Channel.close(Channel.java:1450)
at hudson.remoting.Channel.close(Channel.java:1403)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:821)
at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:105)
at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:737)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

TONS OF PREVIOUS MESSAGES

Feb 27, 2019 3:06:02 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Terminated Kubernetes instance for agent jenkins-ci/slave-oraclejdk-rbd8t
Feb 27, 2019 3:06:02 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminated Kubernetes instance for agent jenkins-ci/slave-oraclejdk-rbd8t
Feb 27, 2019 3:06:02 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Disconnected computer slave-oraclejdk-rbd8t
Feb 27, 2019 3:06:02 PM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
INFO: jenkins-ci-schemas #1 completed: SUCCESS
Feb 27, 2019 3:06:02 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: pollRun
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at hudson.ExtensionList.get(ExtensionList.java:187)
at io.jenkins.blueocean.rest.factory.organization.OrganizationFactory.getInstance(OrganizationFactory.java:94)
at io.jenkins.blueocean.rest.factory.BlueRunFactory.getRun(BlueRunFactory.java:33)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollRun(BuildSyncRunListener.java:216)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.onCompleted(BuildSyncRunListener.java:171)
at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:211)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:807)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$1100(WorkflowRun.java:143)
at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:1220)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1437)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:417)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

TONS OF PREVIOUS MESSAGES

Feb 27, 2019 3:06:02 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener onStarted
INFO: starting polling build job/jenkins-ci-schemas/2/
Feb 27, 2019 3:06:02 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener onCompleted
INFO: onCompleted job/jenkins-ci-schemas/1/
Feb 27, 2019 3:06:02 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: pollRun
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at hudson.ExtensionList.get(ExtensionList.java:187)
at io.jenkins.blueocean.rest.factory.organization.OrganizationFactory.getInstance(OrganizationFactory.java:94)
at io.jenkins.blueocean.rest.factory.BlueRunFactory.getRun(BlueRunFactory.java:33)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.pollRun(BuildSyncRunListener.java:216)
at io.fabric8.jenkins.openshiftsync.BuildSyncRunListener.onFinalized(BuildSyncRunListener.java:193)
at hudson.model.listeners.RunListener.fireFinalized(RunListener.java:257)
at hudson.model.Run.onEndBuilding(Run.java:2003)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:827)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$1100(WorkflowRun.java:143)
at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:1220)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1437)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:417)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Feb 27, 2019 3:06:05 PM io.fabric8.jenkins.openshiftsync.BuildSyncRunListener pollRun
WARNING: Cannot update status: Failure executing: PATCH at: https://kubernetes.default/apis/build.openshift.io/v1/namespaces/jenkins-ci/builds/schemas-1. Message: Build.build.openshift.io "schemas-1" is invalid: status.phase: Invalid value: "Running": phase cannot be updated from a terminal state. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=status.phase, message=Invalid value: "Running": phase cannot be updated from a terminal state, reason=FieldValueInvalid, additionalProperties={})], group=build.openshift.io, kind=Build, name=schemas-1, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Build.build.openshift.io "schemas-1" is invalid: status.phase: Invalid value: "Running": phase cannot be updated from a terminal state, metadata=ListMeta(resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
Feb 27, 2019 3:06:07 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Excess workload after pending Kubernetes agents: 1
Feb 27, 2019 3:06:07 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Template: Kubernetes Pod Template

@disposab1e
Copy link
Author

Perhaps something to do with: #284 ?

@gabemontero
Copy link

hey @disposab1e

I don't think this is directly related to whatever you are seeing with #284 (as I just noted there, I'm not seeing any problems with the folder disable/enable config field).

That said, the log message INFO: triggering job run for previously skipped build schemas-1 is telling in that it appears multiple times as you say.

That message shows up as a result of the sync plugin getting the build events before the build config events. We can't start builds until we know about the build configs.

That logic gets called from a few spots and I suspect we have a concurrency issue.

I think I can have a code change worked up pretty quickly, but reproducing the timing windows to cause the situation could be imprecise.

Since you are able to see it so readily, would you be able to try a test version of the plugin that I could attach to this issue?

thanks

gabemontero added a commit to gabemontero/jenkins-sync-plugin that referenced this issue Mar 12, 2019
possibly avoids duplicate build entries in the skip list by indexing off of build name vs. obj
synchronizes on the flush method since it possible can get called by multiple folks
@gabemontero
Copy link

Have PR #291 up

sync-1.zip
Here is a zip of my locally built hpi file in case you can test locally @disposab1e

gabemontero added a commit to jenkinsci/openshift-sync-plugin that referenced this issue Mar 13, 2019
possibly avoids duplicate build entries in the skip list by indexing off of build name vs. obj
synchronizes on the flush method since it possible can get called by multiple folks
@gabemontero
Copy link

I've initiated v1.0.34 of the plugin and it includes the fix noted above

@disposab1e
Copy link
Author

@gabemontero Sorry for late reaction! Was out for some days. I just updated my jenkins and give the new version a try. Will update this issue with my results soon.

@disposab1e
Copy link
Author

@gabemontero Looks good and it seems the concurrency issue is resolved so far. Great work! I will continue to have an eye on it and will inform you if something changes. Well done!

@gabemontero
Copy link

Great thanks for the feedback @disposab1e.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants