Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Elastic Agent] Add docs for custom ingest pipeline and template #23247

Closed
wants to merge 1 commit into from
Closed

[Elastic Agent] Add docs for custom ingest pipeline and template #23247

wants to merge 1 commit into from

Conversation

ruflin
Copy link
Member

@ruflin ruflin commented Dec 22, 2020

TODO

  • Figure out where to link this from
  • Complete todos in the text
  • Should this also be referenced from Fleet?

@ruflin ruflin self-assigned this Dec 22, 2020
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Dec 22, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Dec 22, 2020 •
edited by jenkins-beats-ci bot
Loading

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Started by user Victor Martinez

  • Start Time: 2021-01-08T09:24:32.422+0000

  • Duration: 28 min 51 sec

Steps errors 2

Expand to view the steps failures

make check
  • Took 17 min 2 sec . View more details on here
  • Description: make check
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 2'

Log output

Expand to view the last 100 lines of log output 

[2021-01-08T09:40:07.428Z] Generated fields.yml for packetbeat to /var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/packetbeat/fields.yml
[2021-01-08T09:40:07.428Z] >> Building packetbeat.yml for linux/amd64
[2021-01-08T09:40:07.428Z] >> Building packetbeat.reference.yml for linux/amd64
[2021-01-08T09:40:07.428Z] >> Building packetbeat.docker.yml for linux/amd64
[2021-01-08T09:40:09.969Z] exec: go list -m
[2021-01-08T09:40:10.903Z] >> check: Checking source code for common problems
[2021-01-08T09:40:14.201Z] make[1]: Leaving directory '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/packetbeat'
[2021-01-08T09:40:14.201Z] make[1]: Entering directory '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/winlogbeat'
[2021-01-08T09:40:22.537Z] >> fmt - go-licenser: Checking for missing headers
[2021-01-08T09:40:27.896Z] >> fmt - go-licenser: Adding missing headers
[2021-01-08T09:40:27.896Z] >> fmt - goimports: Formatting Go code
[2021-01-08T09:40:27.896Z] >> fmt - autopep8: Formatting Python code
[2021-01-08T09:40:29.802Z] >> Building winlogbeat.yml for windows/amd64
[2021-01-08T09:40:29.802Z] >> update:moduleDocs: Collecting module documentation for powershell, security, sysmon.
[2021-01-08T09:40:29.802Z] >> Building winlogbeat.reference.yml for windows/amd64
[2021-01-08T09:40:30.060Z] Generated fields.yml for winlogbeat to /var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/winlogbeat/build/fields/fields.common.yml
[2021-01-08T09:40:30.060Z] Generated fields.yml for winlogbeat to /var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/winlogbeat/build/fields/fields.yml
[2021-01-08T09:40:30.060Z] Generated fields.yml for winlogbeat to /var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/winlogbeat/build/fields/fields.all.yml
[2021-01-08T09:40:30.994Z] >> check: Checking source code for common problems
[2021-01-08T09:40:32.371Z] make[1]: Leaving directory '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/winlogbeat'
[2021-01-08T09:40:32.371Z] make[1]: Entering directory '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/x-pack/functionbeat'
[2021-01-08T09:40:32.371Z] mage check
[2021-01-08T09:40:38.942Z] >> fmt - go-licenser: Adding missing headers
[2021-01-08T09:40:38.942Z] >> fmt - autopep8: Formatting Python code
[2021-01-08T09:40:38.942Z] >> fmt - goimports: Formatting Go code
[2021-01-08T09:40:38.942Z] >> check: Checking source code for common problems
[2021-01-08T09:40:47.107Z] make[1]: Leaving directory '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/x-pack/functionbeat'
[2021-01-08T09:40:47.107Z] make[1]: Entering directory '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/x-pack/elastic-agent'
[2021-01-08T09:40:47.107Z] mage check
[2021-01-08T09:40:53.728Z] >> fmt - go-licenser: Adding missing headers
[2021-01-08T09:40:53.728Z] >> fmt - goimports: Formatting Go code
[2021-01-08T09:40:53.728Z] >> Building elastic-agent.yml for linux/amd64
[2021-01-08T09:40:53.728Z] >> Building elastic-agent.reference.yml for linux/amd64
[2021-01-08T09:40:53.728Z] >> Building elastic-agent.docker.yml for linux/amd64
[2021-01-08T09:40:53.728Z] >> Buildspec from spec/*.yml to pkg/agent/program/supported.go
[2021-01-08T09:40:54.294Z] >> BuildPGP from GPG-KEY-elasticsearch to pkg/release/pgp.go
[2021-01-08T09:40:54.552Z] >> BuildFleetCfg _meta/elastic-agent.fleet.yml to pkg/agent/application/configuration_embed.go
[2021-01-08T09:52:16.492Z] # cd .; git clone -- https://go.googlesource.com/tools /var/lib/jenkins/workspace/Beats_beats_PR-23247/src/golang.org/x/tools
[2021-01-08T09:52:16.492Z] Cloning into '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/golang.org/x/tools'...
[2021-01-08T09:52:16.492Z] fatal: The remote end hung up unexpectedly
[2021-01-08T09:52:16.492Z] fatal: early EOF
[2021-01-08T09:52:16.492Z] fatal: index-pack failed
[2021-01-08T09:52:16.492Z] package golang.org/x/tools/go/ast/astutil: exit status 128
[2021-01-08T09:52:16.492Z] package golang.org/x/tools/go/gcexportdata: cannot find package "golang.org/x/tools/go/gcexportdata" in any of:
[2021-01-08T09:52:16.492Z] 	/var/lib/jenkins/workspace/Beats_beats_PR-23247/.gvm/versions/go1.14.12.linux.amd64/src/golang.org/x/tools/go/gcexportdata (from $GOROOT)
[2021-01-08T09:52:16.492Z] 	/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/golang.org/x/tools/go/gcexportdata (from $GOPATH)
[2021-01-08T09:52:16.492Z] Error: running "go get golang.org/x/lint/golint" failed with exit code 1
[2021-01-08T09:52:16.492Z] ../../dev-tools/make/mage.mk:21: recipe for target 'check' failed
[2021-01-08T09:52:16.492Z] make[1]: *** [check] Error 1
[2021-01-08T09:52:16.492Z] make[1]: Leaving directory '/var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats/x-pack/elastic-agent'
[2021-01-08T09:52:16.492Z] Makefile:98: recipe for target 'check' failed
[2021-01-08T09:52:16.492Z] make: *** [check] Error 1
[2021-01-08T09:52:16.805Z] Client: Docker Engine - Community
[2021-01-08T09:52:16.805Z]  Version:           20.10.2
[2021-01-08T09:52:16.805Z]  API version:       1.41
[2021-01-08T09:52:16.805Z]  Go version:        go1.13.15
[2021-01-08T09:52:16.805Z]  Git commit:        2291f61
[2021-01-08T09:52:16.805Z]  Built:             Mon Dec 28 16:17:32 2020
[2021-01-08T09:52:16.805Z]  OS/Arch:           linux/amd64
[2021-01-08T09:52:16.805Z]  Context:           default
[2021-01-08T09:52:16.805Z]  Experimental:      true
[2021-01-08T09:52:16.805Z] 
[2021-01-08T09:52:16.805Z] Server: Docker Engine - Community
[2021-01-08T09:52:16.805Z]  Engine:
[2021-01-08T09:52:16.805Z]   Version:          20.10.2
[2021-01-08T09:52:16.805Z]   API version:      1.41 (minimum version 1.12)
[2021-01-08T09:52:16.805Z]   Go version:       go1.13.15
[2021-01-08T09:52:16.805Z]   Git commit:       8891c58
[2021-01-08T09:52:16.805Z]   Built:            Mon Dec 28 16:15:09 2020
[2021-01-08T09:52:16.805Z]   OS/Arch:          linux/amd64
[2021-01-08T09:52:16.805Z]   Experimental:     false
[2021-01-08T09:52:16.805Z]  containerd:
[2021-01-08T09:52:16.805Z]   Version:          1.4.3
[2021-01-08T09:52:16.805Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-01-08T09:52:16.805Z]  runc:
[2021-01-08T09:52:16.805Z]   Version:          1.0.0-rc92
[2021-01-08T09:52:16.805Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-01-08T09:52:16.805Z]  docker-init:
[2021-01-08T09:52:16.805Z]   Version:          0.19.0
[2021-01-08T09:52:16.805Z]   GitCommit:        de40ad0
[2021-01-08T09:52:16.805Z] Unable to find image 'alpine:3.4' locally
[2021-01-08T09:52:17.737Z] 3.4: Pulling from library/alpine
[2021-01-08T09:52:17.737Z] c1e54eec4b57: Pulling fs layer
[2021-01-08T09:52:17.995Z] c1e54eec4b57: Verifying Checksum
[2021-01-08T09:52:17.995Z] c1e54eec4b57: Download complete
[2021-01-08T09:52:17.995Z] c1e54eec4b57: Pull complete
[2021-01-08T09:52:17.995Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-01-08T09:52:17.995Z] Status: Downloaded newer image for alpine:3.4
[2021-01-08T09:52:22.377Z] Stage "Build&Test" skipped due to earlier failure(s)
[2021-01-08T09:52:22.403Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-01-08T09:52:22.442Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-23247/src/github.com/elastic/beats
[2021-01-08T09:52:22.697Z] Running on beats-ci-immutable-ubuntu-1804-1610098974059024868 in /var/lib/jenkins/workspace/Beats_beats_PR-23247
[2021-01-08T09:52:22.762Z] [INFO] getVaultSecret: Getting secrets
[2021-01-08T09:52:22.822Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-01-08T09:52:23.972Z] + chmod 755 generate-build-data.sh
[2021-01-08T09:52:23.972Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23247/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23247/runs/2 FAILURE 1670808
[2021-01-08T09:52:23.972Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23247/runs/2/steps/?limit=10000 -o steps-info.json
[2021-01-08T09:52:24.231Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-23247/runs/2/tests/?status=FAILED -o tests-errors.json
[2021-01-08T09:52:24.231Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-01-08T09:52:25.166Z] Retry 2/3 exited 22, retrying in 2 seconds...

❕ Flaky test report

No test was executed to be analysed.

@simitt simitt mentioned this pull request Jan 19, 2021
Closed
16 tasks
@simitt simitt mentioned this pull request Feb 5, 2021
Closed
@simitt simitt mentioned this pull request Jun 1, 2021
Closed
20 tasks
@mergify
Copy link
Contributor

mergify bot commented Sep 22, 2021

This pull request does not have a backport label. Could you fix it @ruflin? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Sep 22, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 27, 2021
@andresrc andresrc added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Nov 3, 2021
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Nov 3, 2021
@ruflin
Copy link
Member Author

ruflin commented Jan 17, 2022

@joshdover I'm closing this PR as I think this is something the Fleet team already start to pick up and fits much more in the area of Fleet.

@ruflin ruflin closed this Jan 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@ruflin ruflin

Labels
backport-skip Skip notification from the automated backport with mergify Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ruflin @elasticmachine @andresrc @jen-huang