Run tests against an actual Kibana build #11237

rhoboat · 2017-04-13T20:06:13Z

This PR updates the jenkins selenium task that runs on jenkins for each build. Instead of starting up the kibana script in bin/kibana, now we will run the build --release task and start up kibana from build/kibana-${version}-linux-x86_64/bin/kibana. Then we run our functional tests on that.

It has passed ✅ green on jenkins thrice and is ready for your review!

rhoboat · 2017-04-14T16:18:07Z

extract the version

rhoboat · 2017-04-14T16:40:55Z

jenkins, test it

rhoboat · 2017-04-19T20:24:35Z

Halle-f***ing-lujah!

rhoboat · 2017-04-19T20:32:12Z

tasks/config/run.js

+        ...kbnServerFlags,
+      ]
+    },
+


This is copypasta from testUIServer. Is there a better way? Maybe reference the previous object and merge/extend it?

It's your judgement call I'd say :). Something like the below isn't all that unwieldy.
const uiConfig = ...the thing...

testUIServer: { ...uiConfig, cmd: binScript }, testUIReleaseServer: { ...uiConfig, cmd: binScript },

Either way's fine IMHO

rhoboat · 2017-04-19T20:33:36Z

tasks/test.js

+    'esvm_shutdown:ui',
+    'stop:testUIReleaseServer'
+  ]);
+


Also copypasta. Hm. I suppose one could pass in an array that's reused except for certain items, but I don't think it looks so great in this case. Would be super unclear indirection.

rhoboat · 2017-04-19T20:34:14Z

test/scripts/jenkins_selenium.sh

@@ -3,4 +3,6 @@
 set -e
 source "$(dirname $0)/_jenkins_setup.sh"

+"$(npm bin)/grunt" build --release;


Is this ok to do? Do we care that we're not running tests against the SNAPSHOT build?

I don't know, doesn't look bad to me. Maybe @LeeDr has input?

rhoboat · 2017-04-20T15:20:11Z

jenkins, test this

cjcenizal · 2017-04-20T16:24:13Z

Does anyone know, or is it written down anywhere, what is responsible for building things in /build? And what is a snapshot build, and when it's built?

thomasneirynck

For consistency, I'd consider adding the testUiReleaseServer task as a runnable to the package.json as well.

thomasneirynck · 2017-04-20T16:34:00Z

Code looks good to me. Not sure why the last build was aborted though...

thomasneirynck · 2017-04-20T16:34:06Z

jenkins, test this

jbudz · 2017-04-20T16:36:47Z

tasks/config/run.js

@@ -6,6 +6,8 @@ module.exports = function (grunt) {
  const root = p => resolve(__dirname, '../../', p);
  const binScript =  /^win/.test(platform) ? '.\\bin\\kibana.bat' : './bin/kibana';
  const buildScript =  /^win/.test(platform) ? '.\\build\\kibana\\bin\\kibana.bat' : './build/kibana/bin/kibana';
+  const pkgVersion = grunt.config.get('pkg.version');
+  const releaseBinScript = `./build/kibana-${pkgVersion}-linux-x86_64/bin/kibana`;


can we make this dynamic? a combination of os.platform() and os.arch() should get us there

make dynamic; a combination of os.platform() and os.arch()
Leaving this undone, we will want to revisit and make sure the tests can run on more than just linux-x86_64, but for now this it.

jbudz · 2017-04-20T16:57:47Z

@cjcenizal there's no difference between a snapshot and release build other than the name. Some portions of the code will reference the name (like advanced settings stored at .kibana/config/{version} and the status api).

The grunt build task is responsible for generating the build folder. It will end up containing init scripts, translations, and folders for the different kibana platforms. The target folder will contain the final built tar/zip/deb/rpms, which are created from the platform folders in build.

rhoboat · 2017-04-20T17:49:55Z

@thomasneirynck re: adding the testUiReleaseServer task as a runnable to package.json

I think we are moving away from having npm run <task> in favor of node <script> so that we run scripts directly rather than packaging them up with npm as a dependency.

epixa · 2017-04-24T18:14:55Z

@archanid Can you backport this? At least to 5.3, though it may be worth putting all the way back to 5.0 if it backports cleanly.

epixa · 2017-04-24T21:47:24Z

Come to think of it, it'll probably be difficult to backport beyond the 5.4 branch since previous versions do not have the new functional test runner. We should at least backport to 5.x and 5.4 though.

* build before running selenium * change directory into built snapshot * WIP: what is in the build dir * WIP: skip unit tests * run selenium on build * [functional tests on build] Derive version * [functional tests on build] Missing folder name prefix * WIP: debug on ubuntu VM * WIP: double kibana stabilize timeout * WIP: remove ubuntu VM debug change * WIP: add logs for docParams * WIP: add logs for es * build release version without -SNAPSHOT- * [functional test release] Cleanup

elastic locked and limited conversation to collaborators Apr 13, 2017

elastic unlocked this conversation Apr 19, 2017

rhoboat requested review from tylersmalley and epixa April 19, 2017 20:30

rhoboat commented Apr 19, 2017

View reviewed changes

rhoboat added the Team:QA Team label for QA Team label Apr 19, 2017

rhoboat changed the title ~~WIP: build before running selenium~~ Run tests against an actual Kibana build Apr 19, 2017

rhoboat added the Team:Operations Team label for Operations Team label Apr 19, 2017

rhoboat requested a review from jbudz April 20, 2017 15:15

thomasneirynck approved these changes Apr 20, 2017

View reviewed changes

jbudz reviewed Apr 20, 2017

View reviewed changes

archanid added 8 commits April 21, 2017 10:33

WIP: build before running selenium

7258df4

WIP: change directory into built snapshot

d9ca7ef

WIP: what is in the build dir

23f920f

WIP: skip unit tests

1a09380

WIP: run selenium on build

95359e5

[functional tests on build] Derive version

3bc0e8d

[functional tests on build] Missing folder name prefix

a3a8c38

WIP: debug on ubuntu VM

63178af

archanid added 6 commits April 21, 2017 10:33

WIP: double kibana stabilize timeout

a862c66

WIP: remove ubuntu VM debug change

cd4d137

WIP: add logs for docParams

d7b3776

WIP: add logs for es

00bed01

WIP: build release version without -SNAPSHOT-

a0f0c5d

[functional test release] Cleanup

9648b91

rhoboat merged commit 476031b into elastic:master Apr 21, 2017

rhoboat deleted the wip-build-selenium branch April 21, 2017 16:08

rhoboat mentioned this pull request Jul 18, 2018

CI never tests the pure runtime build #20922

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run tests against an actual Kibana build #11237

Run tests against an actual Kibana build #11237

rhoboat commented Apr 13, 2017 •

edited

Loading

rhoboat commented Apr 14, 2017 •

edited

Loading

rhoboat commented Apr 14, 2017

rhoboat commented Apr 19, 2017

rhoboat Apr 19, 2017

thomasneirynck Apr 20, 2017 •

edited

Loading

rhoboat Apr 19, 2017

rhoboat Apr 19, 2017

thomasneirynck Apr 20, 2017

rhoboat commented Apr 20, 2017

cjcenizal commented Apr 20, 2017

thomasneirynck left a comment

thomasneirynck commented Apr 20, 2017

thomasneirynck commented Apr 20, 2017

jbudz Apr 20, 2017

rhoboat Apr 20, 2017 •

edited

Loading

jbudz commented Apr 20, 2017

rhoboat commented Apr 20, 2017

epixa commented Apr 24, 2017

epixa commented Apr 24, 2017

Run tests against an actual Kibana build #11237

Run tests against an actual Kibana build #11237

Conversation

rhoboat commented Apr 13, 2017 • edited Loading

rhoboat commented Apr 14, 2017 • edited Loading

rhoboat commented Apr 14, 2017

rhoboat commented Apr 19, 2017

rhoboat Apr 19, 2017

Choose a reason for hiding this comment

thomasneirynck Apr 20, 2017 • edited Loading

Choose a reason for hiding this comment

rhoboat Apr 19, 2017

Choose a reason for hiding this comment

rhoboat Apr 19, 2017

Choose a reason for hiding this comment

thomasneirynck Apr 20, 2017

Choose a reason for hiding this comment

rhoboat commented Apr 20, 2017

cjcenizal commented Apr 20, 2017

thomasneirynck left a comment

Choose a reason for hiding this comment

thomasneirynck commented Apr 20, 2017

thomasneirynck commented Apr 20, 2017

jbudz Apr 20, 2017

Choose a reason for hiding this comment

rhoboat Apr 20, 2017 • edited Loading

Choose a reason for hiding this comment

jbudz commented Apr 20, 2017

rhoboat commented Apr 20, 2017

epixa commented Apr 24, 2017

epixa commented Apr 24, 2017

rhoboat commented Apr 13, 2017 •

edited

Loading

rhoboat commented Apr 14, 2017 •

edited

Loading

thomasneirynck Apr 20, 2017 •

edited

Loading

rhoboat Apr 20, 2017 •

edited

Loading