Add functional and systemtest test pipeline for JDK8 into nightly build #2117

llxia · 2018-06-07T15:19:23Z

Issue: #2037

Signed-off-by: lanxia lan_xia@ca.ibm.com

llxia · 2018-06-07T15:21:04Z

Reviewers: @smlambert @jdekonin @AdamBrousseau

llxia · 2018-06-07T15:22:28Z

Tests:

AdamBrousseau · 2018-06-07T15:45:09Z

buildenv/jenkins/common/pipeline-functions

                case "_sanity":
-                    TEST_JOB_NAME = "Test-Sanity-JDK${SDK_VERSION}-${SPEC}"
+                    if (SPEC.contains("win") || SDK_VERSION.contains("JDK9") || SDK_VERSION.contains("JDK10")) {


I think SDK_VERSION is just the number i.e. 8/9/10
eg. https://github.com/eclipse/openj9/blob/master/buildenv/jenkins/jobs/builds/Build-JDK10-aix_ppc-64_cmprssptrs#L23

thanks for pointing out. I will update the changeset.

AdamBrousseau · 2018-06-07T15:46:10Z

buildenv/jenkins/common/pipeline-functions

+                    if (SPEC.contains("win") || SDK_VERSION.contains("JDK9") || SDK_VERSION.contains("JDK10")) {
+                        TARGET_NAMES = ["Sanity"]
+                    } else {
+                        TARGET_NAMES = ["Sanity", "sanity.functional", "sanity.system"]


Is there a reason why we are still doing Sanity in this case since it should be a duplicate of sanity.functional?

'sanity' target may now == 'sanity.functional' + 'sanity.system' so the 3 are different

I will remove Sanity once sanity.functional is stable. I do not want to cause any testing gap. Sanity should be removed next week.

If you've been running your sanity.functional and it works, then I think its safe to switch over right away. Maybe @pshipton has an opinion? It's the same testing just invoked with a different pipeline script right. The change is only for the nightly job right now, the PRs and the OMR Acceptance will continue to use the existing test jobs for now.

AdamBrousseau · 2018-06-07T15:46:26Z

buildenv/jenkins/common/pipeline-functions

+                    if (SPEC.contains("win") || SDK_VERSION.contains("JDK9") || SDK_VERSION.contains("JDK10")) {
+                        TARGET_NAMES = ["Extended"]
+                    } else {
+                        TARGET_NAMES = ["Extended", "extended.functional"]


Same question about duplicate testing

AdamBrousseau · 2018-06-07T15:46:40Z

buildenv/jenkins/common/pipeline-functions

                    break
                case "_extended":
-                    TEST_JOB_NAME = "Test-Extended-JDK${SDK_VERSION}-${SPEC}"
+                    if (SPEC.contains("win") || SDK_VERSION.contains("JDK9") || SDK_VERSION.contains("JDK10")) {


same comment about SDK_VERSION

AdamBrousseau · 2018-06-07T15:53:18Z

buildenv/jenkins/common/pipeline-functions

-            // run tests against the SDK build in the upstream job: Build-JDK${SDK_VERSION}-${SPEC}
-            jobs["${level}"] = build_with_one_upstream(TEST_JOB_NAME, BUILD_JOB_NAME, jobs["build"].getNumber(), params.VARIABLE_FILE, params.VENDOR_REPO, params.VENDOR_BRANCH, params.VENDOR_CREDENTIALS_ID, params.TEST_NODE)
-            echo "JOB: ${TEST_JOB_NAME} PASSED in: ${jobs[level].getDurationString()}"
+            for ( name in TARGET_NAMES ) {


Is there a reason why this loop is a for and not an each like above?

Is there a reason why I cannot use for?

no, just wondering for consistency.

AdamBrousseau · 2018-06-07T15:59:56Z

buildenv/jenkins/jobs/pipelines/Pipeline-OMR-Acceptance

@@ -33,7 +33,7 @@ def OPENJ9_BRANCH = 'master'
 def OMR_REPO = 'https://github.com/eclipse/openj9-omr.git'
 def OMR_BRANCH = 'master'

-TESTS_TARGETS = '_sanity'
+TESTS_TARGETS = '_sanity.tmp'


Also want to point out for better awareness. This will allow the OMR build to continue to run the same way it does today. Once the new jobs are proven stable in the nightly, we can remove this tmp target and the corresponding case in the workflow function.

AdamBrousseau · 2018-06-07T16:09:31Z

buildenv/jenkins/jobs/pipelines/Pipeline-OMR-Acceptance

@@ -33,7 +33,7 @@ def OPENJ9_BRANCH = 'master'
 def OMR_REPO = 'https://github.com/eclipse/openj9-omr.git'
 def OMR_BRANCH = 'master'

-TESTS_TARGETS = '_sanity'


Perhaps a question for @pshipton , when we change this target, will it be _sanity (functional and system) or will it be sanity.functional? If it is the latter, I think your default case would need the if win/9/10 also since the OMR build currently does 8 & 10 on all platforms (including win)

I think the answer of this question should not block this PR. By the time we switch OMR build to use AOJ test pipeline, hopefully win/9/10 issues are fixed. if not, we can add if in default case at that time.

AdamBrousseau · 2018-06-07T16:10:30Z

buildenv/jenkins/common/pipeline-functions

+        targets.each { target ->
+            switch (target.trim().toLowerCase()) {
+                // Due to the following reason, OpenJ9 test pipelines are still used temporarily: 
+                //  - waiting for wins machine configuration to be finalized


Should probably add some issue number(s) here

AdamBrousseau · 2018-06-07T16:11:23Z

buildenv/jenkins/common/pipeline-functions

+            switch (target.trim().toLowerCase()) {
+                // Due to the following reason, OpenJ9 test pipelines are still used temporarily: 
+                //  - waiting for wins machine configuration to be finalized
+                //  - there is no need to enable JDK9 as it is not supported anymore


AdamBrousseau · 2018-06-07T16:12:01Z

buildenv/jenkins/common/pipeline-functions

+                // Due to the following reason, OpenJ9 test pipelines are still used temporarily: 
+                //  - waiting for wins machine configuration to be finalized
+                //  - there is no need to enable JDK9 as it is not supported anymore
+                //  - switch to use AdoptOpenJDK test pipelines for JDK8 and we will switch for JDK10 next step


Is there an issue blocking us from doing 10 now too?

no, there is no issue blocking for jdk10. OpenJ9 Jenkins server has been very flaky recently, so I have hard time to run tests. That's why I will enable JDK8 first.

Issue: eclipse-openj9#2037 [ci skip] Signed-off-by: lanxia <lan_xia@ca.ibm.com>

llxia · 2018-06-07T17:44:10Z

changeset updated for SDK_VERSION and issue links

AdamBrousseau

LGTM other than my 1 question about duplicating sanity functional temporarily.

[skip ci] Issue eclipse-openj9#2117 eclipse-openj9#2037 Signed-off-by: Adam Brousseau <adam.brousseau88@gmail.com>

- Remove old style sanity & extended (functional) from nightly builds. - Machine resources are limited and new style (Adopt Pipelines) functional testing is stable, so this removes the old style (OpenJ9 pipelines) Related eclipse-openj9#2117 Issue eclipse-openj9#1450 [skip ci] Signed-off-by: Adam Brousseau <adam.brousseau88@gmail.com>

llxia force-pushed the test3 branch from 2b800e0 to 19f45a2 Compare June 7, 2018 15:20

llxia added the comp:test label Jun 7, 2018

AdamBrousseau requested changes Jun 7, 2018

View reviewed changes

AdamBrousseau mentioned this pull request Jun 7, 2018

Add generic Jenkins files pipelines #2093

Merged

llxia force-pushed the test3 branch from 19f45a2 to 8041bee Compare June 7, 2018 17:32

Add functional and systemtest test pipeline for JDK8 into nightly build

5977ff4

Issue: eclipse-openj9#2037 [ci skip] Signed-off-by: lanxia <lan_xia@ca.ibm.com>

llxia force-pushed the test3 branch from 8041bee to 5977ff4 Compare June 7, 2018 17:38

smlambert approved these changes Jun 13, 2018

View reviewed changes

AdamBrousseau approved these changes Jun 13, 2018

View reviewed changes

smlambert merged commit cd12bdc into eclipse-openj9:master Jun 14, 2018

AdamBrousseau mentioned this pull request Jun 14, 2018

Fix syntax problem in jobs array variable index reference #2200

Merged

AdamBrousseau added a commit to AdamBrousseau/openj9 that referenced this pull request Jun 14, 2018

Fix syntax problem in jobs array variable index reference

f1c48aa

[skip ci] Issue eclipse-openj9#2117 eclipse-openj9#2037 Signed-off-by: Adam Brousseau <adam.brousseau88@gmail.com>

AdamBrousseau mentioned this pull request Jun 16, 2018

Remove duplicate functional testing #2210

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add functional and systemtest test pipeline for JDK8 into nightly build #2117

Add functional and systemtest test pipeline for JDK8 into nightly build #2117

llxia commented Jun 7, 2018

llxia commented Jun 7, 2018

llxia commented Jun 7, 2018

AdamBrousseau Jun 7, 2018

llxia Jun 7, 2018

AdamBrousseau Jun 7, 2018

smlambert Jun 7, 2018

llxia Jun 7, 2018 •

edited

Loading

AdamBrousseau Jun 13, 2018

AdamBrousseau Jun 7, 2018

AdamBrousseau Jun 7, 2018

AdamBrousseau Jun 7, 2018

llxia Jun 7, 2018

AdamBrousseau Jun 13, 2018

AdamBrousseau Jun 7, 2018

AdamBrousseau Jun 7, 2018

llxia Jun 13, 2018

AdamBrousseau Jun 7, 2018

AdamBrousseau Jun 7, 2018

AdamBrousseau Jun 7, 2018

llxia Jun 7, 2018

llxia commented Jun 7, 2018

AdamBrousseau left a comment •

edited

Loading

Add functional and systemtest test pipeline for JDK8 into nightly build #2117

Add functional and systemtest test pipeline for JDK8 into nightly build #2117

Conversation

llxia commented Jun 7, 2018

llxia commented Jun 7, 2018

llxia commented Jun 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

llxia Jun 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

llxia commented Jun 7, 2018

AdamBrousseau left a comment • edited Loading

Choose a reason for hiding this comment

llxia Jun 7, 2018 •

edited

Loading

AdamBrousseau left a comment •

edited

Loading