Capturing coverage of vmdkops service #1421

pshahzeb · 2017-06-14T23:35:22Z

Adding extra target to capture coverage.
Modifying drone script to call the target on ESX

CI ESX box is patched to have python coverage package.

Testing: Done locally.
Following is the end of: make build-all deploy-all test-all coverage

PASS
coverage: 40.6% of statements
/Library/Developer/CommandLineTools/usr/bin/make --directory=vmdk_plugin coverage
Makefile:152: warning: overriding commands for target `clean'
../plugin/Makefile:87: warning: ignoring old commands for target `clean'

=> Running target coverage Thu Jun 15 13:25:26 PDT 2017

/Library/Developer/CommandLineTools/usr/bin/make  --directory=../esx_service coverage
python coverage package exists.
ssh  -kTax -o StrictHostKeyChecking=no root@10.192.63.232 '/etc/init.d/vmdk-opsd stop ; coverage combine .coverage* ; coverage report ; rm .coverage*'
Stopping vmdkops-opsd with PID=674977
675010
vmdkops-opsd is not running
Name                                                Stmts   Miss  Cover
-----------------------------------------------------------------------
/usr/lib/vmware/vmdkops/Python/__init__.py              0      0   100%
/usr/lib/vmware/vmdkops/Python/auth.py                323     90    72%
/usr/lib/vmware/vmdkops/Python/auth_api.py            657    182    72%
/usr/lib/vmware/vmdkops/Python/auth_data.py           623    203    67%
/usr/lib/vmware/vmdkops/Python/auth_data_const.py      23      0   100%
/usr/lib/vmware/vmdkops/Python/cli_table.py            82      1    99%
/usr/lib/vmware/vmdkops/Python/convert.py              14      2    86%
/usr/lib/vmware/vmdkops/Python/error_code.py           57      0   100%
/usr/lib/vmware/vmdkops/Python/kvESX.py               189     56    70%
/usr/lib/vmware/vmdkops/Python/log_config.py           57     17    70%
/usr/lib/vmware/vmdkops/Python/test_utils.py           94     12    87%
/usr/lib/vmware/vmdkops/Python/threadutils.py          52      4    92%
/usr/lib/vmware/vmdkops/Python/vm_listener.py         109     21    81%
/usr/lib/vmware/vmdkops/Python/vmdk_utils.py          211     35    83%
/usr/lib/vmware/vmdkops/Python/vsan_info.py            44      6    86%
/usr/lib/vmware/vmdkops/bin/__init__.py                 0      0   100%
/usr/lib/vmware/vmdkops/bin/vmdk_ops.py               994    278    72%
/usr/lib/vmware/vmdkops/bin/vmdkops_admin.py          564    163    71%
/usr/lib/vmware/vmdkops/bin/volume_kv.py               65     19    71%
/usr/lib/vmware/vmdkops/bin/vsan_policy.py            161     72    55%
-----------------------------------------------------------------------
TOTAL                                                4319   1161    73%

Fixes #404

shaominchen

LGTM.

shaominchen · 2017-06-14T23:41:15Z

vmdk_plugin/Makefile

@@ -347,6 +347,26 @@ test-esx:
 testremote: test-esx test-vm
 test-all:  test e2e-test testremote

+# binary for python coverage
+COVERAGE := coverage
+#vmdkops service manager


Nit: add a space after #

shaominchen · 2017-06-14T23:41:32Z

vmdk_plugin/Makefile

+coverage:
+	$(log_target)
+ifdef COV_EXIST
+		@echo "coverage package exists."


Nit: coverage => python coverage

shaominchen · 2017-06-14T23:41:45Z

vmdk_plugin/Makefile

+		$(SSH) root@$(ESX) '$(COVERAGE) report'
+		$(SSH) root@$(ESX) 'rm .coverage*'
+else
+		@echo "coverage package doesn't exist."


Same as above.

shuklanirdesh82

Thanks for working on this quickly!

shuklanirdesh82 · 2017-06-15T02:33:38Z

misc/drone-scripts/deploy-and-test-wrapper.sh

@@ -108,7 +108,7 @@ runtests)
          TARGET+=" e2e-test-runalways test-vm"
        else
          touch /tmp/$ESX
-          TARGET+=" deploy-vm e2e-test-runalways e2e-test-runonce testasroot test-esx test-vm"
+          TARGET+=" deploy-vm e2e-test-runalways e2e-test-runonce testasroot test-esx test-vm coverage"


we need to move this after L118

added a switch case as per discussed offline

shuklanirdesh82 · 2017-06-15T02:34:56Z

vmdk_plugin/Makefile

@@ -347,6 +347,26 @@ test-esx:
 testremote: test-esx test-vm
 test-all:  test e2e-test testremote

+# binary for python coverage


let's not pollute this file .. this target should move to esx_service Makefile.

shuklanirdesh82 · 2017-06-15T02:54:22Z

@pshahzeb Please trigger the CI run for this PR.

Testing: Done locally.

Good to have some information from local run target.

A quick question: what happens incase of error while capturing the coverage data (fail the PR test run or ignores silently)?

pdhamdhere · 2017-06-15T18:40:39Z

CI ESX box is patched to have python coverage package.

Where are changes documented? What additional configuration is required if I need to run this on my local ESX.

Testing: Done locally.

Please add details like target compiled, output of coverage etc.

pshahzeb · 2017-06-15T22:06:36Z

@shuklanirdesh82 @pdhamdhere

Updated the PR description for a sample output end section.
Updated contributing.md to document the changes to setup the esx box

A quick question: what happens incase of error while capturing the coverage data (fail the PR test run or ignores silently)?

If the tests passed, coverage capture should not fail. If ever it does, that means
call to ./misc/drone-scripts/deploy-and-test-wrapper.sh coverage failed.
I think it will fail PR. We can only verify in on CI once it actually fails for any PR.

Updating contributing.md

shuklanirdesh82

LGTM! You could cut/paste coverage info from CI run in the PR description or paste the CI run URL along with local run for future reference.

Supplying minor nits to address.

shuklanirdesh82 · 2017-06-16T01:47:00Z

vmdk_plugin/Makefile

@@ -347,6 +347,11 @@ test-esx:
 testremote: test-esx test-vm
 test-all:  test e2e-test testremote

+# capture coverage of ESX Python code.


minor: capture coverage of ESX Python code => capture coverage of vDVS driver code

Ref: https://i2.wp.com/www.virtuallyghetto.com/wp-content/uploads/2016/05/docker-volume-driver-for-vsphere-00.png

shuklanirdesh82 · 2017-06-16T01:51:12Z

CONTRIBUTING.md

+User can configure the setup and use this target to see the coverage.
+### Setup ESX box to install python coverage package
+* Install https://coverage.readthedocs.io/en/coverage-4.4.1/ on your machine using pip
+- scp the coverage package dir installed on ur machine i.e. <Python Location>/site-packages/coverage to /lib64/python3.5/site-packages/ on ESX box.


would suggest to add the sample command instead of text .. sample command would be handy. Command reference would be easy. :)

added exact commands

shuklanirdesh82 · 2017-06-16T01:53:04Z

CONTRIBUTING.md

+- scp the coverage package dir installed on ur machine i.e. <Python Location>/site-packages/coverage to /lib64/python3.5/site-packages/ on ESX box.
+- scp the coverage binary i.e. /usr/local/bin/coverage to /bin/ on ESX box
+* create sitecustomize.py inside /lib64/python3.5/
+The content of sitecustomize.py is


nit: need one more line (\n) ... it is appending with L389

Relaxing args check for CI deploy and test script

pdhamdhere

Some minor doc comments.

pdhamdhere · 2017-06-16T05:22:54Z

CONTRIBUTING.md

@@ -378,3 +378,31 @@ Test SSH keys, login form the drone node should not require typing in a password
 cd $GOPATH/src/github.com/vmware/docker-volume-vsphere/
 drone exec --trusted --yaml .drone.dev.yml -i ~/.ssh/id_rsa -e VM1=<ip VM1> -e VM2=<ip VM2> -e ESX=<ip ESX>
 ```
+
+## Capturing coverage


ESX Code Coverage

pdhamdhere · 2017-06-16T05:23:51Z

CONTRIBUTING.md

+Coverage is captured using make coverage target (On CI it is called using drone script).
+User can configure the setup and use this target to see the coverage.
+### Setup ESX box to install python coverage package
+* Install https://coverage.readthedocs.io/en/coverage-4.4.1/ on your machine using pip


Add exact command and prefix commands to run on ESX with ESX$ or laptop with Desktop$

pdhamdhere · 2017-06-16T05:24:20Z

CONTRIBUTING.md

+### Setup ESX box to install python coverage package
+* Install https://coverage.readthedocs.io/en/coverage-4.4.1/ on your machine using pip
+- scp the coverage package dir installed on ur machine i.e. <Python Location>/site-packages/coverage to /lib64/python3.5/site-packages/ on ESX box. <br />
+eg:- scp -r /Library/Python/3.5/site-packages/coverage root@$ESX:/lib64/python3.5/site-packages/


Include commands in command-in-single-quote

shuklanirdesh82

LGTM! good to go after addressing open comments.

pshahzeb · 2017-06-16T18:25:11Z

From CI https://ci.vmware.run/vmware/docker-volume-vsphere/565.
Build failed due to volume access test fail due to #1371

Name                                                Stmts   Miss  Cover
-----------------------------------------------------------------------
/usr/lib/vmware/vmdkops/Python/__init__.py              0      0   100%
/usr/lib/vmware/vmdkops/Python/auth.py                329     93    72%
/usr/lib/vmware/vmdkops/Python/auth_api.py            657    182    72%
/usr/lib/vmware/vmdkops/Python/auth_data.py           623    203    67%
/usr/lib/vmware/vmdkops/Python/auth_data_const.py      23      0   100%
/usr/lib/vmware/vmdkops/Python/cli_table.py            82      2    98%
/usr/lib/vmware/vmdkops/Python/convert.py              18      5    72%
/usr/lib/vmware/vmdkops/Python/error_code.py           58      0   100%
/usr/lib/vmware/vmdkops/Python/kvESX.py               208     68    67%
/usr/lib/vmware/vmdkops/Python/log_config.py           57     19    67%
/usr/lib/vmware/vmdkops/Python/test_utils.py           94     12    87%
/usr/lib/vmware/vmdkops/Python/threadutils.py          52      4    92%
/usr/lib/vmware/vmdkops/Python/vm_listener.py         109     21    81%
/usr/lib/vmware/vmdkops/Python/vmdk_utils.py          211     35    83%
/usr/lib/vmware/vmdkops/Python/vsan_info.py            44      6    86%
/usr/lib/vmware/vmdkops/bin/__init__.py                 0      0   100%
/usr/lib/vmware/vmdkops/bin/vmdk_ops.py               994    249    75%
/usr/lib/vmware/vmdkops/bin/vmdkops_admin.py          564    156    72%
/usr/lib/vmware/vmdkops/bin/volume_kv.py               65     19    71%
/usr/lib/vmware/vmdkops/bin/vsan_policy.py            161     72    55%
-----------------------------------------------------------------------
TOTAL                                                4349   1146    74%

pshahzeb · 2017-06-16T20:20:02Z

https://ci.vmware.run/vmware/docker-volume-vsphere/582 pass with coverage indicated.. Merging this.

This reverts commit d735de5.

vmwclabot added the cla-not-required label Jun 14, 2017

pshahzeb changed the title ~~Capturing coverage of vmdkops service~~ [WIP]Capturing coverage of vmdkops service Jun 14, 2017

pshahzeb changed the title ~~[WIP]Capturing coverage of vmdkops service~~ Capturing coverage of vmdkops service Jun 14, 2017

shaominchen approved these changes Jun 14, 2017

View reviewed changes

shuklanirdesh82 suggested changes Jun 15, 2017

View reviewed changes

shuklanirdesh82 assigned pshahzeb Jun 15, 2017

pshahzeb force-pushed the coverage_vmdkops.pshahzeb branch from 2885ded to 0ee9a45 Compare June 15, 2017 22:00

Capturing coverage of vmdkops esx side code

23ca315

Updating contributing.md

shuklanirdesh82 force-pushed the coverage_vmdkops.pshahzeb branch from 0ee9a45 to 23ca315 Compare June 15, 2017 23:52

shuklanirdesh82 reviewed Jun 16, 2017

View reviewed changes

Addressing nits.

cbb2199

Relaxing args check for CI deploy and test script

pdhamdhere approved these changes Jun 16, 2017

View reviewed changes

shuklanirdesh82 approved these changes Jun 16, 2017

View reviewed changes

Minor changes to Contributing.md

ac78fec

pshahzeb merged commit d735de5 into master Jun 16, 2017

shuklanirdesh82 deleted the coverage_vmdkops.pshahzeb branch June 16, 2017 20:28

shuklanirdesh82 added a commit that referenced this pull request Jun 16, 2017

Revert "Capturing coverage of vmdkops service (#1421)"

1b7d35f

This reverts commit d735de5.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Capturing coverage of vmdkops service #1421

Capturing coverage of vmdkops service #1421

pshahzeb commented Jun 14, 2017 •

edited

Loading

shaominchen left a comment

shaominchen Jun 14, 2017

pshahzeb Jun 15, 2017

shaominchen Jun 14, 2017

pshahzeb Jun 15, 2017

shaominchen Jun 14, 2017

pshahzeb Jun 15, 2017

shuklanirdesh82 left a comment

shuklanirdesh82 Jun 15, 2017

pshahzeb Jun 15, 2017

shuklanirdesh82 Jun 15, 2017

pshahzeb Jun 15, 2017

shuklanirdesh82 commented Jun 15, 2017

pdhamdhere commented Jun 15, 2017

pshahzeb commented Jun 15, 2017

shuklanirdesh82 left a comment •

edited

Loading

shuklanirdesh82 Jun 16, 2017

pshahzeb Jun 16, 2017

shuklanirdesh82 Jun 16, 2017

pshahzeb Jun 16, 2017

shuklanirdesh82 Jun 16, 2017

pshahzeb Jun 16, 2017

pdhamdhere left a comment

pdhamdhere Jun 16, 2017

pdhamdhere Jun 16, 2017

pshahzeb Jun 16, 2017

pdhamdhere Jun 16, 2017

pshahzeb Jun 16, 2017

shuklanirdesh82 left a comment

pshahzeb commented Jun 16, 2017

pshahzeb commented Jun 16, 2017

Capturing coverage of vmdkops service #1421

Capturing coverage of vmdkops service #1421

Conversation

pshahzeb commented Jun 14, 2017 • edited Loading

shaominchen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shuklanirdesh82 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shuklanirdesh82 commented Jun 15, 2017

pdhamdhere commented Jun 15, 2017

pshahzeb commented Jun 15, 2017

shuklanirdesh82 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pdhamdhere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shuklanirdesh82 left a comment

Choose a reason for hiding this comment

pshahzeb commented Jun 16, 2017

pshahzeb commented Jun 16, 2017

pshahzeb commented Jun 14, 2017 •

edited

Loading

shuklanirdesh82 left a comment •

edited

Loading