New feature: Resource count (CPU/RAM) take only running vms into calculation

[ustcweizhou]

Description

Stopped VMs do not use CPU/RAM actually.
A new global configuration resource.count.running.vms.only is added to determine whether resource (cpu/memory) of only running vms (including Starting/Stopping) will be taken into calculation of resource consumption.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

all testing in test/integration/component/test_resource_count_running_vms.py passed, including
deploy/start/stop/reboot a vm
update a vm with displayvm
destroy/expunge vm

tested with resource.count.running.vms.only is true and false both.

Two issues found in the testing: #3758 #3759

[luhaijiao]

This is introducing a flexible way to calculate the resource only for running VMs. It's particulalry useful for some customer senario, e.g. quota calculation.

Thanks weizhou !

[svenvogel]

@weizhouapache nice feature. its good to get the calculation in a better state.

[andrijapanicsb]

@blueorangutan package

[blueorangutan]

@andrijapanicsb a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-528

[andrijapanicsb]

@blueorangutan test

[wido]

LGTM based on code

[blueorangutan]

@andrijapanicsb a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[andrijapanicsb]

wow, it only took 2 days for the monkey to kick the tests... nice

[blueorangutan]

Trillian test result (tid-780)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40049 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3760-t780-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 76 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_hostha_enable_ha_when_host_in_maintenance	Error	303.51	test_hostha_kvm.py

[yadvr]

@ustcweizhou can you fix the merge conflict?

[weizhouapache]

@rhtyd rebased with latest master

[yadvr]

@blueorangutan package

[blueorangutan]

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✖centos6 ✖centos7 ✔debian. JID-688

[DaanHoogland]

good code with tests added. work to Wei standard! standard style remark; big methods are getting bigger, no biggy (in this case)

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-698

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-848)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 30664 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3760-t848-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 78 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

[yadvr]

@ustcweizhou can you check the one failing Travis test?

[weizhouapache]

@rhtyd
rebased with latest master
let's see if travis test fails again.

[DaanHoogland]

all travis failures are long logs. merging

[yadvr]

@ustcweizhou a new marvin test was added to component, if it's a fast tests (doesn't take time to run) - please move to smoke and also add it to .travis.yml cc @DaanHoogland

[DaanHoogland]

@rhtyd we already suffer from failures due to long logs. I think we must skim the load we lay on travis. (someday soon)

[yadvr]

@DaanHoogland tests added to component are not run by default, so it's like not adding the test in the first place. A new job runner/block can be easily added to Travis. We've like 20 odd jobs/runs per PR.

[DaanHoogland]

we are using 18. is 20 the maximum that you know, @rhtyd ? I would like to see each test run for each change, but that may be too costly.