RvR: VPC redundant vrs run on same hypervisor by ustcweizhou · Pull Request #3421 · apache/cloudstack

ustcweizhou · 2019-06-24T14:48:58Z

Description

For VPC supports redundant VRs, when start the second VR, the pod/cluster/host of first VR should be added to avoid list. This provides higher availability.

The network VRs have the same process already.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

yadvr · 2019-06-25T08:14:49Z

+            final List<DomainRouterVO> routerList = _routerDao.listByVpcId(router.getVpcId());
+            for (final DomainRouterVO rrouter : routerList) {
+                if (rrouter.getHostId() != null && rrouter.getIsRedundantRouter() && rrouter.getState() == State.Running) {
+                    if (routerToBeAvoid != null) {


should this also check that the routerList.size() is >= 2?

it should be checked at some other places I think.
I am too lazy I just copied the lines from 380 to 390
https://github.com/apache/cloudstack/blob/4.11/server/src/com/cloud/network/router/NetworkHelperImpl.java#L380,L390

anuragaw

Overall LGTM but code could use some cleanup.

anuragaw

Thanks @ustcweizhou !

LGTM

yadvr · 2019-06-27T07:20:47Z

@blueorangutan package

blueorangutan · 2019-06-27T07:21:09Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-06-27T07:42:25Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-47

yadvr · 2019-06-27T10:42:53Z

@blueorangutan test

blueorangutan · 2019-06-27T10:43:08Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2019-06-27T20:34:26Z

Trillian test result (tid-41)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33954 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3421-t41-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 67 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces	`Error`	197.01	test_internal_lb.py
test_05_rvpc_multi_tiers	`Failure`	418.29	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	442.26	test_vpc_redundant.py

yadvr · 2019-06-27T20:47:15Z

@blueorangutan test

blueorangutan · 2019-06-27T20:48:11Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

ustcweizhou · 2019-06-27T20:58:54Z

@rhtyd if it still fails, I will look into it tomorrow.

blueorangutan · 2019-06-28T06:08:47Z

Trillian test result (tid-42)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31840 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3421-t42-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 68 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_04_rvpc_network_garbage_collector_nics	`Failure`	330.93	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	412.71	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	437.94	test_vpc_redundant.py

ustcweizhou · 2019-06-28T12:40:48Z

@rhtyd
I checked the result but did not understand why the issue happened...

======================================================================
FAIL: Create a redundant VPC with 1 Tier, 1 VM, 1 ACL, 1 PF and test Network GC Nics
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/marvin/tests/smoke/test_vpc_redundant.py", line 597, in test_04_rvpc_network_garbage_collector_nics
    self.do_vpc_test(False)
  File "/marvin/tests/smoke/test_vpc_redundant.py", line 690, in do_vpc_test
    self.check_ssh_into_vm(vm.get_vm(), vm.get_ip(), expectFail=expectFail, retries=retries)
  File "/marvin/tests/smoke/test_vpc_redundant.py", line 533, in check_ssh_into_vm
    self.fail("Failed to SSH into VM - %s" % (public_ip.ipaddress.ipaddress))
AssertionError: Failed to SSH into VM - 10.1.34.69

and

======================================================================
FAIL: Create a redundant VPC with multiple tiers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/marvin/tests/smoke/test_vpc_redundant.py", line 633, in test_05_rvpc_multi_tiers
    self.do_vpc_test(False)
  File "/marvin/tests/smoke/test_vpc_redundant.py", line 690, in do_vpc_test
    self.check_ssh_into_vm(vm.get_vm(), vm.get_ip(), expectFail=expectFail, retries=retries)
  File "/marvin/tests/smoke/test_vpc_redundant.py", line 533, in check_ssh_into_vm
    self.fail("Failed to SSH into VM - %s" % (public_ip.ipaddress.ipaddress))
AssertionError: Failed to SSH into VM - 10.1.34.71

and

======================================================================
ERROR: Create a redundant VPC with multiple tiers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/marvin/tests/smoke/test_vpc_redundant.py", line 284, in tearDown
    raise Exception("Warning: Exception during cleanup : %s" % e)
Exception: Warning: Exception during cleanup : Execute cmd: deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete network offering 78 as its used by 1 networks. To make the network offering unavaiable, disable it

yadvr · 2019-06-28T13:34:44Z

Okay @ustcweizhou I'll kick them one more time. Third's the winner ;)

@blueorangutan package

blueorangutan · 2019-06-28T13:35:14Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-06-28T13:57:29Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-62

yadvr · 2019-06-28T14:04:07Z

@blueorangutan test

blueorangutan · 2019-06-28T14:05:13Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2019-06-28T21:45:52Z

Trillian test result (tid-55)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 25736 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3421-t55-kvm-centos7.zip
Smoke tests completed. 69 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

For VPC supports redundant VRs, when start the second VR, the pod/cluster/host of first VR should be added to avoid list. This provides higher availability. The network VRs have the same process already.

yadvr added this to the 4.13.0.0 milestone Jun 25, 2019

yadvr reviewed Jun 25, 2019

View reviewed changes

yadvr approved these changes Jun 25, 2019

View reviewed changes

svenvogel added component:vpc type:bug labels Jun 25, 2019

anuragaw reviewed Jun 26, 2019

View reviewed changes

Comment thread server/src/com/cloud/network/router/NetworkHelperImpl.java

ustcweizhou force-pushed the rvr-vpc-same-hv branch from c1aa019 to c6139f9 Compare June 26, 2019 19:02

RvR: VPC redundant vrs run on same hypervisor

cf2ee49

ustcweizhou force-pushed the rvr-vpc-same-hv branch from c6139f9 to cf2ee49 Compare June 26, 2019 19:03

anuragaw approved these changes Jun 27, 2019

View reviewed changes

yadvr merged commit 9ac32aa into apache:4.11 Jun 29, 2019

Conversation

ustcweizhou commented Jun 24, 2019

Description

Types of changes

Uh oh!

yadvr Jun 25, 2019

Choose a reason for hiding this comment

Uh oh!

ustcweizhou Jun 25, 2019

Choose a reason for hiding this comment

Uh oh!

ustcweizhou Jun 25, 2019

Choose a reason for hiding this comment

Uh oh!

anuragaw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

anuragaw left a comment

Choose a reason for hiding this comment

Uh oh!

yadvr commented Jun 27, 2019

Uh oh!

blueorangutan commented Jun 27, 2019

Uh oh!

blueorangutan commented Jun 27, 2019

Uh oh!

yadvr commented Jun 27, 2019

Uh oh!

blueorangutan commented Jun 27, 2019

Uh oh!

blueorangutan commented Jun 27, 2019

Uh oh!

yadvr commented Jun 27, 2019

Uh oh!

blueorangutan commented Jun 27, 2019

Uh oh!

ustcweizhou commented Jun 27, 2019

Uh oh!

blueorangutan commented Jun 28, 2019

Uh oh!

ustcweizhou commented Jun 28, 2019

Uh oh!

yadvr commented Jun 28, 2019

Uh oh!

blueorangutan commented Jun 28, 2019

Uh oh!

blueorangutan commented Jun 28, 2019

Uh oh!

yadvr commented Jun 28, 2019

Uh oh!

blueorangutan commented Jun 28, 2019

Uh oh!

blueorangutan commented Jun 28, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants