Allow users to view reserved System VM IPs, if they're already allocated to user

[SadiJr]

Description

The default behavior of ACS is to try to use all available public IPs, including those reserved for system VMs; by default when a public IP is reserved to System VM, they are used as preferred to be allocated to system VMs, but not restricted to these type of VMs. This behavior can be changed by changing the value of the global setting system.vm.public.ip.reservation.mode.strictness from false to true. However, if an IP reserved for system VMs is already in use for a user VR, the parameter change will make the IP to be hidden from the user, which can lead to all sorts of confusion. Therefore, this PR is intended to address these situations.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

How Has This Been Tested?

It was tested in a local lab:

1. I allocated a range of 10 IPs to System VMs with system.vm.public.ip.reservation.mode.strictness as false;
2. I allocated all the other IPs to another network;
3. I created a new network and started a new VM with this network;
4. Then I changed the system.vm.public.ip.reservation.mode.strictness to true;
5. Before the changes, in the VR details (UI), there is no IP information;
6. With this PR, the user can see correctly the IP.

Also, I added new unit tests.

[RodrigoDLopez]

LGTM

[nvazquez]

@blueorangutan package

[blueorangutan]

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2582

[nvazquez]

@blueorangutan test

[blueorangutan]

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian Build Failed (tid-3307)

[nvazquez]

@blueorangutan test

[blueorangutan]

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian Build Failed (tid-3309)

[GutoVeronezi]

LGTM

[nvazquez]

@blueorangutan test

[blueorangutan]

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-3342)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45780 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5902-t3342-kvm-centos7.zip
Smoke tests completed. 91 look OK, 1 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_invalid_upgrade_kubernetes_cluster	Failure	3600.96	test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster	Failure	3614.94	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	Failure	0.04	test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster	Failure	0.04	test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster	Failure	0.04	test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster	Failure	0.04	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	Failure	0.04	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	Failure	0.04	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	Failure	0.03	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	Error	89.82	test_kubernetes_clusters.py

[github-actions]

Hi @${author}, your pull request has merge conflicts. Can you fix the conflicts and sync your branch with the base branch?

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✖️ el7 ✖️ el8 ✖️ debian ✖️ suse15. SL-JID 3165

[weizhouapache]

@blueorangutan package

[blueorangutan]

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with

SystemVM template(s). I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3170

[weizhouapache]

@blueorangutan test

[blueorangutan]

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-3909)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 32452 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5902-t3909-kvm-centos7.zip
Smoke tests completed. 93 look OK, 0 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File

[weizhouapache]

@SadiJr
I have tested this PR. unforunately the issue you mentioned in description is not fixed.

I have made some changes to your code. see weizhouapache@18137e1
what I have tested are

1. global setting system.vm.public.ip.reservation.mode.strictness is set to false

• 1.1 Free IPs in the range can be listed (OK to me)
• 1.2 Free IPs in the range can be associated to isolated networks (OK to me)
• 1.3 Allocated IPs (in step 1.2) are listed in the "Public IP Addresses" tab in the network details page. (OK to me)

2. global setting system.vm.public.ip.reservation.mode.strictness is set to true

• 2.1 Free IPs in the range can NOT be listed (OK to me)
• 2.2 Free IPs in the range can NOT be associated to isolated networks by cmk (OK to me)
• 2.3 Allocated IPs (in step 1.2) are listed in the "Public IP Addresses" tab in the network details page. (OK to me)

other than that, I change the global configuration to dynamic, so it is not required to restart management server when you change the value.

Could you please test it ? If it works, please advise if you add the commit to your PR or I create a new PR which including all changes.

[SadiJr]

@weizhouapache thanks for yours tests and your help. I will implement your suggestions and test them. If all goes well, I'll add a new commit with your changes.

[SadiJr]

@weizhouapache

I did some more tests, and everything is working good. I also checked that, with the setting as true, when releasing a reserved IP from the system VMs (but in use by a user VM), it is correctly excluded from the listing and it is also not possible for the user to reallocate it via cmk . Also, I created a new commit with your changes.

Again, thank you very much for your testing and your help.

[acs-robot]

Found UI changes, kicking a new UI QA build
@blueorangutan ui

[acs-robot]

Found Java/XML changes, kicking packaging job
@blueorangutan package

[blueorangutan]

@acs-robot a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[acs-robot]

PR Coverage Report

CLASS	INSTRUCTION MISSED	INSTRUCTION COVERED	BRANCH MISSED	BRANCH COVERED	LINE MISSED	LINE COVERED
ClusteredAgentManagerImpl	2361	0	242	0	536	0
DefaultVMSnapshotStrategy	486	677	52	30	100	140
LibvirtVMDef	37	114	6	4	11	36
VmwareManagerImpl	2613	528	296	38	615	117
VmwareStorageProcessor	9886	9	940	0	2121	2
NetScalerControlCenterResource	1943	0	144	0	468	0
NetscalerResource	6882	0	806	0	1623	0
DateraPrimaryDataStoreDriver	3195	0	283	0	748	0
CloudStackPrimaryDataStoreDriverImpl	903	0	114	0	229	0
LinstorPrimaryDataStoreDriverImpl	1442	0	91	0	348	0
ScaleIOPrimaryDataStoreDriver	2537	0	246	0	537	0
SolidFirePrimaryDataStoreDriver	3347	0	284	0	697	0
SAMLUtils	202	465	41	11	53	108
Config	152	5162	30	6	42	342
IpAddressManagerImpl	4045	0	461	0	806	0
ManagementServerImpl	11774	0	1052	0	2350	0
VolumeApiServiceImpl	10826	0	1498	0	2031	0
SnapshotManagerImpl	4186	0	410	0	751	0
TaggedResourceManagerImpl	473	0	58	0	96	0
AccountManagerImpl	6427	0	900	0	1348	0
PremiumSecondaryStorageManagerImpl	775	0	64	0	116	0
SecondaryStorageManagerImpl	3494	149	343	11	623	32

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3203

[weizhouapache]

@blueorangutan test

[blueorangutan]

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[weizhouapache]

@SadiJr
the travis smoke test test_portable_publicip failed, it seems to be related to this PR.

Currently running test: smoke/test_portable_publicip
==== Marvin Init Started ====
=== Marvin Parse Config Successful ===
=== Marvin Setting TestData Successful===
==== Log Folder Path: /tmp/MarvinLogs/Apr_18_2022_23_17_26_IICLVA All logs will be available here ====
=== Marvin Init Logging Successful===
==== Marvin Init Successful ====
=== TestName: test_createPortablePublicIPAcquire | Status : EXCEPTION ===
=== TestName: test_createPortablePublicIPRange | Status : EXCEPTION ===
=== Final results are now copied to: /tmp/MarvinLogs/test_portable_publicip_UW5HUH ===
real	0m9.150s
user	0m0.841s
sys	0m0.068s

[blueorangutan]

Trillian test result (tid-3929)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 30904 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5902-t3929-kvm-centos7.zip
Smoke tests completed. 93 look OK, 0 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File