[hostcfgd] Error while creating management VRF via CLI

[samaity]

Description:
When attempting to create a management VRF using the command sudo config vrf add mgmt, the following error is encountered:

Nov 19 16:04:51.026785 sonic ERR hostcfgd: ['grep', '-E', 'eth0\\s+00000000\\s+[0-9A-Z]+\\s+[0-9]+\\s+[0-9]+\\s+[0-9]+\\s+202'] - failed: return code - 1, output:#012<_io.TextIOWrapper name=9 encoding='UTF-8'>
Nov 19 16:04:51.026966 sonic ERR hostcfgd: MgmtIfaceCfg: Could not delete eth0 route

Even though the route table shows the following entries for eth0:

admin@sonic:~$ cat /proc/net/route | grep eth0
eth0    00000000    0184F90A    0003    0    0    0    00000000    0    0    0
eth0    0084F90A    00000000    0001    0    0    0    C0FFFFFF    0    0    0

Steps to Reproduce:

1. Run the following command:

sudo config vrf add mgmt

2. Observe the system logs using:

sudo journalctl -u hostcfgd

Expected Behavior:
The hostcfgd service should successfully manage the routes associated with eth0 and complete the VRF creation process without errors.

Observed Behavior:
The route manipulation for eth0 fails with an error, indicating that the command
grep -E 'eth0\s+00000000\s+[0-9A-Z]+\s+[0-9]+...' did not find the expected entry.

[samaity]

@qiluo-msft, Do you happen to know why this error occurs? If not, could you kindly point me to the right person or resource to troubleshoot this?

[vivekrnv]

Hi @samaity, Did you see this on images 202311 and before?

[samaity]

Hi @samaity, Did you see this on images 202311 and before?

I noticed it in 202305, but should be in 202311 too