[QST] Possible host KV memory leak / incomplete ownership cleanup in `kvcache_manager_impl.cpp`

[ShaobinChen-AH]

What is your question?

---

By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.

I notice that in "examples/commons/ops/cuda_ops/csrc/kvcache_manager_impl.cpp" function "GPUKVCacheMangerImpl::offload_loop()"

near line945: uint16_t *host_kv_ptr = static_cast<uint16_t *>(aligned_alloc(sysconf(_SC_PAGESIZE), host_bytes));

I could not find a complete cleanup path for those host allocations:
HostKVStorageImpl::~HostKVStorageImpl() is empty.
GPUKVCacheMangerImpl::~GPUKVCacheMangerImpl() only frees GPU buffers.
GPUKVCacheMangerImpl::evict_all() clears _uid_to_mempool, but the actual free(...) logic is commented out.
GPUKVCacheMangerImpl::invalid(uid) also does not appear to release host-side KV storage.

If this is indeed an unresolved issue and there is no ongoing fix, I would be happy to work on a PR.

[shijieliu]

@geoffreyQiu could you help take a look? thanks!

[geoffreyQiu]

@ShaobinChen-AH Hi, this issue is under ongoing fixing during the update to the kvcache interface in #370 .