Question about K-L term objective

[ZZdozeoff]

Thanks for your contribution, this novel VED form pretrained dialog model is very helpful !
After reading your reasearch, I have a question about K-L regularization term.
Most variational generation in dialog models use CVAE objective, they have prior and posterior network and use K-L term $KL (q(z|c,r) || p(z|c) )$
But in your work, this posterior $q(z|c,r)$ is replaced by $q(z)$.
May I ask why do you use this replacement ?

[lemuria-wchen]

Hi! I'm sorry for the late reply.

The method you mentioned is the standard CVAE method, but our paper was originally intended to introduce a hidden variable to make a VAE-like model.

Our initial assumption is that the prior distribution of hidden variables is assumed to be standard normal distribution, and there is a posterior distribution due to the influence of context. We found that this method can already achieve better results in downstream tasks, so we finished the work first.

However, we are trying to use CVAE to conduct dialogue pre-training, which is the follow-up work of DialogVED and is preparing for submission. Welcome to follow.