Adaptive Pseudo-Labeling via Word Coherence for Topic Modeling

This repository is the official implementation of Adaptive Pseudo-Labeling via Word Coherence for Topic Modeling. We propose adaptive pseudo-labeling for topic modeling (APT), a self-supervised framework that leverages document embeddings from pretrained transformers, directly reconstructs the Bag-of-Words (BoW) through the inherent relationship between documents and words, dynamically generates pseudo-labels, and integrates proxy-based deep metric learning to enhance coherence and diversity across topics.

Requirements

To install requirements:

pip install -r requirements.txt

Get started

The following lists the statistics of the datasets we used.

Dataset	Source link	Docs	Words	Categories
20Newsgroups	20NG	18846	9994	20
Wikitext-103	Wiki	28591	10000	N/A
Web of Science	WoS	11967	8813	7
Yahoo Answers Topics	Yahoo	29156	8902	10

After pre-processing, we divided the dataset into training and testing. Additionally, we removed words that exist only on training or testing datasets. We uploaded the code for dataset pre-processing in a folder named 'dataset'. The pre-processed version of benchmark datasets can be downloaded from here.

Training

To train our APT in the paper, run this command:

python train.py --data_path {data.pickle} --topic 50 --alpha 64. --mrg 0.1

Evaluation

To evaluate our model on {data.pickle}, run:

python eval.py --data_path {data.pickle} --model_path {model_weights.pth} --tc_topk 15 --td_topk 15 

Pretrained models

You can download pretrained models here:

• Our model trained on all benchmark datasets using default hyperparmeters.

Usage

Topic's top-k words

We can extract topic information, specifically the top words for each topic.

model.vocab = {data['vocab_dict']}
model.get_topic_word(top_k=k)

[[topic1_word1, topic1_word2, ... , topic1_wordk],
 [topic2_word1, topic2_word2, ... , topic2_wordk],...]

Embedding visualization

We can visualize embeddings for topics and documents using 'eval.py'. The image is stored in the './output'.

Citation

@inproceedings{yoon2026adaptive,
  author = {Bohan Yoon and Hyejin Jang},
  title = {Adaptive Pseudo-Labeling via Word Coherence for Topic Modeling},
  year = {2026}, isbn = {979-8-4007-2259-2/2026/08},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3770855.3817722},
  doi = {10.1145/3770855.3817722},
  booktitle = {Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD '26)},
  location = {Jeju Island, Republic of Korea}
}

The citation information may be updated upon final publication.