This project implements a sarcasm detection model for news headlines using a BERT-based architecture with PyTorch. The dataset is sourced from Kaggle and the workflow is organized in a Jupyter notebook.
- Overview
- Dataset
- Model Architecture
- Training Details
- Results & Visualization
- How to Run
- Requirements
- References
This notebook demonstrates how to classify news headlines as sarcastic or not using a fine-tuned BERT model. The workflow includes data loading, preprocessing, model definition, training, evaluation, and visualization of results.
- Source: Kaggle - News Headlines Dataset For Sarcasm Detection
- Download: The dataset is automatically downloaded using the
opendatasetslibrary. - Format: JSON lines, with fields:
headline,is_sarcastic, andarticle_link(the latter is dropped).
- Base Model: BERT (google-bert/bert-base-uncased from HuggingFace Transformers)
- Custom Layers:
- Dropout
- Linear layers (768→384→1)
- Sigmoid activation for binary classification
- Frozen BERT: Only the custom layers are trained; BERT parameters are frozen.
- Loss Function: Binary Cross Entropy Loss (BCELoss)
- Optimizer: Adam
- Batch Size: 32
- Epochs: 10
- Learning Rate: 1e-4
- Data Split: 70% train, 15% validation, 15% test
- Training and validation loss/accuracy are tracked and plotted over epochs.
- Final test accuracy is reported.
- Clone the repository and open the notebook in Jupyter or VS Code.
- Ensure you have Python 3.7+ and install the required packages:
pip install torch transformers opendatasets scikit-learn matplotlib pandas numpy
- Run all cells in the notebook. The dataset will be downloaded automatically.
- Python 3.7+
- torch
- transformers
- opendatasets
- scikit-learn
- matplotlib
- pandas
- numpy
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- HuggingFace Transformers
- Kaggle Dataset
Feel free to use or modify this project for your own sarcasm detection or NLP experiments!