DeepSeed Training

End-to-end deep learning training with live D3.js dashboards. Train BERT and GPT-2 models on free GPUs and visualize every metric in real time.

Live Dashboard — real training data from BERT-large on Tesla T4

⚡ New: Tiny Transformer (understand it in one sitting)

A 112k-param GPT, written from scratch in ~150 lines, trained on Tiny Shakespeare in ~1 minute on a T4. Every piece of the model is explained. Interactive dashboard lets you drag a slider to watch the model learn across training.

• View the tiny transformer dashboard →
• Tiny Transformer from scratch — start here

Other notebooks:

• BERT distillation:
• GPT-2 LoRA:

What's Here

notebooks/       Colab/Kaggle notebooks (run for free on T4 GPUs)
scripts/         Training scripts with live metric streaming
server.py        Dashboard server with SSE for live training
docs/            GitHub Pages — static dashboard with real results

Quick Start

1. View Results (no GPU needed)

Visit the live dashboard — it loads real training metrics from a BERT-large distillation run (335M params, 92.7% accuracy, Tesla T4).

2. Run Training on Free GPUs

Google Colab (recommended):

1. Open notebooks/deepspeed_bert_colab.ipynb in Colab
2. Select Runtime → T4 GPU
3. Run all cells — metrics stream to the dashboard automatically

Kaggle (2x T4 GPUs free):

1. Upload notebooks/deepspeed_bert_colab.ipynb as a Kaggle notebook
2. Enable GPU accelerator
3. Run all cells

3. Local with Live Dashboard

# Terminal 1: start dashboard
python3 server.py

# Terminal 2: run training
python3 scripts/train_bert_distill.py

Open http://localhost:8080 to see live training charts.

4. Remote Training + Local Dashboard

python3 server.py --tunnel

This creates a public URL via Cloudflare tunnel. Paste it into your Colab/Kaggle notebook to stream metrics back to your local dashboard.

Models

Model	Task	Method	Notebook
BERT-large → 4-layer student	SST-2 sentiment	Knowledge distillation	deepspeed_bert_colab.ipynb
GPT-2	SST-2 / WikiText-2	LoRA (rank=8)	deepspeed_gpt2_colab.ipynb

Dashboard Features

• Loss, accuracy, F1 curves with epoch boundaries
• GPU utilization and memory tracking
• Per-layer timing breakdown
• Learning rate schedule visualization
• Gradient norm monitoring
• Drag-and-drop any training_metrics.json to compare runs

Requirements

torch transformers datasets peft accelerate

All notebooks install dependencies automatically — no local setup needed for Colab/Kaggle.