SegFormer.AI is a premium, production-grade web application designed for real-time semantic segmentation of urban road scenes. Built with a state-of-the-art SegFormer-B5 Transformer architecture and a high-performance FastAPI backend, it provides pixel-perfect classification of Cityscapes dataset categories.
- 🚀 Real-time Inference: Ultra-low latency processing powered by NVIDIA SegFormer-B5.
- 🎨 Premium UI/UX: A modern, responsive dashboard with glassmorphism aesthetics and fluid Framer Motion animations.
- 🔍 Inspection Mode: Interactive high-resolution zoom modal for detailed mask analysis.
- 📊 Analytics Dashboard: Real-time breakdown of detected object classes (e.g., Road, Building, Vegetation, Pedestrian) with percentage distribution.
- 📥 One-Click Export: Instant download for both classification masks and semantic overlays.
- 🔗 Dynamic Backend Bridging: Seamlessly connect the frontend to remote GPU inference nodes via Ngrok.
- Framework: Next.js 15 (App Router)
- Styling: Tailwind CSS
- Animations: Framer Motion
- Icons: Lucide React
- Deployment: Vercel
- Engine: FastAPI
- Model: SegFormer-B5 (Transformer-based)
- Environment: Google Colab / Linux with CUDA Support
- Exposure: Ngrok
Since the heavy lifting (GPU inference) is handled on a remote node (Google Colab), you need to bridge the frontend to the backend using Ngrok.
- Open the SegFormer Backend Notebook in Google Colab.
- Set your
NGROK_AUTHTOKENin the secrets/environment variables. - Run all cells (
Ctrl + F9). - Copy the generated Ngrok Public URL (e.g.,
https://xxxx-xx-xx-xx.ngrok-free.app).
- Navigate to the SegFormer.AI Playground.
- Locate the Backend Node configuration field.
- Paste your Ngrok URL and hit enter.
- The system will now route all segmentation requests to your live Colab GPU node.
If you wish to run the frontend locally:
- Node.js 18.x or higher
- npm / yarn / pnpm
# Clone the repository
git clone https://github.com/BehzadHassan/SegFormer.AI.git
# Navigate to the frontend directory
cd SegFormer.AI
# Install dependencies
npm install
# Start the development server
npm run devThe app will be available at http://localhost:3000.
The project uses a proxy architecture to handle communication between the Vercel-deployed frontend and the Colab-hosted backend:
- Client-Side: The user uploads an image and provides a dynamic API URL.
- Next.js API Route: The
/api/segmentroute acts as a secure proxy, injecting necessary headers (likengrok-skip-browser-warning) to ensure smooth communication with Ngrok. - FastAPI Backend: Receives the image, processes it through the SegFormer-B5 model using PyTorch, and returns base64-encoded masks and analytics.
- Model: Developed by NVIDIA, fine-tuned on the Cityscapes Dataset.
- Inspiration: Designed for developers and researchers to visualize Transformer-based segmentation in a professional web environment.
Developed with ❤️ by Behzad Hassan