CodeLens AI: Full-Stack Code Intelligence Workspace

This repository contains a complete code intelligence system with:

• A FastAPI backend that ingests repositories, parses code with Tree-sitter, builds embeddings, indexes with FAISS, and answers grounded questions with Qwen.
• A Vite + React frontend that manages projects, uploads ZIPs, triggers indexing, and provides docs + chat UI.

Repository Structure

this_studio/
  backend/
    app/
      api/            # FastAPI routes (projects, chat, docs)
      core/           # settings, logging, DB models/wrapper
      ingestion/      # project creation + ZIP extraction + file scanning
      parsing/        # tree-sitter parsing + chunking
      embeddings/     # embedding generation
      indexing/       # indexing orchestration + FAISS persistence
      retrieval/      # chunk retrieval for Q&A
      llm/            # Qwen service + prompts
      docsgen/        # summary/doc generation
    data/             # sqlite DB, repos, vector indexes, cache
    scripts/          # smoke test helpers
    requirements.txt
    .env.example
    main.py
    README.md

  frontend/
    src/
      api/            # typed API client + route wrappers
      hooks/          # useProjects/useProject/useProjectDocs/useProjectChat
      utils/          # error normalization + dev logging
      App.tsx         # UI views and interaction flows
    .env.example
    package.json
    README.md

  README.md           # this file

High-Level Flow

1. Frontend signs in with Google and calls POST /auth/google.
2. Backend verifies Google ID token and issues an app session cookie.
3. Frontend creates a project: POST /projects.
4. Frontend uploads a ZIP: POST /projects/{id}/upload-zip.
5. Backend extracts allowed source files into backend/data/repos/{id}.
6. Frontend triggers indexing: POST /projects/{id}/index.
7. Backend parses/chunks/embeds and persists FAISS + metadata.
8. Frontend loads docs: GET /projects/{id}/docs.
9. Frontend sends chat question: POST /projects/{id}/chat.

Backend API Contract (Used by Frontend)

• POST /auth/google -> { user: { ... } } + session cookie
• GET /auth/me -> { user: { ... } }
• POST /auth/logout -> { ok: true }
• GET /projects -> { projects: [...] }
• POST /projects -> Project
• GET /projects/{id} -> Project
• POST /projects/{id}/upload-zip -> { project_id, accepted, skipped, repo_path }
• POST /projects/{id}/index -> { project_id, status, files_indexed, chunks_indexed, symbols_extracted }
• GET /projects/{id}/docs -> { project_id, summaries: [...] }
• POST /projects/{id}/chat -> { project_id, answer, citations }

Prerequisites

• Python 3.10+
• Node.js 18+
• Enough disk + memory for model downloads and inference

Quick Start (End-to-End)

1. Start backend

cd backend
pip install -r requirements.txt
cp .env.example .env
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

2. Start frontend

In a separate terminal:

cd frontend
npm install
cp .env.example .env.local
# ensure VITE_API_BASE_URL points to backend
npm run dev

Frontend default dev URL: http://localhost:5173 Backend default URL: http://localhost:8000

Environment Variables

Backend (backend/.env)

Important settings:

• APP_ENV, LOG_LEVEL
• QWEN_MODEL_ID, EMBEDDING_MODEL_ID
• QWEN_MAX_INPUT_TOKENS, QWEN_MAX_NEW_TOKENS, QWEN_TEMPERATURE
• GOOGLE_CLIENT_ID, SESSION_SECRET, SESSION_COOKIE_NAME, SESSION_TTL_SECONDS, SESSION_COOKIE_SECURE
• DATA_ROOT, REPO_ROOT, INDEX_ROOT, CACHE_ROOT, TMP_ROOT, SQLITE_PATH
• CORS_ALLOW_ORIGINS (JSON array)

Frontend (frontend/.env.local)

• VITE_API_BASE_URL (required for backend connectivity)
• VITE_GOOGLE_CLIENT_ID (required for Google Identity Services)
• VITE_APP_URL (optional, deployment-specific)

CORS and Frontend Connectivity

Backend uses FastAPI CORSMiddleware and allows local dev origins by default:

• http://localhost:5173
• http://127.0.0.1:5173
• http://localhost:3000
• http://127.0.0.1:3000

If your frontend origin is different, update CORS_ALLOW_ORIGINS in backend/.env.

Data and Persistence

• SQLite metadata: backend/data/app.db
• Extracted repositories: backend/data/repos/{project_id}
• FAISS index files: backend/data/indexes/project_{id}.index and metadata JSON
• Cache/model files: under backend/data/cache

Validation Commands

Backend

cd backend
python scripts/smoke_test.py

Frontend

cd frontend
npm run lint
npm run build
npm run preview

Common Issues and Fixes

• POST /projects hangs:
  • fixed in current backend by removing nested lock acquisition in DB create path.
• Indexing fails with missing package errors:
  • ensure pip install -r backend/requirements.txt completed (includes einops and accelerate).
• Browser CORS errors:
  • verify backend is running with correct CORS_ALLOW_ORIGINS.
• Chat is slow on first run:
  • expected while Qwen weights load/download; subsequent calls are faster.

Related Module Docs

• Backend module docs: backend/README.md
• Frontend module docs: frontend/README.md

Current Development Focus

• Preserve stable API contracts between frontend and backend.
• Keep frontend data flow fully backend-driven (no browser-side direct LLM calls).
• Prefer targeted fixes over architecture rewrites.