TuneForge - LLM Fine-Tuning Monitoring Dashboard

TuneForge is a self-hosted monitoring dashboard for LLM fine-tuning runs. Your trainer streams metrics to your own server over a simple HTTP API, and TuneForge turns them into live charts, alerts and dataset diagnostics. No third-party cloud logging, no per-seat fees: your data stays on your infrastructure.

Built with Next.js 16, TypeScript, Tailwind CSS and Prisma (SQLite). Includes full source code, setup guide, connector API docs, a Python connector example, demo-data seeding and a connector simulator.

The live demo runs on simulated data (no backend). The real product connects to your own training runs. Demo login password: TuneForgeDemo2026!

Features

- Real-time training metrics over Server-Sent Events: loss, validation loss, learning rate, tokens/sec - GPU monitoring: temperature, VRAM fragmentation, thermal-throttle detection, predictive alerts, cost meter - Dataset insights: loss-spike attribution to samples, label-confidence histograms, train/eval contamination scans - Model health: gradient norms, update ratios, forgetting evaluation - Experiment registry with side-by-side run comparison and config diff - Alerts to Discord/Slack-style webhooks with six configurable triggers - Checkpoint storage with upload, download, tagging and optional HuggingFace push - Inference playground for any OpenAI-compatible endpoint with token streaming - Simple HTTP ingest API with API-key auth, plus an MLflow / W&B import script - Single-admin password login, rate-limited, with a CLI password reset

Requirements

- Node.js 20.9+ (22 recommended) and npm 10+ - No separate database server needed (SQLite is included via Prisma) - Optional: webhook URL for alerts, HuggingFace token for checkpoint push, an OpenAI-compatible endpoint for the playground - HTTPS (e.g. Caddy or nginx) for any access other than localhost

Instructions

1. Extract the ZIP and open the tuneforge folder. 2. Copy .env.example to .env. Set JWT_SECRET and INGEST_API_KEY (generate each with: openssl rand -hex 32). 3. Run: npm install 4. Run: npx prisma migrate deploy 5. Run: npm run build 6. Run: npm start 7. Open http://localhost:3000/setup and choose your admin password. 8. Connect your trainer using docs/connector-api.md (includes a Python example). Optional: load demo data with  npx tsx scripts/seed-demo-data.ts Full details are in README.md and SETUP.md.