LLM Red Team Lab
Interactive AI security lab scoring prompt-injection defenses with measurable block rates.
- Role
- Project Lead / AI Security Engineer
- Year
- 2026
- Category
- Software Dev
About This Project
Built a standalone Python red-team lab that compares vulnerable and hardened LLM chat pipelines side by side. Curated 27 attack prompts across OWASP LLM Top 10 categories, added rule-based input and output guardrails, and shipped a scorecard that reports block rate, benign pass rate, and latency overhead. The demo runs as a separate FastAPI service deployable via Railway or Render and links back to the portfolio as a live AI security case study.
Technologies Used
Engineering Challenges
Attack Coverage vs False Positives: Tuned regex-based input scanning so obvious prompt-injection patterns were blocked while benign recruiter questions about APIs, skills, and experience still passed through.
Dual Pipeline Parity: Ensured both vulnerable and hardened paths used the same mock LLM provider so comparison results reflected defense layers rather than model variance.
Measurable Security Story: Designed a scorecard endpoint and UI so block rate, benign regression pass rate, and latency overhead could be demonstrated in interviews without hand-waving.
Technical Highlights
Layered Guardrails: Implemented input scanning, server-side system prompt isolation, and output leak/PII scanning as separate defense layers with explicit audit results per request.
YAML Attack Catalog: Structured 21 attack and 6 benign prompts with OWASP categories and expected outcomes to drive automated regression testing.
Production-Ready Scaffold: Added FastAPI routes, Jinja lab UI, pytest suite, Docker image, Railway/Render deploy config, and GitHub Actions CI.
Business Logic
Recruiter-Friendly Demo: Visitors can break the unprotected pipeline, watch the protected pipeline block the same attack, and review a scorecard in under a minute.
Portfolio Separation: Kept Python runtime and demo hosting outside the Cloudflare Workers portfolio site while still linking the case study back to gssimao.com.
Interview Artifacts: Documented threat model, defense tradeoffs, and benchmark metrics recruiters and security engineers can discuss concretely.