Data Processing AI Agents

4-agent CrewAI pipeline for automated data analysis

← AI Lab
Open Live App ↗GitHub ↗Devpost

Submitted to the International AI Agents Hackathon 2026 · Hackathon code: backend/bots.py

Demo

Try It — Pipeline Simulator

Agent Pipeline Simulator — Watch the 4-agent CrewAI pipeline process data step by step.
1
Context Agent
Rewrites request into precise directive
▸
2
Prompt Engineer
Builds step-by-step instructions
▸
3
Senior Data Analyst
Flags outliers, summarises findings
▸
4
Output Formatter
Enforces schema, rates quality
▸
Live at data-processing-ai-agents.vercel.app ↗

Description

What it does: Upload a data file through the React frontend. It gets pushed to a CrewAI backend on HuggingFace where four specialised agents process it in sequence — a context agent figures out what you want, a prompt engineer writes precise instructions, a senior data analyst identifies outliers and summarises findings, and an output agent formats the result and rates how well it matched your original request.

How it was built: React + Vite frontend hosted on Vercel, connected via REST API to a CrewAI backend on HuggingFace. LLMs sourced through OpenRouter's free tier with a model rotation fallback in backend/bots.py — when one model hits rate limits, the system automatically switches to the next available provider.

Why it exists: Built to understand how agentic frameworks like CrewAI work under the hood — and to make multi-step data analysis accessible without needing to write code.

How the Agents Work

  Data Analyst AI Agents — CrewAI Pipeline
  ══════════════════════════════════════════

  ┌─────────────────────────────────────────┐
  │        User Upload (React + Vite)       │
  │              Vercel frontend            │
  └──────────────────┬──────────────────────┘
                     │  REST API (async)
                     ▼
  ┌─────────────────────────────────────────┐
  │      CrewAI Backend (HuggingFace)       │
  │                                         │
  │  1. Context Agent                       │
  │     rewrites request into a precise,    │
  │     unambiguous directive               │
  │                   │                     │
  │                   ▼                     │
  │  2. Prompt Engineer                     │
  │     builds step-by-step instructions   │
  │     tailored to the data and goal       │
  │                   │                     │
  │                   ▼                     │
  │  3. Senior Data Analyst                 │
  │     reads file, flags outliers,         │
  │     summarises findings                 │
  │                   │                     │
  │                   ▼                     │
  │  4. Output Formatter                    │
  │     enforces JSON schema, picks         │
  │     chart type, rates answer quality    │
  └──────────────────┬──────────────────────┘
                     ▼
  ┌─────────────────────────────────────────┐
  │  Structured result + quality rating     │
  │  (bar / line / pie chart or report)     │
  └─────────────────────────────────────────┘

  LLMs: OpenRouter free tier · Groq fast pool with fallback rotation
bots.py handles rate limits with per-model cooldown tracking — when one provider fails, the pipeline automatically retries on the next.

Dev Notes

Tech Stack

React + Vite (Vercel), Python + CrewAI (HuggingFace), REST API, OpenRouter free LLMs with Groq fast pool and automatic fallback rotation in backend/bots.py.

Hardest Part

API keys maxing out constantly during testing. LLM models erroring despite correct OpenRouter calls — flaky free-tier availability made debugging slow and unpredictable.

What I Learned

CrewAI manages agentic workflows cleanly when the prompts are precise. The hard part isn't wiring the agents — it's writing instructions specific enough that each agent stays in its lane.