Watch the SmartEngine turn one prompt into a planned, dependency-aware tool chain — then learn from the run. Scripted client-side; no live model is called.
Press “Step” to run the engine…
0/7
The actual engine code:
engine/engine.py
def execute(self, task: str, auto_synthesize=True) -> str:
tools = self.trainer.find_best_tools(task)
chain = self.trainer.suggest_tool_chains(task) # learned patterns
if chain:
steps = [PlanStep(step_id=i + 1, action=t, tool=t,
params={"task": task}, description=f"Execute {t}")
for i, t in enumerate(chain)]
steps.append(PlanStep(step_id=len(steps) + 1, action="synthesize",
tool="think", depends_on=list(range(1, len(steps) + 1))))
else:
steps = self.planner.plan(task, tools)
start = time.time()
result = self._execute_plan(task, steps, auto_synthesize)
self.trainer.learn_from_execution(task, steps=[s.__dict__ for s in steps],
success="error" not in result.lower()[:100],
duration=time.time() - start)
return result
Self-hosted CLI + web UI. Provider chain is Groq → Gemini → Claude → Ollama → local models, so it runs fully offline on Ollama when no cloud keys are set.
Description
What it does: Pixel Code Engine is an autonomous orchestration core. Given a task, its SmartEngineranks a dynamic tool registry, builds a plan of dependency-aware steps, executes them while piping each step's output into the next, and synthesizes a single grounded answer. When no existing tool fits, it can generate or compose a new one and register it at runtime.
Security-first: Every prompt and system message is run through a secret scanner that redacts API keys, tokens, and passwords beforeanything reaches a model provider. The assistant is built to introspect and modify its own source, so generated code is validated and security-scanned before it's ever applied.
It learns: After each run, the trainer records the task, the tool chain that worked, whether it succeeded, and how long it took. Next time a similar task arrives, that proven chain is suggested directly — skipping re-planning.
How it was built: Python, with a clean separation between the engine (planner, composer, generator, trainer, registry) and a library of sandboxed skills (file ops, shell, search, web fetch, screenshot, network, code execution). Pluggable providers cover Groq, Gemini, Claude, and local Ollama models with rate-limit-aware failover.
Status: Actively developed. Self-hosted; ships with Docker and Cloud Run deploy configs.
Plan → execute → synthesize, then learn. Secrets are stripped before the model; new tools are generated and validated on demand.
Dev Notes
Hardest Part
Dependency-aware execution. Each step declares depends_on; the executor only runs a step once its inputs exist, then injects prior results as result_from_step_N params — so a plan is a tiny DAG, not a flat list.
Security Design
The model is treated as untrusted infrastructure: inputs are secret-scanned and redacted before they leave the machine, and any self-generated code is validated and scanned before it can run. The engine never asks for keys.
What's Next
Richer learned-pattern reuse, broader tool generation, and a tighter local-model path so the whole engine runs offline on Ollama with no quality cliff.