Architecture

Prax is organized in five layers with a clear direction of dependency: blueprints → services → agent → plugins (plus the legacy readers/ helpers being migrated into plugins). The hub-and-spoke architecture keeps each agent’s tool count low while providing deep domain capabilities.

Contents

• Hub-and-Spoke Architecture — Orchestrator, spoke agents, sub-hubs, and delegation patterns
• The lang* Stack — How Prax uses LangChain/LangGraph: the build_agent_loop construction seam (layer rule 4), in-loop middleware (AGENT_MIDDLEWARE_ENABLED), version policy, and the durable-resume decision gate
• Request Flows — SMS, Discord, TeamWork, sandbox, and scheduling flows
• Workspace — Per-user git-backed file layout, TeamWork integration, Dropbox sync
• Interoperability frontier — Where Prax sits in the agent-interop ecosystem across five axes (expose / consume / discover / interchange / identity). Already a first-class provider (MCP server + OKF interchange); the highest-leverage gap is the consumer half — an outbound MCP/A2A client that makes any external agent a spoke. Content provenance is the natural second frontier; capability discovery (ARD) stays deferred
• Memory System — STM scratchpad, LTM vector + graph, hybrid retrieval, consolidation

High-Level System Overview

graph TB
    User["User (Phone / Discord / Web)"]

    subgraph TeamWork["TeamWork Web UI"]
        TWChat["Chat Channels"]
        TWKanban["Kanban Board"]
        TWTerminal["Terminal"]
        TWBrowser["Browser Screencast"]
    end

    subgraph Twilio["Twilio Cloud (optional)"]
        Voice["Voice Webhook"]
        SMS["SMS Webhook"]
    end

    subgraph Discord["Discord (optional)"]
        DiscordBot["Discord Bot\n(WebSocket)"]
    end

    subgraph Flask["Flask App"]
        MainRoutes["/transcribe, /respond"]
        SmsRoute["/sms"]
        VoiceSvc["VoiceService"]
        SmsSvc["SmsService"]
        DiscordSvc["DiscordService"]
        ConvoSvc["ConversationService"]
    end

    subgraph Agent["LangGraph ReAct Agent (Prax)"]
        Orchestrator["ConversationAgent"]
        LLMFactory["LLM Factory"]

        subgraph Tools["Orchestrator Tools (45 as of 2026-09, sandbox enabled)"]
            BuiltIn["Kernel (4)\nsearch, datetime, URL fetch, sandbox_shell"]
            WS["Planning / meta / introspection (24)\nagent_plan_*, progress_*, user_notes_*, think,\nrun_python, trace_*, review_my_traces, doctor, ..."]
            Spokes["Spoke Delegates (15)\nbrowser, content_editor, course, desktop,\nenvironment, finetune, knowledge, plugins,\nprofessor, research, sandbox, scheduler,\nsysadmin, tasks, workspace"]
            SA["Sub-Agent (2)\ndelegate_task, delegate_parallel"]
        end
    end

    subgraph PluginSys["Plugin System (Hot-Swappable)"]
        PluginLoader["Plugin Loader\n(folder-per-plugin discovery)"]
        PluginSandbox["Subprocess Sandbox"]
        PluginRegistry["Version Registry\n(registry.json)"]
        PluginCatalog["CATALOG.md\n(auto-generated)"]
        PromptMgr["Prompt Manager"]
        LLMConfig["LLM Routing\n(llm_routing.yaml)"]
        BuiltInPlugins["Built-In Plugins\n(NPR, PDF, YouTube, arXiv, ...)"]
        CustomPlugins["Custom Plugins\n(plugins/tools/custom/)"]
        SysPrompt["System Prompt\n(plugins/prompts/)"]
    end

    subgraph PluginRepo["Plugin Repository (optional)"]
        RemoteRepo["Private Git Repo\n(SSH deploy key)"]
        RepoBranch["Branch: plugins"]
        RemoteCatalog["CATALOG.md"]
    end

    subgraph Storage["Persistence"]
        SQLite["SQLite\n(conversation memory)"]
        Workspace["Git-Backed Workspace\n(per-user files)"]
        ScheduleYAML["schedules.yaml\n(cron definitions)"]
        AdapterReg["adapter_registry.json\n(LoRA adapters)"]
        Todos["todos.json\n(user to-do list)"]
        Links["links.md\n(link history)"]
        Instructions["instructions.md\n(prompt reference)"]
        AgentPlan["agent_plan.yaml\n(task decomposition)"]
    end

    subgraph Memory["Memory System (optional)"]
        STM["STM Scratchpad\n(workspace JSON)"]
        Qdrant["Qdrant\n(dense + sparse vectors)"]
        Neo4j["Neo4j\n(entity graph)"]
        Embedder["Embedder\n(OpenAI / Ollama / local)"]
    end

    subgraph Sandbox["Docker Sandbox"]
        Container["Docker Container"]
        Exec["Pure exec env\n(shell · Python · DuckDB · Lean · desktop)"]
    end

    subgraph LocalML["Local ML Stack (optional)"]
        vLLM["vLLM Server\n(OpenAI-compat API)"]
        LoRA["LoRA Adapters\n(hot-swappable)"]
        Unsloth["Unsloth QLoRA\n(training subprocess)"]
    end

    subgraph Browser["Browser (Playwright)"]
        Chromium["Headless Chromium"]
        SiteCreds["sites.yaml\n(credentials)"]
        Profiles["Persistent Profiles\n(cookies/sessions)"]
        VNCServer["Xvfb + x11vnc\n(manual login)"]
    end

    Worktree["Git Worktree\n(self-modification)"]

    Scheduler["APScheduler\n(background cron)"]

    User -->|Web| TWChat
    User -->|Call| Voice
    User -->|Text| SMS
    User -->|Message| DiscordBot
    TWChat -->|Webhook| ConvoSvc
    TWTerminal -->|docker exec| Container
    TWBrowser -->|CDP| Chromium
    Voice --> MainRoutes --> VoiceSvc
    SMS --> SmsRoute --> SmsSvc
    DiscordBot --> DiscordSvc
    VoiceSvc --> ConvoSvc
    SmsSvc --> ConvoSvc
    DiscordSvc --> ConvoSvc
    ConvoSvc --> Orchestrator
    Orchestrator --> LLMFactory
    LLMFactory -->|cloud| BuiltIn
    LLMFactory -->|local| vLLM
    Orchestrator --> Tools
    WS --> Workspace
    Spokes -->|sandbox| Container
    Container --> Exec
    Spokes -->|scheduler| ScheduleYAML
    Spokes -->|finetune| Unsloth
    Spokes -->|finetune| vLLM
    vLLM --> LoRA
    Spokes -->|sysadmin → self-improve| Worktree
    Spokes -->|sysadmin| PluginLoader
    PluginLoader --> BuiltInPlugins
    PluginLoader --> CustomPlugins
    PluginLoader --> PluginSandbox
    PluginLoader --> PluginRegistry
    PluginLoader --> PluginCatalog
    CustomPlugins -.->|push| RemoteRepo
    RemoteRepo --> RepoBranch
    RepoBranch --> RemoteCatalog
    Orchestrator --> PromptMgr
    PromptMgr --> SysPrompt
    LLMFactory --> LLMConfig
    Spokes -->|browser| Chromium
    Chromium --> SiteCreds
    Chromium --> Profiles
    VNCServer --> Chromium
    ConvoSvc --> SQLite
    Scheduler -->|reads| ScheduleYAML
    Scheduler -->|fires| ConvoSvc
    ConvoSvc -->|SMS / Discord reply| User
    Worktree -->|PR| Workspace
    Orchestrator -->|context inject| STM
    Orchestrator -->|turn-end consolidation| Qdrant
    Orchestrator -->|turn-end consolidation| Neo4j
    Embedder --> Qdrant
    STM --> Workspace

Concepts — What Lives Where

Prax is organized in five layers. Each has a clear job and a single direction of dependency: blueprints → services → agent → plugins, with readers/ as a legacy fifth layer that is being folded into plugins.

How they connect:

User ──▸ Twilio/Discord
            │
        Blueprints          (HTTP layer — routes, auth)
            │
        Services             (business logic — workspace, sandbox, browser, scheduler, …)
            │
        Agent                (LangGraph ReAct loop — orchestrator, tools, governance)
            │
        Plugins              (hot-swappable tools — news, PDF, YouTube, custom, …)

Rules of thumb:

• Need a new channel? Add a blueprint + a channel service.
• Need a new capability (e.g., email sending)? Add a service, then wrap it with a tool builder in agent/ or a plugin in plugins/tools/.
• Need a new tool the agent can call? If it’s a core, always-on tool, add it to an agent/*_tools.py builder. If it’s optional, content-focused, or user-modifiable, make it a plugin.
• Need to change the system prompt? Edit plugins/prompts/system_prompt.md (or let the agent do it at runtime via prompt_write).

Memory System

Prax has a two-layer memory system inspired by human cognition, implemented across three data stores:

┌─────────────────────────────────────────────────────┐
│                  Orchestrator                        │
│  (injects STM + LTM context into system prompt)     │
└────────────┬────────────────────────┬───────────────┘
             │                        │
     ┌───────▼───────┐       ┌───────▼───────────┐
     │  STM (fast)   │       │  LTM (durable)    │
     │  Workspace    │       │  ┌─────────────┐  │
     │  JSON file    │       │  │ Qdrant      │  │
     │  (always on)  │       │  │ dense+sparse│  │
     │               │       │  └──────┬──────┘  │
     │  key → value  │       │         │ RRF     │
     │  + importance │       │  ┌──────┴──────┐  │
     │  + tags       │       │  │ Neo4j       │  │
     │               │       │  │ entities    │  │
     └───────────────┘       │  │ temporals   │  │
                             │  │ causals     │  │
                             │  └─────────────┘  │
                             └───────────────────┘
                                      │
                             ┌────────▼────────┐
                             │ Consolidation   │
                             │ (scheduled)     │
                             │ traces → LLM    │
                             │ → conf gate≥0.6 │
                             │ → graph + vector│
                             │ → dual decay    │
                             └─────────────────┘

Short-term memory (STM) is a per-user JSON scratchpad stored in the workspace ({workspace}/memory/stm.json). It requires no infrastructure — works even when Qdrant/Neo4j are unreachable. The orchestrator injects STM entries into the system prompt on every turn.

Long-term memory (LTM) requires Qdrant (vector store) and Neo4j (knowledge graph) — both are compiled into the prax Docker image and start with the container (no compose profile; make run-local-all starts them as containers for a host-process Prax). Memories are stored as both dense embeddings (semantic similarity) and sparse TF-IDF vectors (keyword matching) in Qdrant, plus entities, typed relations, temporal events, and causal links in Neo4j (multi-graph separation). At query time, all three retrieval arms (dense, sparse, graph neighbourhood) run in parallel and results are fused via query-adaptive weighted RRF — factual queries boost sparse+graph, semantic queries boost dense.

Consolidation converts conversation traces into durable memories: LLM extraction of entities/relations/facts/temporal events/causal links → confidence validation gate (≥0.6 → LTM, below → STM pending review) → graph upsert with bi-temporal edges (valid_from/valid_until for supersession tracking) → vector upsert → dual decay (Ebbinghaus time-based + interaction-based, using the stronger signal) → daily summaries.

Embedding providers are pluggable: OpenAI (default, cloud), Ollama (local, no data leaves your machine), or fastembed (in-process, zero dependencies). The system gracefully degrades — if Qdrant or Neo4j are unreachable, LTM returns empty results without errors. STM always works.

A memory spoke (prax/agent/spokes/memory/) with explicit remember/recall/forget/graph tools exists but is deliberately not registered with the orchestrator (see the note in prax/agent/spokes/__init__.py): memory writes happen automatically through the turn-end consolidation hook and reads through the per-turn context injection. See Memory deep-dive for full documentation.