Agents & LLM Orchestration¶

Build a multi-provider agent ensemble with expert system fallback that routes tasks to the best-suited LLM provider and falls back to deterministic rules when confidence is low.

Illustrative end-to-end example

A worked illustration of multi-provider routing — every component and operation named below is real, but it is not a copy-paste script. Steps 1–6 call paid third-party LLM provider APIs (Anthropic, OpenAI, etc.), so they need ALLOW_PAID_API=1 plus the relevant provider key; only Step 7 (adapt_experta, the deterministic fallback) runs offline at zero cost.

GoalInput¶

{
  "goal": "Build a multi-provider agent ensemble with expert system fallback",
  "context": "Route complex tasks across six LLM providers (Claude, OpenAI, LangChain, LangGraph, AutoGen, SmolagentS) based on task characteristics, and fall back to deterministic expert system rules when LLM confidence drops below threshold. Target: a customer support triage system that handles billing disputes, technical issues, account recovery, and escalation workflows.",
  "constraints": [
    "Each provider must handle only the task class it excels at",
    "LLM confidence below 0.60 triggers expert system fallback",
    "Multi-agent debate must reach consensus before escalation decisions",
    "Stateful workflows must persist across turns via LangGraph state"
  ],
  "resource_bounds": {
    "max_execution_seconds": 240,
    "max_tokens_per_hour": 150000
  },
  "checkpoints": [
    {
      "name": "confidence_gate",
      "predicate": "metric_below",
      "params": {"metric": "confidence", "threshold": 0.60},
      "description": "Log warning when agent confidence drops below 0.60 (triggers expert fallback)"
    }
  ],
  "guardrails": [
    {
      "name": "token_budget",
      "predicate": "resource_limit",
      "params": {"max_tokens_per_hour": 150000},
      "message": "Halt if hourly token budget exceeded"
    }
  ],
  "subtasks": [
    {
      "goal": "Perform complex reasoning and nuanced judgment on the incoming request",
      "context": "Use agent_claude to classify the support ticket, identify emotional tone, assess severity, and produce a structured triage report. Claude excels at multi-dimensional analysis with long context.",
      "constraints": ["Output structured JSON with severity, category, and tone fields"]
    },
    {
      "goal": "Execute structured tool calls to fetch account data",
      "context": "Use agent_openai for function-calling tasks: retrieve billing history, subscription status, and prior ticket history via structured tool definitions.",
      "constraints": ["All tool calls must use strict JSON schema definitions", "Max 5 tool rounds"]
    },
    {
      "goal": "Build a chain-of-thought resolution with tool retrieval",
      "context": "Use agent_langchain to reason step-by-step through resolution options, pulling in knowledge base articles and policy documents via retriever tools.",
      "constraints": ["Chain must include at least 3 reasoning steps", "Retriever top_k: 5"]
    },
    {
      "goal": "Manage stateful multi-step workflow for escalation",
      "context": "Use agent_langgraph to track the ticket through states: triage -> investigation -> resolution -> verification -> closed. Persist state across turns, handle conditional branching (escalate vs resolve).",
      "constraints": ["State graph must be acyclic", "Maximum 10 state transitions per ticket"]
    },
    {
      "goal": "Run multi-agent debate to validate escalation decisions",
      "context": "Use agent_autogen to spawn three agents (support_agent, quality_agent, policy_agent) that debate whether the ticket warrants escalation. Consensus requires 2-of-3 agreement.",
      "constraints": ["Maximum 5 debate rounds", "Consensus threshold: 2-of-3 agents"]
    },
    {
      "goal": "Execute lightweight tool calls for notification dispatch",
      "context": "Use agent_smolagents for simple, low-latency tool-calling tasks: send confirmation emails, update ticket status in CRM, trigger Slack notifications.",
      "constraints": ["Max latency: 2 seconds per call", "Fire-and-forget for non-critical notifications"]
    },
    {
      "goal": "Apply deterministic expert system rules when LLM confidence is low",
      "context": "Use adapt_experta for rule-based fallback. When any LLM agent returns confidence below 0.60, route to the expert system which applies hard-coded business rules: auto-refund below $50, escalate fraud signals, reject out-of-policy requests.",
      "constraints": ["Rules must be auditable and version-controlled", "Zero LLM cost for fallback path"]
    }
  ]
}

Pipeline Diagram¶

graph TD
    A[agent_claude<br/>reasoning + triage] -->|triage report| B[agent_openai<br/>function-calling]
    B -->|account data| C[agent_langchain<br/>chain-of-thought]
    C -->|resolution plan| D[agent_langgraph<br/>stateful workflow]
    D -->|escalation?| E[agent_autogen<br/>multi-agent debate]
    D -->|resolved| G[agent_smolagents<br/>notifications]
    E -->|consensus: escalate| G
    E -->|consensus: resolve| G
    A -->|confidence < 0.60| F[adapt_experta<br/>rule-based fallback]
    B -->|confidence < 0.60| F
    C -->|confidence < 0.60| F
    F -->|deterministic decision| G
    G --> H((Ticket Resolved))

What You Need¶

Tier: Researcher
Components: agent_claude, agent_openai, agent_langchain, agent_langgraph, agent_autogen, agent_smolagents, adapt_experta

Step-by-Step¶

Step 1: Triage with Claude (Reasoning-Heavy Analysis)¶

{
  "component": "agent_claude",
  "operation": "infer",
  "params": {
    "messages": [
      {
        "role": "user",
        "content": "Classify this support ticket and produce a structured triage report.\n\nTicket: 'I was charged twice for my Pro subscription last month. I contacted support three times and nobody helped. I want a refund and I'm considering cancelling.'\n\nOutput JSON with fields: severity (critical/high/medium/low), category (billing/technical/account/general), tone (frustrated/neutral/satisfied), summary, confidence."
      }
    ],
    "max_tokens": 1024
  }
}

Claude returns a structured triage with severity, category, emotional tone, and a confidence score. If confidence is below 0.60, the pipeline routes directly to adapt_experta for deterministic handling.

Step 2: Fetch Account Data with OpenAI (Structured Tool Use)¶

{
  "component": "agent_openai",
  "operation": "infer",
  "params": {
    "messages": [
      {
        "role": "user",
        "content": "Retrieve the billing history and subscription status for customer ID cust_29481. Check for duplicate charges in the last 60 days."
      }
    ],
    "tools": [
      {
        "name": "get_billing_history",
        "description": "Fetch billing transactions for a customer",
        "parameters": {
          "type": "object",
          "properties": {
            "customer_id": {"type": "string"},
            "days_back": {"type": "integer"}
          },
          "required": ["customer_id"]
        }
      },
      {
        "name": "get_subscription_status",
        "description": "Fetch current subscription details",
        "parameters": {
          "type": "object",
          "properties": {
            "customer_id": {"type": "string"}
          },
          "required": ["customer_id"]
        }
      }
    ],
    "max_tokens": 1024
  }
}

OpenAI's function-calling produces structured tool invocations with strict JSON schema adherence. The results feed into the next stage.

Provider Strengths

Claude excels at nuanced reasoning and long-context analysis. OpenAI excels at structured function-calling with strict schema adherence. Matching task to provider avoids forcing one model to do everything poorly.

Step 3: Chain-of-Thought Resolution with LangChain (Tool Retrieval)¶

{
  "component": "agent_langchain",
  "operation": "infer",
  "params": {
    "messages": [
      {
        "role": "user",
        "content": "Given this triage report and account data, reason step-by-step through the resolution options.\n\nTriage: [agent_claude output inserted]\nAccount data: [agent_openai output inserted]\n\nConsider: refund policy, duplicate charge detection, customer retention risk, and prior interaction history. Retrieve relevant policy documents."
      }
    ],
    "tools": ["policy_retriever", "knowledge_base"],
    "max_tokens": 2048
  }
}

LangChain chains together retrieval and reasoning steps, pulling in policy documents and knowledge base articles to ground the resolution plan.

Step 4: Stateful Workflow with LangGraph (Multi-Step Orchestration)¶

{
  "component": "agent_langgraph",
  "operation": "infer",
  "params": {
    "messages": [
      {
        "role": "user",
        "content": "Execute the support ticket workflow. Current state: triage_complete.\n\nResolution plan: [agent_langchain output inserted]\n\nState transitions: triage -> investigation -> resolution -> verification -> closed. If resolution requires escalation, branch to escalation state."
      }
    ],
    "state": {
      "ticket_id": "TKT-29481",
      "current_state": "triage_complete",
      "history": []
    },
    "max_tokens": 2048
  }
}

LangGraph maintains persistent state across workflow transitions. If the resolution plan calls for escalation, it branches to the debate stage rather than auto-resolving.

Why Stateful Matters

Stateless LLM calls lose context between turns. LangGraph persists the full workflow state — ticket history, branching decisions, and rollback points — so the system can resume from any checkpoint without replaying the entire conversation.

Step 5: Multi-Agent Debate with AutoGen (Consensus Through Disagreement)¶

{
  "component": "agent_autogen",
  "operation": "infer",
  "params": {
    "messages": [
      {
        "role": "user",
        "content": "Three agents must debate whether ticket TKT-29481 warrants escalation to a human supervisor.\n\nContext: [full pipeline context inserted]\n\nAgents:\n- support_agent: advocates for customer satisfaction\n- quality_agent: evaluates resolution quality and completeness\n- policy_agent: ensures compliance with company policies\n\nDebate until 2-of-3 consensus or 5 rounds, whichever comes first."
      }
    ],
    "max_rounds": 5,
    "max_tokens": 4096
  }
}

AutoGen spawns three specialised agents that argue from different perspectives. Consensus requires 2-of-3 agreement, preventing any single viewpoint from dominating.

Step 6: Lightweight Notifications with SmolagentS (Low-Latency Tool-Calling)¶

{
  "component": "agent_smolagents",
  "operation": "infer",
  "params": {
    "messages": [
      {
        "role": "user",
        "content": "Execute the following notification tasks for ticket TKT-29481:\n1. Send refund confirmation email to customer\n2. Update CRM ticket status to 'resolved'\n3. Post summary to #support-escalations Slack channel"
      }
    ],
    "tools": ["send_email", "update_crm", "post_slack"],
    "max_tokens": 512
  }
}

SmolagentS handles simple, low-latency tool calls. No complex reasoning needed — just structured dispatch.

Step 7: Expert System Fallback with Experta (Deterministic Rules)¶

{
  "component": "adapt_experta",
  "operation": "run",
  "params": {
    "rules": {
      "auto_refund": "IF duplicate_charge AND amount < 50 THEN refund(amount)",
      "escalate_fraud": "IF fraud_signals > 2 THEN escalate('fraud_team')",
      "reject_out_of_policy": "IF days_since_charge > 90 THEN reject('outside_refund_window')"
    },
    "facts": {
      "duplicate_charge": true,
      "amount": 29.99,
      "fraud_signals": 0,
      "days_since_charge": 12
    }
  }
}

When any upstream LLM agent returns confidence below 0.60, the pipeline short-circuits to adapt_experta. The expert system fires rules deterministically: no hallucination, no token cost, full auditability. In this case, the auto-refund rule fires because the charge is under $50.

(adapt_experta is stateful — in practice you load facts and rules with add_facts / add_rules, then call run to fire the engine and query to read results. The single call above is condensed for illustration.)

When to Fall Back

Expert system fallback is not a failure mode — it is a design choice. For high-stakes decisions (refunds, escalations, account changes), deterministic rules provide auditability and consistency that probabilistic LLM outputs cannot guarantee.

What Happened¶

G6 orchestrated seven components across three intelligence classes:

agent_claude performed deep reasoning to triage the ticket with severity, category, and tone classification
agent_openai executed structured function calls to retrieve account and billing data
agent_langchain built a chain-of-thought resolution plan grounded in policy documents
agent_langgraph managed the stateful workflow, tracking the ticket through state transitions
agent_autogen ran a multi-agent debate to validate the escalation decision via 2-of-3 consensus
agent_smolagents dispatched lightweight notifications (email, CRM, Slack) at low latency
adapt_experta stood ready as a deterministic fallback, applying auditable business rules when LLM confidence dropped below threshold

Each provider handled the task class it was built for. No single LLM was asked to do everything.

Why G6 Over a Bare LLM¶

A capable LLM can handle multi-step tasks and make decisions. G6 adds multi-provider routing that matches each task to the best-suited model, deterministic rule engines for zero-token-cost business logic, and persistent workflow state across turns. Prebuilt templates compose six LLM providers with fallback logic and consensus protocols into a reliable pipeline — one GoalInput JSON triggers the entire workflow with auditable decision trails.