Business Document Classifier¶

Turn labeled examples into a tested, self-improving classification harness with quality evidence and cost reporting.

Overview¶

The Business Document Classifier is G6's reference workflow — a complete demonstration of the self-training improvement loop. It takes your labeled data and produces:

1. A classification harness tuned to your categories
2. Held-out evaluation proving quality
3. Failure taxonomy explaining what went wrong
4. Iterative improvements until a quality target is met
5. A markdown report with metrics, cost, and safety evidence

Architecture¶

graph TD
    A[CSV Data] --> B[Load & Split]
    B --> C[Train Set]
    B --> D[Test Set]
    C --> E[Build Harness]
    E --> F[Classify Test Set]
    D --> F
    F --> G{Target Met?}
    G -->|Yes| H[Generate Report]
    G -->|No| I[Diagnose Failures]
    I --> J[Improve Harness]
    J --> F
    H --> K[workflow_report.md]

Quick start¶

cd workflows/business_classifier
python run_workflow.py

Output: workflow_report.md with full quality/cost/safety evidence.

Data format¶

Your CSV must have these columns:

Minimum 20 rows recommended. The workflow splits 80/20 into train/test automatically.

Configuration¶

python run_workflow.py \
  --data my_data.csv \
  --target-accuracy 0.85 \
  --max-iterations 5 \
  --report-path results/report.md

How it works¶

Step 1: Build harness¶

The harness is a keyword-based classifier that maps signal words to categories. Initial keywords are seeded from common domain patterns.

Step 2: Evaluate¶

The harness classifies each test example. Metrics computed: - Accuracy — overall correct rate - Precision — per-category, how many predicted X were actually X - Recall — per-category, how many actual X were found - F1 — harmonic mean of precision and recall

Step 3: Diagnose¶

Failed predictions are classified into root causes:

Step 4: Improve¶

Based on the diagnosis, the harness is improved: - New keywords extracted from misclassified examples - Training frequency analysis adds high-signal words - Each improvement maps to a specific failure category

Step 5: Report¶

A markdown report documents everything: metrics, failures, improvements, cost, and safety.

Component mapping¶

In demo mode, the workflow uses pure Python. In production (via MCP/REST), G6 components handle each step:

Using via MCP (Claude Code)¶

See the full MCP First Workflow Guide for step-by-step instructions.

Quick version:

You: "I have a CSV of support tickets. Help me build a classifier."
Claude Code: [calls guide_plan_workflow] → plans the multi-step workflow
Claude Code: [calls run_data_pipeline] → loads and profiles data
Claude Code: [calls invoke_component align_evals] → evaluates baseline
Claude Code: [calls self_training_audit] → diagnoses quality
Claude Code: [calls nav_recommend] → suggests improvements

Using via REST API¶

See the full REST Workflow Guide.

Quick version:

# 1. Submit goal
curl -X POST http://localhost:8010/api/v1/goals/decompose \
  -H "Content-Type: application/json" \
  -d '{"goal": "Classify support tickets", "context": "5 categories"}'

# 2. Run evaluation
curl -X POST http://localhost:8010/api/v1/invoke \
  -H "Content-Type: application/json" \
  -d '{"component": "align_evals", "op": "infer", "params": {...}}'

Customization¶

Adding categories¶

Edit your CSV to include new category labels. The harness adapts automatically.

Changing the improvement strategy¶

The improve_harness() function in _engine.py can be extended: - Add bigram matching instead of single keywords - Weight keywords by TF-IDF scores - Integrate LLM-based classification (use --live flag)

Setting quality targets¶

Different use cases need different bars: - Routing/triage (non-critical): 70-80% accuracy is useful - Customer-facing (moderate): 85-90% accuracy needed - Regulated/financial (high stakes): 95%+ with human review

Limitations¶

• Demo mode: Uses keyword matching, not LLM reasoning. Production mode (via MCP/REST) uses real LLM backends for higher accuracy.
• Cold start: First run builds from scratch. Subsequent runs benefit from cached improvements (see Cold Start Explained).
• Single-label: Currently supports one category per document. Multi-label classification requires workflow extension.