Data Analysis Pipeline¶

Train an ML classifier with genetic hyperparameter optimisation, ground the methodology against established ML practices, and formally evaluate results — all orchestrated through G6's component pipeline.

Illustrative end-to-end example

A worked illustration of how these components compose — every component and operation named below is real, and these steps all run offline against built-in defaults. It is a worked example, not a copy-paste script (some param shapes are simplified for readability).

GoalInput¶

{
  "goal": "Build a customer churn predictor with genetic hyperparameter optimisation and formally grounded methodology",
  "context": "Telco customer dataset with 7,043 records, 21 features (demographics, account info, services). Binary classification: churn vs. retain. The model will drive proactive retention campaigns — precision matters more than recall to avoid wasting outreach budget.",
  "constraints": [
    "F1 score must exceed 0.90 on held-out test set",
    "Genetic algorithm must converge within 50 generations",
    "Methodology must be grounded against ai_ml knowledge base with confidence >= 0.70",
    "All evaluation metrics computed via pure Python (no sklearn.metrics dependency)"
  ],
  "resource_bounds": {
    "max_execution_seconds": 600,
    "max_tokens_per_hour": 150000,
    "max_disk_bytes": 268435456
  },
  "checkpoints": [
    {
      "name": "f1_quality_gate",
      "predicate": "metric_above",
      "params": {"metric": "f1", "threshold": 0.90},
      "description": "Log warning if F1 drops below 0.90 during training"
    }
  ],
  "guardrails": [
    {
      "name": "token_budget",
      "predicate": "resource_limit",
      "params": {"max_tokens_per_hour": 150000},
      "message": "Halt if hourly token budget exceeded"
    }
  ],
  "subtasks": [
    {
      "goal": "Ingest and profile the customer dataset",
      "context": "Use adapt_pandas to load CSV, compute descriptive statistics, correlation matrix, and identify high-cardinality categoricals for encoding.",
      "constraints": [
        "Drop columns with >30% missing values",
        "One-hot encode categoricals with cardinality < 10",
        "Output numeric-only DataFrame"
      ],
      "resource_bounds": {
        "max_execution_seconds": 60
      }
    },
    {
      "goal": "Engineer predictive features",
      "context": "Use adapt_pandas advanced operations: create tenure bins, compute monthly charge ratios, flag contract type interactions. Use adapt_pandas profiling to identify feature importance candidates.",
      "constraints": [
        "Minimum 5 engineered features",
        "No data leakage from target variable"
      ]
    },
    {
      "goal": "Train classifier with genetic hyperparameter optimisation",
      "context": "Use adapt_sklearn for random forest training. Use adapt_pygad for genetic search over hyperparameter space: n_estimators (50-500), max_depth (3-20), min_samples_split (2-20). Fitness function: macro-averaged F1 on 5-fold cross-validation.",
      "constraints": [
        "Population size: 20",
        "Number of generations: 50",
        "Crossover type: single_point",
        "Mutation probability: 0.1",
        "Final F1 must exceed 0.90"
      ],
      "resource_bounds": {
        "max_execution_seconds": 300
      }
    },
    {
      "goal": "Optimise decision threshold via scipy",
      "context": "Use adapt_optimisation with scipy minimize_scalar to find the classification threshold that maximises F1 on validation set. Search range [0.3, 0.7].",
      "constraints": [
        "Method: scipy",
        "Objective: maximise F1",
        "Bounds: [0.3, 0.7]"
      ]
    },
    {
      "goal": "Evaluate and ground the final model",
      "context": "Use align_evals for multi-metric evaluation (accuracy, precision, recall, F1, MAE). Use grounding (domain: ai_ml) to check the methodology against retrieved ML best-practice sources.",
      "constraints": [
        "Compute all 5 metrics",
        "Grounding confidence for methodology must exceed 0.70",
        "Report per-class precision and recall"
      ]
    }
  ]
}

Pipeline Diagram¶

graph TD
    A[adapt_pandas<br/>ingest + profile] -->|clean DataFrame| B[adapt_pandas<br/>feature engineering]
    B -->|features| C[adapt_sklearn<br/>random forest]
    C <-->|fitness function| D[adapt_pygad<br/>genetic search]
    D -->|best hyperparams| C
    C -->|trained model| E[adapt_optimisation<br/>threshold tuning]
    E -->|optimised model| F[align_evals<br/>multi-metric eval]
    F -->|methodology| G[grounding<br/>ai_ml domain]
    G --> H((Verified Model))

What You Need¶

Tier: Researcher
Components: adapt_pandas, adapt_sklearn, adapt_pygad, adapt_optimisation, align_evals, grounding

Step-by-Step¶

Step 1: Ingest and Profile¶

{
  "component": "adapt_pandas",
  "operation": "load",
  "params": {
    "data": [{"tenure": 12, "MonthlyCharges": 29.85, "Churn": "No"}, "..."]
  }
}

Then profile:

{
  "component": "adapt_pandas",
  "operation": "describe",
  "params": {}
}

{
  "component": "adapt_pandas",
  "operation": "correlate",
  "params": {}
}

Persistent Storage

adapt_pandas backs data to SQLite (PANDAS_DB_PATH env var, default ~/.pandas_mcp/pandas.db). DataFrames persist across invocations — load once, query many times.

Step 2: Feature Engineering¶

{
  "component": "adapt_pandas",
  "operation": "transform",
  "params": {
    "operations": [
      {"type": "bin", "column": "tenure", "bins": [0, 12, 24, 48, 72]},
      {"type": "ratio", "numerator": "MonthlyCharges", "denominator": "tenure"}
    ]
  }
}

Step 3: Train with Genetic Optimisation¶

First, set up the base model:

{
  "component": "adapt_sklearn",
  "operation": "train",
  "params": {
    "model_type": "random_forest",
    "task": "classification",
    "model_config_data": {
      "n_estimators": 100,
      "max_depth": 10
    }
  }
}

Then optimise hyperparameters with a genetic algorithm:

{
  "component": "adapt_pygad",
  "operation": "solve",
  "params": {
    "gene_space": [
      {"low": 50, "high": 500},
      {"low": 3, "high": 20},
      {"low": 2, "high": 20}
    ],
    "num_generations": 50,
    "sol_per_pop": 20,
    "crossover_type": "single_point",
    "mutation_percent_genes": 10
  }
}

Why Genetic Optimisation?

Grid search explores a fixed grid. Bayesian optimisation builds a surrogate model. Genetic algorithms explore the space via evolution — crossover and mutation discover hyperparameter combinations that neither grid nor Bayesian would reach in bounded time. G6 uses PyGAD for this.

Step 4: Optimise Decision Threshold¶

{
  "component": "adapt_optimisation",
  "operation": "solve",
  "params": {
    "function": "neg_f1_score",
    "method": "scipy",
    "bounds": [0.3, 0.7]
  }
}

The adapt_optimisation component supports scipy (gradient-based) and grid (exhaustive) methods, with built-in test functions (sphere, rosenbrock, rastrigin, ackley).

Step 5: Evaluate and Ground¶

Multi-metric evaluation:

{
  "component": "align_evals",
  "operation": "evaluate",
  "params": {
    "predictions": [1, 0, 1, 1, 0],
    "ground_truth": [1, 0, 0, 1, 0],
    "metrics": ["accuracy", "precision", "recall", "f1", "mae"]
  }
}

Ground the methodology:

{
  "component": "grounding",
  "operation": "ground",
  "params": {
    "query": "Random forest with genetic hyperparameter optimisation and threshold tuning is an appropriate methodology for binary churn classification",
    "domain": "ai_ml",
    "top_k": 5
  }
}

The grounding confidence here is the mean retrieval similarity between the methodology statement and the ai_ml seed facts — it signals how textually close your methodology is to known ML practice, not that the methodology is correct. Use the 0.70 threshold as a "this looks unfamiliar, review it" trigger, not a correctness stamp.

Pure Python Metrics

align_evals computes accuracy, precision, recall, F1, exact match, MSE, and MAE in pure Python with macro-averaging. No sklearn.metrics dependency required.

What Happened¶

G6 orchestrated six components in a data science pipeline:

adapt_pandas ingested the dataset and persisted it to SQLite
adapt_pandas engineered predictive features (tenure bins, charge ratios)
adapt_sklearn trained a random forest classifier
adapt_pygad genetically optimised hyperparameters over 50 generations
adapt_optimisation tuned the decision threshold via scipy
align_evals evaluated with 5 metrics; grounding scored the methodology against the corpus

Why G6 Over a Bare LLM¶

A capable LLM can generate ML code and explain statistical methods. G6 adds actual execution — genetic optimisation runs as real code, evaluation metrics are computed (not estimated), and methodology is grounded against a curated knowledge base. Prebuilt pipeline templates compose training, optimisation, evaluation, and grounding into a reproducible workflow triggered by one GoalInput JSON.