Skip to content

Align Verbsamp

Align VerbSamp — mvp.align_verbsamp

Cluster: Safety & Alignment | Type: component | MCP Tools: 31

Overview

Verbal alignment signal sampler that detects alignment-relevant linguistic patterns in free text — uncertainty, refusal, agreement, disagreement, hedging, safety, and confidence — using a curated regex library. When use_semantic=True, it delegates to a sentence-transformer cosine similarity engine (with negation detection) before falling back to regex for signals below threshold. Produces per-signal SignalSample records and an overall alignment score in [0, 1].

Production caveat

align_verbsamp is best treated as a lightweight monitoring and review signal, not as an adversarially robust safety gate. It can help flag refusals, uncertainty, safety language, and drift-like changes in model output, but its detections are heuristic and can miss paraphrased or deliberately evasive text. overall_alignment_score is not safety, truth, legal, clinical, regulatory, or compliance proof; only refusal reduces the canonical score, while uncertainty, disagreement, and hedging are reported without lowering it. For hard blocking or regulated workflows, combine it with enforcement-oriented components such as align_csf, align_specs, and align_evals, and validate false-positive and false-negative rates on the target workload.

Canonical MCP and context semantics

The canonical MCP surface is components/mvp/align_verbsamp/align_verbsamp_mcp/server.py, which exposes the package verbsamp_* tools plus bounded verbsamp_capabilities and verbsamp_source_health discovery. The older skill/mcp/server.py surface is legacy/local skill tooling. context is evidence-only and is surfaced as context_used_for_scoring=False; it does not affect signal detection or scoring.

When to use:

  • Auditing LLM responses for alignment signals before logging them as training data
  • Flagging refusal or safety signals in agent output for human review
  • Scoring conversational alignment in real-time to steer prompt or strategy adjustments

Example:

from mvp.align_verbsamp import AlignVerbSampBlock, VerbSampInput

block = AlignVerbSampBlock(name="verbsamp")
result = block.infer(VerbSampInput(text="I'm not sure, but I think this might work."))
# result.value.overall_alignment_score → float; signals include "uncertainty" detected

Works well with: align_coconstructive, align_csf, align_evals

Public API

SignalSample(BaseModel)

Field Type Default
signal_type str required
detected bool required
confidence float Field(ge=0.0, le=1.0)
excerpt str ''
explanation str ''

VerbSampInput(BaseModel)

Field Type Default
text str required
signal_types list[str] Field(default_factory=list)
context str ''
use_semantic bool True

Methods:

text_not_empty(v: str) -> str

signal_types_supported(v: list[str]) -> list[str]

VerbSampOutput(BaseModel)

Field Type Default
signals list[SignalSample] Field(default_factory=list)
overall_alignment_score float Field(default=0.0, ge=0.0, le=1.0)
n_signals_detected int Field(default=0, ge=0)
summary str ''
degraded bool False
degradation_reason str ''
agentic_evidence dict[str, Any] Field(default_factory=dict)
completion_state str 'qualified-draft'
warning_card dict[str, Any] Field(default_factory=dict)
evidence dict[str, Any] Field(default_factory=dict)
request_id str ''
task_id str ''
run_id str ''
detector_backend str 'unknown'

AlignVerbSampBlock(AIBlock[VerbSampInput, VerbSampOutput, None])

Verbal alignment signal sampler.

Field Type Default
name str 'align_verbsamp'
resource_bounds ResourceBounds \| None None
usage ResourceUsage field(default_factory=ResourceUsage)
planner Any \| None None

Methods:

infer(data: VerbSampInput) -> Result[VerbSampOutput]

verify_alignment(text: str, min_score: float = 0.4, forbidden_signals: list[str] | None = None, required_signals: list[str] | None = None) -> Result[dict[str, Any]]

Verify text meets alignment signal constraints.

MCPVerbSampInput(BaseModel)

Field Type Default
op VerbSampMCPOp required
text str \| None None
texts list[str] Field(default_factory=list)
signal_types list[str] Field(default_factory=list)
context str \| None None
signal_id str \| None None
signal_type str \| None None
signal_value float \| None None
signal_context str \| None None
text_id str \| None None
text_label str \| None None
session_id str \| None None
session_name str \| None None
pattern_name str \| None None
pattern_regex str \| None None
pattern_signal_type str \| None None
calibration_signal_type str \| None None
threshold float \| None None
items list[dict[str, Any]] Field(default_factory=list)
content str \| None None
query str \| None None
limit int 20
metadata dict[str, Any] Field(default_factory=dict)
request_id str ''
task_id str ''
run_id str ''

Methods:

signal_types_supported(v: list[str]) -> list[str]

scalar_signal_type_supported(v: str | None) -> str | None

limit_non_negative(v: int) -> int

MCPVerbSampOutput(BaseModel)

Field Type Default
op str required
success bool required
signals list[dict[str, Any]] Field(default_factory=list)
signal_id str \| None None
signal dict[str, Any] \| None None
text_id str \| None None
session_id str \| None None
sessions list[dict[str, Any]] Field(default_factory=list)
patterns list[dict[str, Any]] Field(default_factory=list)
pattern_id str \| None None
calibration dict[str, Any] Field(default_factory=dict)
overall_score float 0.0
n_signals_detected int 0
analysis dict[str, Any] Field(default_factory=dict)
analyses list[dict[str, Any]] Field(default_factory=list)
explanation str ''
classification str ''
count int 0
stats dict[str, Any] Field(default_factory=dict)
message str ''
error str ''
degraded bool False
degradation_reason str ''
completion_state str 'qualified-draft'
warning_card dict[str, Any] Field(default_factory=dict)
evidence dict[str, Any] Field(default_factory=dict)
request_id str ''
task_id str ''
run_id str ''
detector_backend str 'unknown'
detector_mode str 'unknown'

Methods:

model_post_init(__context: Any) -> None

AlignVerbSampMCPBlock(AIBlock[MCPVerbSampInput, MCPVerbSampOutput, dict])

Field Type Default
name str 'align_verbsamp_mcp'
state dict \| None None
db_path str ':memory:'
resource_bounds ResourceBounds \| None None
usage ResourceUsage field(default_factory=ResourceUsage)

Methods:

infer(inp: MCPVerbSampInput) -> Result[MCPVerbSampOutput]

MCP Tools

Operation Source
ops align_verbsamp_mcp
help align_verbsamp_mcp
sample align_verbsamp_mcp
detect align_verbsamp_mcp
classify align_verbsamp_mcp
score align_verbsamp_mcp
explain align_verbsamp_mcp
store_signal align_verbsamp_mcp
get_signal align_verbsamp_mcp
list_signals align_verbsamp_mcp
delete_signal align_verbsamp_mcp
store_text align_verbsamp_mcp
analyze_text align_verbsamp_mcp
create_session align_verbsamp_mcp
load_session align_verbsamp_mcp
list_sessions align_verbsamp_mcp
delete_session align_verbsamp_mcp
add_pattern align_verbsamp_mcp
get_pattern align_verbsamp_mcp
list_patterns align_verbsamp_mcp
calibrate align_verbsamp_mcp
get_calibration align_verbsamp_mcp
list_calibrations align_verbsamp_mcp
batch_analyze align_verbsamp_mcp
compare_signals align_verbsamp_mcp
export_analysis align_verbsamp_mcp
import_corpus align_verbsamp_mcp
stats align_verbsamp_mcp
list_applied_patterns align_verbsamp_mcp
capabilities align_verbsamp_mcp
source_health align_verbsamp_mcp