Invarra Blog
Field notes on AI assurance.
Short research notes on invariance, semantic evaluation, and why reliable AI systems need to keep the right behavior when language changes shape.
Correct Once Is Not Enough
A correct answer to one prompt does not prove that a model tracked the underlying target. The stronger question is what happens when meaning stays fixed and representation changes.
Read article
Latest notes
Research notes
The Row Is Often The Wrong Unit
Semantic evaluation often starts with a row. Canonical semantic units give evaluators a better unit of analysis when the same meaning has several surface forms.
Disagreement Is Data
When valid representations of the same underlying case produce different outcomes, disagreement may not be noise. It may be the measurement signal.
Valid Variation Needs A Contract
Semantic-preservation contracts make variation interpretable by stating what must stay fixed, what may change, and how validity is checked.
The Right Null Hypothesis For Indirect Observation
When a target cannot be observed directly, evaluators should assume observed behavior may be representation-sensitive until valid variation supports a stronger claim.
Semantic Brittleness Should Be Attributable
A useful evaluation should not only say that a semantic system is brittle. It should help locate where the brittleness enters the measurement stack.