Long-Horizon AI Evaluation
Project Overview
AstraEthica’s Long-Horizon AI Evaluation project develops practical methods for studying how AI behavior changes across sustained interaction and changing operating conditions. The work focuses on behavioral trajectories in persistent and agentic AI systems, including how trust, memory, permissions, authority, and human oversight shift as context accumulates.
The central question is not only whether a system fails, but when accumulated interactions begin changing the conditions of future interactions. At that threshold, a behavioral pattern can become self-reinforcing, persistent, propagating, and harder to reverse. Small shifts in language, scope, reliance, or delegated authority can then develop into larger safety, security, and reliability failures.
The project brings together scenario design, operating conditions, trajectory analysis, and applied evaluation tools to determine whether important boundaries, safeguards, and forms of human oversight remain stable over time.
The Trajectory Threshold
The trajectory threshold marks a critical transition: the point at which accumulated interactions begin changing the conditions governing future behavior.
One tree is not a forest. In the same way, one interaction does not create a consequential trajectory. A forest emerges as individual trees form a connected system; a trajectory becomes a condition when earlier interactions begin shaping what happens next.
Interaction → Pattern → Trajectory → Condition
Self-reinforcing · Persistent · Propagating · Harder to reverse
The threshold is crossed when earlier interactions begin influencing how later interactions are interpreted, constrained, or acted upon. At that point, the trajectory is no longer merely a record of prior behavior. It has become part of the operating environment governing what happens next.
Behavior Under Conditions
The Behavior Under Conditions series provides the conceptual foundation for the project. It moves from model behavior inside environments, to agent behavior under delegated action, to the point at which accumulated interactions begin reshaping the conditions of future interaction.
Models Inside Environments
Behavior Under Conditions I · BUC-001 · 18 June 2026
How narrative, incentives, ambiguity, and time shape the behavior of models inside environments.
Agents Inside Environments
Behavior Under Conditions II · BUC-002 · 10 July 2026
How ordinary conditions, delegated action, ambiguity, and time shape the behavior of agents inside environments.
When Trajectories Become Conditions
Behavior Under Conditions III · BUC-003 · 20 July 2026
How accumulated human–AI interactions become trajectories that reshape future behavior, safeguards, and recoverability.
Methods and Evaluation Materials
Practical frameworks, operating guides, and evaluation tools developed through the project.
Beyond One-Shot Red Teaming
Long-horizon failure testing for conversational AI systems
Framework Series • FRM-001
Practitioner Field Guide · PDF · Version 1.2 · June 2026
A Field Manual for Long-Horizon Failure Testing in Conversational AI
Scenario Cards
Personas, drift families, and probes for long-horizon failure testing
Companion Series • CAT-002
Methods Companion • PDF • Version 1.0 • June 2026
A field-ready deck of eight personas, drift families, and compressed probes for evaluating long-horizon interaction risks in conversational AI systems.
Companion to Beyond One-Shot Red Teaming and Operating Conditions.
Operating Conditions
Action, persistence, and proxy envelopes for long-horizon testing
Companion Series • CAT-003
Methods Companion • PDF • Version 1.0 • July 2026
An operating guide that names the action, persistence, and proxy envelopes for long‑horizon tests and introduces the Drift Trace, a one‑page analysis sheet for reading scenario trajectories.
Companion to Beyond One-Shot Red Teaming and Scenario Cards.
In Development
The next phase of the project extends the existing methods into complete evaluation workflows, applied risk kits, and design-facing guidance.
Youth-Facing Conversational AI Risk Kit
A structured evaluation toolkit for testing how youth-facing chatbots behave across long, gradually escalating conversations. Includes failure modes, test scripts, a scoring rubric, an evaluation worksheet, and an action checklist.
In development
Design Notes for Sustained Youth–Chatbot Interaction
A design-facing companion translating long-conversation risks into interaction patterns, design directions, workshop prompts, and practical principles for safer youth-facing systems.
In development
Run, Trace, and Rate
An operational guide for conducting a long-horizon evaluation, documenting the behavioral trajectory, and interpreting the resulting safety, security, and reliability findings.
In development
Explore AstraEthica
Frameworks
Evaluation methods, operating guides, and practical tools developed through AstraEthica’s research.
Foundations
Plain-language guides for building a foundational understanding of AI in everyday life.
Field Notes
Essays, visual models, and research notes documenting emerging patterns in human-AI interaction.