Evals With Memory Guide
Three concrete, working ways to run **Mastra evals** against an agent that has **memory** turned on — including observational-memory in thread scope (the configuration that triggers ObservationalMemory (scope: 'thread') requires a threadId, but none was found in RequestContext or MessageList.). Everything in this example uses Mastra evals primitives (runEvals, createScorer, Dataset.startExperiment). No custom evaluation harness. The agent in every script uses @mastra/memory + @mastra/libsql for storage and observational memory in thread scope. Each script writes to a fresh temp DB and cleans up after itself. A deterministic mock model is used so no API key is required and runs are reproducible in CI.
When to use Evals With Memory
Three concrete, working ways to run **Mastra evals** against an agent that has **memory** turned on — including observational-memory in thread scope (the configuration that triggers ObservationalMemory (scope: 'thread') requires a threadId, but none was found in RequestContext or MessageList.). Everything in this example uses Mastra evals primitives (runEvals, createScorer, Dataset.startExperiment). No custom evaluation harness. The agent in every script uses @mastra/memory + @mastra/libsql for storage and observational memory in thread scope. Each script writes to a fresh temp DB and cleans up after itself. A deterministic mock model is used so no API key is required and runs are reproducible in CI.
How to use Evals With Memory
Evals With Memory is a single agent agent built on the Mastra framework. Set it up from the source repository, configure your model credentials, and invoke it for tasks that match its description. Review the safety profile below before running it against production data or systems.
Safety profile
Autonomy
Semi-autonomous
Sandbox-aware
No declared sandbox guidance
Network access
Unspecified
Filesystem access
Unspecified