Record experiment variants in Agent Analytics

The experiments option records which A/B test or feature flag variant an agent ran under. Pass the assignment your flag client already returned, and every [Agent] event in the session carries [Agent] Experiments and, for a single experiment, [Agent] Experiment Variant. Group any chart by that property to compare quality, cost, or abandonment across the arms of a prompt, model, router, or tool test.

Use experiments for the variant that changes agent behavior. Use [Agent] Context for other segmentation dimensions, such as the UI surface, and agentVersion (Node) or agent_version (Python) for releases that aren't randomized.

The experiments option requires a Node @amplitude/ai release later than 0.24.1 or a Python amplitude-ai release later than 1.20.1. Amplitude doesn't assign variants: keep your existing flag or experimentation SDK, such as Amplitude Experiment, Statsig, or LaunchDarkly, and pass its result.

Pass the assignment with the AI SDK

Set experiments on the agent, the session, or a single tracking call. Each entry takes a key and a variant, plus an optional provider (the flag system's name) and unit.

const variant = await flags.getVariant('planner'); // your existing flag client
const agent = ai.agent('support-bot', { userId });

await agent
  .session({ sessionId, experiments: [{ key: 'planner', variant, provider: 'statsig' }] })
  .run(async (s) => {
    // Provider wrappers, tools, and tracking calls in the session all carry the experiment.
  });

The SDK applies the entries in this order:

• Agent. Applies to every session of the agent. Child agents inherit the parent's experiments and can add or override entries.
• Session. Overrides an agent entry with the same key.
• Call. A single agent or session tracking call overrides both, for that event only.

An override replaces the whole entry, including its provider and unit. An entry keeps the position where its key first appeared.

Properties on each event

With two or more experiments, the event carries only the array. Every privacy mode sends these properties.

Choose the unit

unit records what your flag client randomized, so analysis compares like with like:

• agent_session (default). One assignment per session, such as a prompt or model chosen when the conversation starts.
• turn. A new assignment per user message, such as a router evaluated on each turn.
• span. A new assignment per operation inside a turn, such as a tool or retrieval strategy. OpenTelemetry evaluations default to span.

Validation and limits

The SDKs, the OpenTelemetry receiver, and the warehouse import templates apply the same rules. Invalid input never throws an error.

• key, variant, and provider are 1 to 256 characters after trimming ASCII whitespace.
• A key can't contain :. A variant can, for example openai:gpt-4o.
• Amplitude drops an entry without a valid key or variant. The SDKs log each dropped entry at debug level.
• An unknown unit falls back to the default, and an invalid provider is left out.
• At most 10 experiments apply to an event, counted after the agent, session, and call entries merge.

Keys, variants, and providers aren't redacted. Use flag and variant names, never user data.

Send assignments from OpenTelemetry

Agent Analytics reads OpenTelemetry feature flag evaluations from GenAI spans. Each feature_flag.evaluation span event, and the span's own feature_flag.* attributes, become one experiment on every event the span produces. For the attribute names, refer to Feature flag attributes in the OTLP reference.

Send assignments from the HTTP API or a warehouse import

Events you send yourself carry the same five properties. Set [Agent] Experiments as an array of key:variant strings, and the single-experiment properties only when one experiment applies. Refer to Send agent events without the AI SDK.

The warehouse import templates read optional experiment_key and experiment_variant columns, one experiment per row, with unit agent_session. Refer to Import agent conversations from your data warehouse.

Split a chart by variant

1. Open any chart built on [Agent] events, such as session cost or abandonment.
2. Add a group-by on [Agent] Experiment Variant. For sessions with several experiments, group by [Agent] Experiments instead: each value of the array becomes its own group.
3. Filter on [Agent] Experiment Key to keep one test in view.

Migrate from [Agent] Context

Earlier guidance put the variant in context, for example context: { experiment_variant: 'treatment' }. Those values keep working in [Agent] Context filters. For new code, pass the assignment as experiments and remove it from context, so each event records the assignment once and charts can group by a dedicated property.