---
title: Record experiment variants in Agent Analytics
description: "Record which A/B test or feature flag variant each agent session ran under, from the AI SDK, OpenTelemetry, the HTTP API, or a warehouse import, and split any Agent Analytics chart by variant."
product: general
lang: en
last_updated: 2026-11-03
token_estimate: 2512
---
# Record experiment variants in Agent Analytics

The `experiments` option records which A/B test or feature flag variant an agent ran under. Pass the assignment your flag client already returned, and every `[Agent]` event in the session carries `[Agent] Experiments` and, for a single experiment, `[Agent] Experiment Variant`. Group any chart by that property to compare quality, cost, or abandonment across the arms of a prompt, model, router, or tool test.

> For AI agents: a documentation index is available at [/docs/llms.txt](/docs/llms.txt). Append `.md` to any page URL for markdown, or send `Accept: text/markdown`.

Use `experiments` for the variant that changes agent behavior. Use `[Agent] Context` for other segmentation dimensions, such as the UI surface, and `agentVersion` (Node) or `agent_version` (Python) for releases that aren't randomized.

The `experiments` option and the helpers on this page require Node `@amplitude/ai` 0.25.0 or later, or Python `amplitude-ai` 1.22.0 or later. Amplitude doesn't assign variants: keep your existing flag or experimentation SDK, such as Amplitude Experiment, Statsig, or LaunchDarkly, and pass its result. For the three shapes an agent experiment usually takes, in one agent loop, refer to [Run an agent experiment](https://amplitude.com/docs/amplitude-ai/agent-analytics/run-an-agent-experiment).

## Pass the assignment with the AI SDK

Set `experiments` on the agent, the session, or a single tracking call. Each entry takes a `key` and a `variant`, plus an optional `provider` (the flag system's name) and `unit`.

#### Node

```typescript
const variant = await flags.getVariant('planner'); // your existing flag client
const agent = ai.agent('support-bot', { userId });

await agent
  .session({ sessionId, experiments: [{ key: 'planner', variant, provider: 'statsig' }] })
  .run(async (s) => {
    // Provider wrappers, tools, and tracking calls in the session all carry the experiment.
  });
```

#### Python

```python
variant = flags.get_variant("planner")  # your existing flag client
agent = ai.agent("support-bot", user_id=user_id)

with agent.session(
    session_id=session_id,
    experiments=[{"key": "planner", "variant": variant, "provider": "statsig"}],
) as s:
    ...  # provider wrappers, tools, and tracking calls in the session all carry the experiment
```

The SDK applies the entries in this order:

- **Agent.** Applies to every session of the agent. Child agents inherit the parent's experiments and can add or override entries.
- **Session.** Overrides an agent entry with the same key.
- **Call.** A single agent or session tracking call overrides both, for that event only.

An override replaces the whole entry, including its provider and unit. An entry keeps the position where its key first appeared.

## Use the helpers for your flag tool

Three helpers build `experiments` entries from the flag client the application already calls. None of them sends an exposure event; the flag tool keeps that job.

- **Amplitude Experiment.** `experimentsFromAmplitude(variants, { keys })` (Node) and `experiments_from_amplitude(variants, keys=)` (Python) turn the result of `fetchV2()`, `evaluateV2()`, `fetch_v2()`, `evaluate_v2()`, or `all()` into entries with `provider: "amplitude"`, skipping fallback variants. `keys` names the experiment flags, so configuration flags in the same result stay out of the charts. Without `keys`, the helper records every flag and logs a warning when more than 10 would apply. Numeric flag values become decimal labels such as `3` or `0.25`.
- **Any OpenFeature provider.** `AmplitudeAIHook` from `@amplitude/ai/openfeature` (Node) or `amplitude_ai.openfeature` (Python) is an [OpenFeature](https://openfeature.dev) hook. Register it once with `OpenFeature.addHooks([hook])` or `openfeature.api.add_hooks([hook])`, and every flag the application evaluates inside a session becomes an experiment on the events that follow, with `provider` set from the OpenFeature provider's name. `flags` limits the hook to the flags you list, `unit` sets the unit, and `variantOf` (Node) or `variant_of` (Python) names the arm for object-valued flags whose provider returns no variant, such as Statsig experiments and dynamic configs.
- **An assignment learned mid-session.** `session.addExperiments(entries)` (Node) and `session.add_experiments(entries)` (Python) record an arm after the session started, for example a router decision. Events tracked from then on carry it, through provider wrappers and tools inside the session and across `runAs()` or `run_as()` delegation. An entry with an existing key replaces that entry.

#### Node

```typescript
import { experimentsFromAmplitude } from '@amplitude/ai';
import { AmplitudeAIHook } from '@amplitude/ai/openfeature';

// Amplitude Experiment: name the experiment flags.
const variants = await experiment.fetchV2({ user_id: userId });
const session = agent.session({ sessionId, experiments: experimentsFromAmplitude(variants, { keys: ['planner'] }) });

// OpenFeature: register once at startup.
OpenFeature.addHooks([new AmplitudeAIHook({ flags: ['planner'] })]);

// Learned mid-session.
session.addExperiments([{ key: 'router', variant: 'fast', unit: 'turn' }]);
```

#### Python

```python
from amplitude_ai import experiments_from_amplitude
from amplitude_ai.openfeature import AmplitudeAIHook

# Amplitude Experiment: name the experiment flags.
variants = experiment.fetch_v2(User(user_id=user_id))
session = agent.session(session_id=session_id, experiments=experiments_from_amplitude(variants, keys=["planner"]))

# OpenFeature: register once at startup.
openfeature.api.add_hooks([AmplitudeAIHook(flags=["planner"])])

# Learned mid-session.
session.add_experiments([{"key": "router", "variant": "fast", "unit": "turn"}])
```

For a flag tool without a helper, build the entry from the value it returned: `{ key: 'planner', variant, provider: 'launchdarkly' }`. `experimentFromEvaluation()` (Node) and `experiment_from_evaluation()` (Python) are the mapping behind the OpenFeature hook, for an SDK that returns evaluation details in a different shape.

## Properties on each event

| Property | Type | When it's set |
| --- | --- | --- |
| `[Agent] Experiments` | array of strings | Every event with at least one valid experiment. Each value is `key:variant`, for example `planner:v2`. |
| `[Agent] Experiment Key` | string | Exactly one experiment applies. |
| `[Agent] Experiment Variant` | string | Exactly one experiment applies. |
| `[Agent] Experiment Unit` | string | Exactly one experiment applies: `agent_session`, `turn`, or `span`. |
| `[Agent] Experiment Provider` | string | Exactly one experiment applies and it has a valid provider. |

With two or more experiments, the event carries only the array. Every privacy mode sends these properties.

## Choose the unit

`unit` records what your flag client randomized, so analysis compares like with like:

- **`agent_session`** (default). One assignment per session, such as a prompt or model chosen when the conversation starts.
- **`turn`**. A new assignment per user message, such as a router evaluated on each turn.
- **`span`**. A new assignment per operation inside a turn, such as a tool or retrieval strategy. OpenTelemetry evaluations default to `span`.

## Validation and limits

The SDKs, the OpenTelemetry receiver, and the warehouse import templates apply the same rules. Invalid input never throws an error.

- `key`, `variant`, and `provider` are 1 to 256 characters after trimming ASCII whitespace.
- A `key` can't contain `:`. A `variant` can, for example `openai:gpt-4o`.
- Amplitude drops an entry without a valid key or variant. The SDKs log each dropped entry at debug level.
- An unknown `unit` falls back to the default, and an invalid `provider` is left out.
- At most 10 experiments apply to an event, counted after the agent, session, and call entries merge.

Keys, variants, and providers aren't redacted. Use flag and variant names, never user data.

## Send assignments from OpenTelemetry

Agent Analytics reads OpenTelemetry feature flag evaluations from GenAI spans. Each `feature_flag.evaluation` span event, and the span's own `feature_flag.*` attributes, become one experiment on every event the span produces. For the attribute names, refer to [Feature flag attributes](https://amplitude.com/docs/amplitude-ai/agent-analytics/otlp-reference#feature-flag-attributes) in the OTLP reference.

## Send assignments from the HTTP API or a warehouse import

Events you send yourself carry the same five properties. Set `[Agent] Experiments` as an array of `key:variant` strings, and the single-experiment properties only when one experiment applies. Refer to [Send agent events without the AI SDK](https://amplitude.com/docs/amplitude-ai/agent-analytics/instrument-http-api).

The warehouse import templates read optional `experiment_key` and `experiment_variant` columns, one experiment per row, with unit `agent_session`. Refer to [Import agent conversations from your data warehouse](https://amplitude.com/docs/amplitude-ai/agent-analytics/integrations/warehouse-import#what-carries-over).

## Split a chart by variant

1. Open any chart built on `[Agent]` events, such as session cost or abandonment.
2. Add a group-by on `[Agent] Experiments`. Each `key:variant` value of the array becomes its own group, so the chart splits by arm whether one experiment ran or several. When exactly one experiment runs, `[Agent] Experiment Variant` gives the same split with shorter labels.
3. Filter on `[Agent] Experiments` contains `planner:` to keep one test in view, or on `[Agent] Experiment Key` when only one experiment ran.

## Migrate from `[Agent] Context`

Earlier guidance put the variant in `context`, for example `context: { experiment_variant: 'treatment' }`. Those values keep working in `[Agent] Context` filters. For new code, pass the assignment as `experiments` and remove it from `context`, so each event records the assignment once and charts can group by a dedicated property.

