Verify your Agent Analytics instrumentation

Verify your instrumentation by sending one real interaction and checking nine gates on the events it produces in Live Events. The gates confirm that events arrive, carry model, token, cost, and latency data, use your real session, agent, and user IDs, and include message content when your privacy mode sends it. Run this check after you instrument with any path, and again before rollout with the production checklist.

Check the nine gates

Send one real interaction from your instrumented code, open the project's Live Events stream, and confirm:

1. Events arrive at all.
2. Model Name and Provider are populated.
3. Token counts are populated.
4. Cost USD is populated.
5. Latency is populated.
6. Session ID is your real ID, not an auto-generated placeholder.
7. Agent ID is set per component.
8. User ID comes from your auth layer.
9. Message content is present on user messages, when your privacy mode sends content.

If you instrumented with your AI coding agent, it writes a verification test as part of instrumentation. Gates 6 through 8 intentionally fail on placeholder IDs, so replace placeholders with real IDs before rollout.

Gate 9 is the one that fails silently everywhere else. The SDK drops an empty message string instead of sending it, so an agent wired to a variable that isn't populated yet clears gates 1 through 8 and still delivers nothing to read. The thread view stays empty, and Amplitude can't score the content-based signals. The gate doesn't apply in metadata_only or customer_enriched, which strip content on purpose.

For OpenTelemetry, the key checks are gates 6 through 8. Refer to Verify on the OpenTelemetry page.

Check the AI SDK before you deploy

If you use the AI SDK, run the doctor to validate environment variables, installed dependencies, and the event-pipeline connection:

npx amplitude-ai doctor

The doctor reads AMPLITUDE_AI_API_KEY by default. When your app names the key differently, point the doctor at the right variable with -key-env:

npx amplitude-ai doctor -key-env MY_KEY_NAME

If you pass -key-env without a value, the doctor exits with a usage message instead of falling back to the default.

Get a fill-rate report with summary()

Before deploying, use MockAmplitudeAI.summary() to get a fill-rate report of all captured events. It checks the verification gates and flags gaps before data reaches Amplitude.

import { AIConfig } from "@amplitude/ai";
import { MockAmplitudeAI } from "@amplitude/ai/testing";

const mock = new MockAmplitudeAI(new AIConfig({ contentMode: "full" }));
const agent = mock.agent("test-agent", { userId: "u1" });

await agent.session({ sessionId: "s1" }).run(async (s) => {
  s.trackUserMessage("hello");
  s.trackAiMessage("response", "gpt-4o-mini", "openai", 150);
});

console.log(mock.summary());

The summary output looks like:

Agent Analytics fill-rate report
================================
Events captured: 2
  [Agent] User Message:  1
  [Agent] AI Response:   1

Verification gates (8/8 passing):
  ✓ user_id or device_id present
  ✓ [Agent] Session ID present
  ✓ [Agent] Agent ID present
  ✓ [Agent] Model Name present
  ✓ [Agent] Provider present
  ✓ [Agent] Latency Ms > 0
  ✓ [Agent] Input Tokens > 0
  ✓ [Agent] Output Tokens > 0
  ✓ [Agent] Cost USD > 0

To keep these checks in CI, assert on events with the mock client. Refer to Test against a mock client in the SDK reference.

Fix failing gates

For other symptoms, refer to Troubleshoot Agent Analytics setup.

Production checklist

Once your events start flowing into Amplitude, confirm that:

• Events carry real user IDs.
• Events carry real session IDs.
• Sessions close explicitly where the outcome is known, and by idle timeout only when expected.
• You've decided and reviewed your privacy mode.
• Agent IDs are stable and human-readable, and you've wired child agents for delegation.