Track local and on-device models in Agent Analytics

Agents that run on a local or on-device model, such as Ollama, LM Studio, llama.cpp, or any inference server on localhost, reach Agent Analytics through the AI SDK's manual tracking methods. The AI SDK has no provider wrapper for local models, so you track each user message, tool call, and model response yourself. Sessions, built-in signals, and evaluators then work the same way as for hosted models.

Set up with your AI coding agent (recommended)

Paste this prompt into your AI coding agent, such as Cursor, Claude Code, Windsurf, GitHub Copilot, or Codex. The agent installs the AI SDK, reads its bundled instructions, finds every model call, and instruments it.

Instrument this app with Amplitude Agent Analytics using the Node SDK. Our agent runs on a local model, so track each user message, tool call, and model response with the manual tracking methods, and pass a cost of 0 on each response.

Install the SDK:

npm install @amplitude/ai @amplitude/analytics-node

Then follow `node_modules/@amplitude/ai/amplitude-ai.md`.

Review the changes it proposes, then verify with the steps in Verify.

What this means for you

• You call: trackUserMessage, trackToolCall, and trackAiMessage (Node), or track_user_message, track_tool_call, and track_ai_message (Python), inside an agent session.
• Cost: Local models have no price, so pass a cost of 0 on every response. The response then records zero cost instead of a missing value.
• Tokens: Pass input and output token counts if your runtime reports them. They're optional.

Track a local model

await agent.session({ userId, sessionId }).run(async (s) => {
  s.trackUserMessage(userInput);

  const start = Date.now();
  const response = await localModel.complete(prompt);
  const latencyMs = Date.now() - start;

  s.trackAiMessage(response.text, "llama-3.2-3b", "local", latencyMs, {
    inputTokens: response.usage?.inputTokens,
    outputTokens: response.usage?.outputTokens,
    totalCostUsd: 0,
  });
});

Track tool calls the same way with trackToolCall (Node) or track_tool_call (Python). For the full method signatures, refer to the AI SDK reference.

Verify

Run npx amplitude-ai doctor (Node) or amplitude-ai-doctor (Python). On a local-model project, doctor reports provider_dependency because it finds no supported provider package. Expect and ignore that result for local models. Every other check still applies.

Privacy

Local inference keeps prompts on your machine, but the AI SDK still sends message text to Amplitude, with PII redacted by default.