Send GitHub Copilot telemetry to Agent Analytics

GitHub Copilot CLI and Copilot Chat in VS Code export OpenTelemetry GenAI traces. Point them at Amplitude. You don't need the Amplitude AI SDK in the agent. Each Copilot session becomes one Agent Analytics session.

Last verified: pending a live capture. The variables, settings, and span attributes on this page come from GitHub's Copilot CLI command reference and the VS Code Copilot extension source, for Copilot CLI 1.0.4 or later.

Copilot usually runs for your own engineers, so the user is an employee who usually isn't in your product analytics. The outcome to chart is what happened to the code. Refer to Connect to outcomes.

Set up with your AI coding agent (recommended)

Paste this prompt into your AI coding agent, such as Cursor, Claude Code, Windsurf, GitHub Copilot, or Codex.

Connect GitHub Copilot to Amplitude Agent Analytics with native OpenTelemetry.

Fetch https://raw.githubusercontent.com/amplitude/Amplitude-AI-Node/main/docs/integrations/copilot.md
and follow it.
If @amplitude/ai is installed, the same file is under node_modules/@amplitude/ai/docs/integrations/.
Also follow https://raw.githubusercontent.com/amplitude/Amplitude-AI-Node/main/docs/integrations/privacy.md

Review the changes, then verify with the steps in Get started.

What this means for you

• You run: Copilot CLI or Copilot Chat in VS Code, with its OpenTelemetry exporter pointed at Amplitude.
• You need: Copilot CLI 1.0.4 or later or the VS Code Copilot extension, your Amplitude project API key, and the user ID to record for each engineer.
• Protocol: OTLP over HTTP. Amplitude doesn't accept gRPC.
• Privacy: Amplitude doesn't scrub content after receipt. Message and tool content is off unless you turn it on.

Export

Copilot reads its telemetry settings from the environment (CLI) or from your VS Code user settings, so init writes no Copilot file. It installs the commit hook and the outcomes workflow, and prints both blocks with a placeholder for the key. It requires @amplitude/ai 0.28.0 or amplitude-ai 1.25.0 or later.

npx -y -p @amplitude/ai amplitude-ai coding-agents init --harness copilot
# or, in a Python project
amplitude-ai coding-agents init --harness copilot

With no --harness, init includes Copilot when the repository has .github/copilot-instructions.md or .github/hooks/.

For Copilot CLI, add this function to your shell profile (bash or zsh). init prints it with your email filled in.

copilot() {
  local key="${AMPLITUDE_AI_API_KEY:-$(sed -n 's/^AMPLITUDE_AI_API_KEY=//p' "$(git rev-parse --show-toplevel 2>/dev/null)/.amplitude-ai/local.env" 2>/dev/null)}"
  COPILOT_OTEL_ENABLED=true \
  OTEL_EXPORTER_OTLP_ENDPOINT=https://api.amplitude.com/otlp \
  OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20$key" \
  OTEL_RESOURCE_ATTRIBUTES="enduser.id=<your git email>" \
  OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true \
  command copilot "$@"
}

These are the standard OpenTelemetry variables. Exported from your profile, they would send every OpenTelemetry-instrumented process in your shell to Amplitude, including local services and test runs. The function sets them for one copilot run only, though processes that Copilot starts still inherit them. The function reads the key when it runs, from AMPLITUDE_AI_API_KEY or else from the repository's gitignored .amplitude-ai/local.env, so the key isn't stored in a dotfile.

For Copilot Chat in VS Code, add this to your user settings.json. Don't add it to the workspace settings, because it holds the key.

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.otlpEndpoint": "https://api.amplitude.com/otlp",
  "github.copilot.chat.otel.headers": { "Authorization": "Bearer AMPLITUDE_API_KEY" },
  "github.copilot.chat.otel.resourceAttributes": { "enduser.id": "<your git email>" },
  "github.copilot.chat.otel.captureContent": true
}

With --region eu, the endpoint is https://api.eu.amplitude.com/otlp. Enterprise managed settings take precedence over both blocks and can turn telemetry or content capture off.

Identity

Copilot doesn't report how a session started. To set [Agent] Session Trigger, add amplitude.session_trigger to the resource attributes with one of interactive, headless, ci, scheduled, or event: after enduser.id and a comma in OTEL_RESOURCE_ATTRIBUTES for Copilot CLI, or as a second key in github.copilot.chat.otel.resourceAttributes in VS Code. Amplitude sets it on every event and ignores other values.

What carries over

Privacy gates

OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true (CLI) or captureContent (VS Code) adds message text, tool arguments, and tool results. init --privacy metadata leaves it out. Models, tokens, tool names, and errors still arrive. Refer to Agent Analytics privacy modes.

Known gaps

• No session on commits. Copilot gives its shell commands no session ID, so the commit hook adds no trailer. The outcomes workflow scores only commits with an Agent-Session-Id trailer, so Copilot commits get no outcome scores.
• Copilot coding agent. It runs in GitHub's sandbox, which your environment doesn't reach, so it sends no sessions. Its commits carry no session trailer, so they get no outcome scores either.
• Session trigger rollout. amplitude.session_trigger needs the Amplitude OTLP receiver release that maps it. Until it's live in your data center, Copilot events have no [Agent] Session Trigger.
• Metrics. Copilot also posts metrics. Amplitude accepts and discards them.
• Policy. Enterprise policy can block telemetry or content capture.

Connect to outcomes

Copilot commits carry no session trailer, so the outcomes workflow sends no scores for them. When a commit from another agent in the same repository carries a session trailer, that session gets its scores.

In-session outcomes come from your [Agent] Tool Call events, with no extra setup. Binary evaluators can label sessions that made a commit, opened a pull request, or ended with passing tests. A code evaluator with a regex over tool turns, such as \bgit\s+commit\b, finds commits and pull requests. An LLM evaluator judges whether the last test command succeeded. Refer to Create and calibrate evaluators.

Code evaluators read the tool input of every turn, so a commit at the end of a long session still counts. Each evaluator writes an [Agent] Evaluator Result, so you can compare it with built-in signals: do Task Completed sessions actually produce commits? A tool call that creates a commit can also carry a commit_sha event property. It's read from the [branch sha] line that git commit prints, or from the sha in a JSON result when the tool name or input names a commit, pull request, or file push as a whole word, such as create_pull_request or push_files. Output from git log, git show, and other hex strings is never read as a commit.

Downstream outcomes, such as merged, reverted, and deployed, need two pieces that coding-agents init writes into the repository:

1. A prepare-commit-msg hook in .githooks/ that stamps Agent-Session-Id and Agent-Name trailers on every commit the agent makes. Human commits stay unchanged, and --no-verify doesn't skip this hook. It also adds an Agent-Trigger trailer when it knows how the session started, as described after the score tables.
2. A GitHub Actions workflow, .github/workflows/agent-outcomes.yml, that reads those trailers and sends each outcome to the Amplitude HTTP API as an [Agent] Score on the session that wrote the commit. The outcome appears next to the session's own events and evaluator results, with no new event type. Store the project API key in the AMPLITUDE_API_KEY repository secret. These repository variables are optional:

The workflow asks for read access to contents, pull-requests, deployments, actions, checks, and statuses. It reads workflow runs, check runs, and commit statuses for ci_passed_first_run. For a pull request from a fork, meaning one whose head branch is in another repository, GitHub doesn't pass repository secrets to the run, so the workflow logs a notice and sends nothing instead of failing.

Each score sets these properties:

The event time is the commit's author date, so the score falls inside the session that wrote the commit. outcome_time holds when the outcome happened: the merge, the revert's merge, or the deployment.

Every score also carries commit_sha, agent_session_id, agent_name, agent_trigger, agent_setup_version, repo, outcome_time, and outcomes_workflow_version, set to 3.

• Only agent sessions get scores. A commit without an Agent-Session-Id trailer sends nothing. That covers human commits, Gemini CLI commits, commits made through the GitHub API or MCP tools, and cloud-agent commits known only by their author or Co-authored-by line. For a pull request with no such commit, the workflow also skips the reviews and CI lookups.
• A pull request counts once for each session. When several sessions wrote one pull request, each gets the pull request scores, and pr_session_count says how many sessions there were. To count pull requests, count distinct pr_number values.
• The user ID is the commit author email, or the GitHub login with AMPLITUDE_USER_ID_SOURCE=login. It must match the user ID your agent sessions use. coding-agents init sets the session user ID to the git email. With login, the pull request scores fall back to the pull request author when the session's first commit has no GitHub account. A commit with no user ID gets no scores.
• A score with no value isn't sent, such as hours_to_first_review for a pull request with no human review.
• Insert IDs are the score name, the repository, and the commit SHA, plus the deployment ID for deploy scores, the revert's pull request number for commit_reverted, and the pull request number and session instead of the SHA for the pull request scores. A rerun within Amplitude's 7-day deduplication window doesn't count twice.
• Outcome scores aren't user feedback. Agent Analytics reads only scores with the user source as user feedback, so an outcome never counts as a thumbs up or down.

agent_trigger says how the session that wrote the commit started, with the same values as [Agent] Session Trigger on the session's events: interactive, headless, ci, scheduled, or event. It comes from the Agent-Trigger trailer. The hook adds that trailer only to commits it stamps with an agent session or name, and only when it knows the value. It resolves the value the way the Codex and Cursor hooks resolve [Agent] Session Trigger, so a commit and its session get the same value:

1. AMPLITUDE_SESSION_TRIGGER in the environment, when it's one of the five values. Set it where the agent runs, for example AMPLITUDE_SESSION_TRIGGER=headless for a script that runs the agent without a person, or scheduled for a scheduled job. When the variable is set, the hook doesn't read .amplitude-ai/local.env for it.
2. Otherwise AMPLITUDE_SESSION_TRIGGER in .amplitude-ai/local.env at the repository root. The hook reads the file as KEY=value lines and never runs it.
3. Otherwise ci, when GITHUB_ACTIONS is true, or CI is set to anything other than an empty value, false, or 0.
4. Otherwise, for Codex, how the session started, from source in the first line of the session transcript under $CODEX_HOME/sessions (by default ~/.codex/sessions): codex exec or the MCP server gives headless, and the CLI or VS Code gives interactive.
5. Otherwise no trailer.

Without a valid trailer, agent_trigger is event when a coding agent is the commit author, as when a cloud agent works from an issue or an assignment, and null otherwise.

The pull request scores use these definitions:

commit_count is the pull request's commit count. GitHub lists at most 250 commits, so commit_merged, agent_commit_count, and session_commit_count cover the first 250.

agent_setup_version identifies the agent setup a commit was written with. It's the first 12 hex characters of a SHA-256 over the root-level entries named in AMPLITUDE_AGENT_SETUP_PATHS, read from the commit's parent. Any change to one of those files, or to a file inside one of those folders, produces a new value. It's null when none of the entries exist. A commit that edits AGENTS.md carries the version from before the edit, and the next commit carries the new one. On the pull request scores, it's the value for the session's first commit in the pull request.

agent_setup_version and the review and CI scores are best effort. Together they make at most 300 GitHub API requests in a run, counting each page of results. When one fails or the budget runs out, the workflow leaves out that property or score, logs a warning, and still sends the rest. The run log includes an enrichments: line that shows which ones succeeded and how many API calls the run made. Treat a missing value as unknown, not as 0.

Because each outcome is a score on its session, you can chart it next to the session's own events with no join. Every chart below starts from [Agent] Score filtered on [Agent] Score Name. Charts by event time bucket outcomes by when the work was written. The step-by-step recipe is the code outcomes recipe.

• How much agent work holds? Chart the count of commit_merged minus the count of commit_reverted, grouped by [Agent] Agent ID. Both are timed at the commit's author date, so a date range that ends 30 days ago counts the changes written at least 30 days ago that are still unreverted. The revert rate is commit_reverted divided by commit_merged.
• Did a change to the agent setup help? Divide commit_reverted by commit_merged, and deploy_failed by both deploy scores, grouped by agent_setup_version. A deploy score counts one commit in one deployment attempt, so a commit in a failed deployment that later succeeds counts once in each.
• Do agent pull requests pass CI the first time, and how much review do they need? Chart the average [Agent] Score Value of ci_passed_first_run, which is the pass rate, and of review_rounds and changes_requested_count, grouped by [Agent] Agent ID or agent_setup_version. Every session in a pull request carries these scores, so for rates by pull request, count distinct pr_number values.
• Which sessions reach production? Build a funnel from an [Agent] Tool Call where commit_sha is set, to commit_merged, to deploy_succeeded, held on [Agent] Session ID.
• Which session patterns get reverted? Group commit_reverted by an evaluator's result, such as the Session Shape evaluator.

These charts compare work across stages: a person at the keyboard, a headless run, CI, a schedule, or an event.

• Sessions by stage: Count distinct [Agent] Session ID values on [Agent] User Message, grouped by [Agent] Session Trigger. Add a filter on [Agent] Agent Chain Depth greater than 1 to see delegated work. Sessions whose harness can't tell how they started have no trigger.
• Grind rate by intent and model: Chart [Agent] Evaluator Result where [Agent] Evaluator Name is the Session Shape evaluator, grouped by [Agent] Primary Label and [Agent] Models Used. The Grind share is the grind rate. Chart the Coding Intent evaluator's results the same way, as the second breakdown next to it. Each Evaluator Result carries one evaluator's label, so the two results don't combine on one event. Refer to Coding agent templates.
• Claimed done without verification, by stage: Chart [Agent] Evaluator Result where [Agent] Evaluator Name is the Claimed Done Without Verification evaluator, grouped by [Agent] Session Trigger, with the average of [Agent] Binary Value as the metric. That average is the rate for each stage. Evaluator Results carry the session's trigger, so this is one event and no join.
• Review effort by stage: Chart the average [Agent] Score Value of review_rounds and hours_to_first_review, grouped by agent_trigger.
• Cost per merged change, by stage or by agent tree: A formula that divides the sum of [Agent] Cost USD on [Agent] AI Response by the count of [Agent] Score where [Agent] Score Name is commit_merged. Group the two sides on the matching value: [Agent] Session Trigger against agent_trigger, or [Agent] Root Agent Name against agent_name. It's an aggregate ratio, not a join for each change, and it counts cost only from harnesses that report one.

Measure a prompt that went viral. A prompt or skill can spread to many teams before anyone checks whether it works on their code. To find out across every session, by outcome:

• A pasted prompt: Create an evaluator from the Prompt or Skill Used template, a Regex evaluator that scans user messages, and replace its placeholder pattern with the prompt's text. It reads user message text, so it matches only sessions that send prompts, as in the full content mode or with OTEL_LOG_USER_PROMPTS=1 for Claude Code. Then compare Session Shape and Claimed Done Without Verification on the matching sessions against the rest: filter the Sessions tab by the Prompt or Skill Used result, once True and once False, and read the other two evaluators' results.
• A prompt saved as a command or skill: Saving it under a path in AMPLITUDE_AGENT_SETUP_PATHS, such as the default .claude, .codex, or .cursor, changes agent_setup_version. Compare the review_rounds and ci_passed_first_run scores, the revert rate (commit_reverted divided by commit_merged), and the deploy failure rate (deploy_failed divided by both deploy scores) by agent_setup_version, before and after the version that added it.
• Across harnesses: Group the session charts by [Agent] Agent ID, for example claude-code against codex, to see whether the prompt works as well in each one.

Upgrade from an earlier version: Run init again, with npx -y -p @amplitude/ai amplitude-ai coding-agents init, or amplitude-ai coding-agents init in a Python project. It replaces an unchanged copy of an earlier workflow. init reports a workflow you edited as customized and leaves it alone. --force replaces it, or you can move your edits to the new workflow by hand. coding-agents doctor reports the workflow as current, outdated, or customized. Version 3 sends [Agent] Score events in place of the Commit Merged, Commit Reverted, Pull Request Merged, Deploy Succeeded, and Deploy Failed events that earlier versions sent, so rebuild charts that use those events on [Agent] Score Name.

Get started

Follow the GitHub Copilot integration guide. From the shell that starts copilot, run coding-agents doctor. It can't read VS Code settings. Then run one Copilot session in the repository and confirm in Live Events:

1. Session ID is the Copilot conversation ID.
2. User ID is your enduser.id, not unknown.
3. The session shows its model calls, with the model name and token counts, and its tool calls.