For AI agents: a documentation index is available at /docs/llms.txt. Append .md to any page URL for markdown, or send Accept: text/markdown.
Set up custom evaluators in Agent Analytics
이 페이지는 아직 귀하의 언어로 번역되지 않았습니다. 현재 작업 중이므로 곧 다시 확인해 주십시오.
Custom evaluators score your agent sessions against quality criteria you define, and calibration runs compare an evaluator's judgments against your own labeled examples to tune its accuracy. Both make model calls with a model provider key you supply (bring your own key, or BYOK). Amplitude's built-in signals run without any setup and don't need a key.
Use custom evaluators when the built-in signals, such as task completion and response quality, don't capture a criterion specific to your product. Each active evaluator produces one [Agent] Evaluator Result event per closed session. Refer to [Agent] Evaluator Result.
Add a model provider key
Add a key under Settings > AI Controls > Model Providers for any of the supported providers:
- OpenAI
- Azure OpenAI
- Anthropic
- Google Gemini
- Fireworks AI
- Amazon Bedrock
- Google Vertex AI
Custom evaluators and calibration runs use this key to make model calls on your behalf. Amplitude encrypts keys at rest and doesn't show them again after you save them, so keep your own copy if you need to reference or rotate it later. For more on AI settings, refer to AI controls.
이 내용이 도움이 되었나요?