Your agents are only as good as your data context

We read and analyze data with implicit business context that helps us interpret it. We know what data points mean and how they fit together. We know broader business objectives, which help us zoom out and see the bigger picture. We can recognize when something doesn’t smell right and dig in further before proceeding.

Agents just have data points. If you give agents a bunch of ones and zeros, they won’t automatically know their significance or how to connect them. Even worse, they think they understand the data and make a series of bad decisions from that assumption.

Agents need to understand what you’re feeding them before they can provide value. They need context and need to be pointed at specific use cases and data sets to start.

“AI can’t squint” is a great way of saying it doesn’t inherently have the context to sniff out the good from the bad.

Context: How you teach agents to think like humans

Let’s say you feed your agent sales data that shows a sudden spike in orders for a product. A human immediately realizes that the spike aligns with a limited-time promotion or a seasonal event and treats it as temporary.

But your agent doesn’t automatically have that context. It interprets the spike as the beginning of a long-term trend and recommends higher production. You end up with overstock and squandered resources. This could have been avoided had your agent understood the “why” behind your numbers.

When you define your metrics and the context around them, agents start to understand more of your universe and accurately connect the dots across your data points.

Your agents need both quantitative and qualitative context to do good work. Without qualitative data, you’re only getting half the picture.

Quantitative data can tell you what’s happening, but your qualitative data explains the drivers behind it. Voice of the customer (VoC) channels, session replays, and other qualitative touchpoints create tons of rich, unstructured data. Agents need to learn context from all these inputs.

How do you communicate the context of your data to agents? Via your semantic layer.

Building your semantic layer

A semantic layer is your company’s data translator. It uses plain language to explain what your columns of data mean. It also tells agents how to read the data and gives them everyday terminology to use when they talk back to your team.

Let’s say you have an event in your data called “GHI completed.” Humans on your team know that stands for “guided home in installation” and that it happens at the end of your onboarding flow. But if you ask your agent how many users completed onboarding, they won’t know to look at the “GHI completed” event. There’s nothing connecting the dots.

That's what context solves. It defines what an event means, when it occurs, and why it matters so everyone can interpret it correctly.

Last year, the Amplitude data team worked on getting our marketing attribution product data ready for AI. We had to build our semantic layer to do that, so our data team:

• defined every column being used by the business across Salesforce objects
• captured business definitions using real words people use in Slack and meetings to talk about their data
• flagged deprecated fields, duplicates, and added synonyms

This semantic layer provides all the context our agents need to produce accurate outputs. It also lets anyone ask questions about our data and get valid answers.

With a fully-built semantic layer, our agent answered questions correctly 80 to 90 percent of the time. The team tuned the last 10-20 percent using custom instructions and enriched the semantics even more based on user testing and feedback.

Rather than starting with a blank page, our data team used AI to speed up the process of building the semantic layer. our semantic building. AI generated the first draft of the definitions. Then data owners and stewards reviewed and edited them.

You can build your semantic layer in a data catalog, DBT YAML, using database metadata, or a system that makes the most sense for your business. Even a spreadsheet works if that’s where your team works.

Keep these things in mind as you build your semantic layer:

• Write business definitions using the words your team actually uses in Slack and meetings, not technical field names
• Add synonyms, including the UI label context, not just the API name
• Map data lineage so agents can trace where data comes from and understand the downstream impact of changes
• Tag sensitive data and give data a certification status so agents know what's trusted and what isn't
• Assign clear ownership: data owners, stewards, and technical contacts for each area
• Flag deprecated fields, duplicates, and add synonyms so agents don't get confused by outdated or overlapping terminology
• Use AI to generate a first draft of definitions, then have data owners review and refine them rather than starting from scratch
• Connect quality signals so agents know how much to trust the data when they answer questions

Start with context and contained agent inquiries

If your kid needs to learn about dog training for a school project, you wouldn’t drop them at the front door of the library and let them wander around pulling every book that has “dog” in the title. You’d tell them to look in the animal section.

But before your kid can search that section, the librarians need to curate it. They need to comb through their books and identify the right ones to make that search as simple and direct as possible. Data teams need to do the same them when organizing data sets for agents. Start by picking the data that makes sense for the specific questions you're trying to answer.

A solid semantic layer does double duty here. It not only embeds context and definitions into your data. It also makes your data easy to get around. It gives agents the business context they need to read your data correctly, and it gives data teams a place to say which datasets apply to which questions. Control the scope, give agents the right context, and you'll get answers that make sense.

For example, let’s say you have a question about customer churn for mobile app users in Europe. Feeding the agent your entire global customer database, including unrelated product lines, regions, and inactive segments, would be a waste of time and energy and could divert or dilute your agent’s attention. It’s introducing too much room for error.

Instead, start by defining a specific question: Why are European mobile app users churning in Q2? Then you'd pull together a focused dataset just for that group: recent usage logs, support conversations, satisfaction scores, and cancellation notes. This keeps the agent from drawing the wrong conclusions from data that doesn't belong, like behavior in other markets or channels that have nothing to do with this.

Have humans double-check (even with solid context)

Context is great, but you still shouldn’t give your agents the thumbs up without someone double-checking the work. You can automate and accelerate verification by giving your agents a certified result or sample dataset that they can measure against. These provide “the rules of the game” for value ranges, statistical thresholds, and business logic rules. Then, automation can check, flag, and send to humans for review.

Treat each incorrect agent answer as a chance to fine-tune your approach and close gaps. Maybe it didn’t have enough context of the problem you were trying to solve, so you need to provide more information.

Having humans double-check results keeps things accurate, lowers the risk of mistakes, and builds trust in what these agents can do. The more people see your agents getting it right, the more they'll trust them with real work. For data teams, that means fewer manual checks, faster decisions, and more time for the analysis that actually matters, instead of QA and data wrangling.

Start building your AI Context in Amplitude today.