Default prompts are tanking your agent’s retention

Most teams building agents care a lot about getting users to send a first message. Turns out, that isn’t the most important lever for retention.

After analyzing retention patterns with our own Global Agent, we found that users who sent two or more messages had more than double the retention of users who sent only one. That means encouraging users to send a second message is the activation behavior you should optimize for.

When we dug into this further, we learned that our first-time agent experience and default prompts were doing a poor job of getting users to a second message. The biggest culprit was a single default prompt that more than half of our new users started with.

If you’re building an agent and haven’t audited your defaults, you probably have the same problem. Here’s the data and what we’re doing to fix it.

Retention starts at the second message

Most teams treat a user’s first critical action (completing a purchase, booking a flight, uploading a file, sending a message) as the North Star for activation. The assumption is that doing it once means the user got value.

For agents, that first message is a low bar. Users will type something just to see what the thing does. The harder question is whether the response was good enough to earn a follow-up. That’s what the second message tells you.

In our own data, users who sent two or more messages to our Global Agent in their first week retained at more than double the rate of those who sent only one. Part of that gap is selection (users who were already going to retain are also more likely to send another message), but the signal is too sharp to ignore. A low second-message rate is a strong indicator that your first responses aren’t landing, and it’s worth diagnosing why.

Your defaults do more work than you think

If getting users to send a second message is key to retention, then the user’s first message to your agent still carries a lot of weight. It needs to be a compelling enough first-time experience that they want to send another message.

For most agents, the user’s first message (and thus, their first impression) is heavily shaped by what the product puts in front of them: pre-selected starter prompts, placeholder text in the chatbox, and suggested questions. That makes these defaults an important product choice, and if you’re like us, a choice that you’ve probably forgotten about.

Most default prompts are designed wrong

When we looked at the most common first messages new users sent to Global Agent, a single default prompt accounted for more than half of them. Turns out, it was a pretty bad one: “Explain what I can do in Amplitude.”

At first glance, this reads like a useful prompt. We thought it would help new users get oriented with Amplitude’s features and functionality. But users who started with this prompt retained at only 14%.

That’s when it clicked. We built one of the most capable analytics agents on the market, yet half of our new users treated it like a support bot. Because that’s what we told them to ask.

What separates a good default from a bad one

The “feature tour” is one of many bad default prompts. Another is the “I’m feeling lucky” prompt (“Tell me something surprising,” “Show me something interesting,” etc.), which sounds playful but is really just a coin flip. You’re betting the user’s first impression on a randomized output that may or may not produce something interesting. Sometimes it does, and other times it’s mildly amusing but goes nowhere.

None of these prompts connect to the user’s actual work, which is why they don’t drive retention. Now compare that with prompts like:

• “Investigate a specific user’s activity and debug their experience.”
• “What’s changed in my key metrics this week?”
• “Which user segments are growing fastest?”
• “How many unique users used the product last week?”

Users who asked these questions about their own product, metrics, and users retained at 34%, 2.4x the rate of those who asked the “Explain what I can do” prompt.

That was our sign to make a change. We introduced our default prompts without thinking much about them, but the data corrected us. We’re now updating our defaults to point users toward questions about their own data.

How to audit and fix your default prompts

If you’re building an agent, your default prompts are worth paying attention to. Here’s a three-step workflow to improve them:

1. Look at the prompt distribution. Which defaults are users actually picking? If one or two prompts are doing most of the work, your agent’s first impression might come down to UI you haven’t touched in a while.
2. Segment retention by first prompt. Aggregate retention might look okay, but slicing by first message could reveal a different story. The 14% vs. 34% retention difference we found was invisible at the top level and only emerged after we segmented by cohort.
3. Replace underperforming prompts. The bar for what makes a good default is whether the response makes the user want to send a follow-up message. If they don’t, swap it.

We’re continuing to use our own analytics product to run this analysis on Global Agent and improve its retention.

Build better first-time agent experiences

The make-or-break moment for your agent comes after the first message. Sending a second message more than doubles retention. And the default prompts you put in front of users on day one are one of the biggest factors in whether they do.

Most teams aren’t measuring their defaults at all, which means they’re probably leaving a 2–3x retention lift on the table. If you’re building an agent and you haven’t audited your defaults, that’s where I’d start. Try it today with Amplitude Analytics.