CASE№ 010·Experiment analysisUser behaviorEffect over time

The prompt worked as a reminder. The effect did not last.

The team wanted to roll out a homepage prompt. I investigated the raw data and found a short-lived effect for specific users, rather than evidence of a lasting improvement.

Product typeConsumer mobile app
Context checkedRaw experiment data
SurfaceiOS
01 · The proposed rollout

The homepage prompt showed promising results.

The company tested a prompt on the homepage. It showed results, and the team wanted to roll it out. I investigated the raw data to understand what was behind the effect.

02 · What I found

The effect was limited to particular users and a short period.

The prompt worked for specific users, and only for a short time. The pattern was consistent with a reminder effect rather than a lasting improvement in behavior.

03 · What that meant

Keeping it permanently needed a different justification.

The result did not support expecting substantial ongoing benefits for existing and new users from keeping the prompt indefinitely.

My analysis separated the short-term reminder effect from the lasting improvement the team was considering a rollout for.

Let’s talk

Bring me
the problem.

An app that needs building. A migration that needs someone to own it. Numbers that don’t make sense. A product decision your team can’t resolve. Tell me what’s happening. I’ll tell you whether I’m the right person, how I’d approach it, and what the work would cost.

Email Alex ↗[email protected]
You bring the context. I take the work seriously.
Not sure what you’d get? See sample reports →