When I used to examine behavioral logs as part of building products, longer sessions made for cheerful meetings. We took more time spent in the product to mean that the product had improved. A single log could not tell us what filled that time. Someone who kept searching because they could not find an answer and someone absorbed in useful work could produce the same number.
AI products bring that question back. If a person keeps a conversation going, what has improved? We need to distinguish a more useful answer, deeper understanding, and time spent repeatedly correcting the output. Usage time measures the duration of an experience, but does not explain its value.
The experience I want to build begins with interest, develops through understanding and flow, and leads to action. I see this sequence as a way to evaluate a product, rather than a procedure every user must follow. Why does someone start using it? What do they understand? What activity absorbs them? What can they ultimately do?
Interest begins with a clear job to delegate
A blank input box invites a person to ask anything. The possibilities look extensive. Yet the person has to work out how those possibilities relate to today's task. Describing what a tool can do and demonstrating why someone would use it are different jobs.
A product that helps with reports need not promise every possible task on its first screen. It could show whether it compares evidence, produces a draft, or checks how claims relate to their support. A person should be able to adjust the scope, inspect a first result, and decide whether the tool meets their need. The first useful experience makes the problem the product will solve specific.
The experience that follows should stay connected to that job. Does the promised problem become clearer as the conversation develops? Is the person getting closer to the outcome they expected? Answering those questions connects the appeal of the first screen with the value of the product.
Understanding requires questions and revisions
Once someone starts using the product, they need to understand its decisions. Understanding means more than reading an answer to the end. It means being able to explain the conclusion and consider what would change under different conditions.
A report plan, for example, could distinguish the scope of inquiry, comparison criteria, and provisional conclusions. Changing one criterion could reveal which evidence needs another look and which conclusions would change. Examining a particular assumption may serve the task better than reading a long plan from beginning to end. An interface that supports understanding provides specific things to question and revise.
Presentation needs care here. Attractive charts and confident explanations can make a conclusion more persuasive. What matters is whether people can inspect assumptions, alternatives, and uncertainty. The explanation should help them understand the product's decisions more accurately.
Sustained attention depends on understanding progress
While an agent works, a person waits. They may move to another task and return without a clear memory of the original constraints. Reconstructing the context when the result arrives takes effort. If that effort is substantial, review can easily become a quick acknowledgment.
A continuous stream of status messages is not sufficient. The person needs to know which goal is being addressed, which conditions have changed, and what decision they will need to make next. Before an important review, the product can make the criteria available. After an interruption, it can provide a brief account of the context needed to continue.
There is no reason to keep a person watching the screen throughout execution. Provide the necessary context at the right moment so automated progress remains connected to human judgment. Attention can be supported by making the task's purpose and current state understandable, rather than by continually occupying someone's time.
Flow happens in an activity
Psychologist Mihaly Csikszentmihalyi described clear goals, immediate feedback, and a balance between challenge and skill as central conditions of flow.1 Attention brings a task into focus. Flow involves deeper participation in an activity, with performance and feedback unfolding together. Time spent looking at a screen is insufficient evidence of that state.
In an AI product, we should first identify the activity the person actually performs. Comparing evidence and revising the conditions of a conclusion involves judgment. So does choosing a draft that fits a purpose and articulating the reason for that choice. Seeing the result change in response gives those decisions meaningful feedback.
By contrast, repeated conversation while waiting for a suitable answer can leave both the goal and feedback unclear. If the same request produces another different response, it is hard to learn which choice improved the outcome. Designing for flow requires connecting a person's choices with changes in the result.
Action after flow belongs in the product's design
Flow can be enjoyable in its own right. When evaluating an AI product that supports work, we should also examine how the experience relates to its purpose. After finishing a report, can the person explain and use it? After studying, can they perform a similar task?
The METUX model connects technology and wellbeing by examining how experiences support autonomy, competence, and relatedness in activities and life, as well as engagement with an interface.2 From that perspective, evaluation continues after the conversation window closes. What a person understands and can do after using the product matters.
Increasing review has costs too. In a study of 199 participants, designs that encouraged analytical engagement with AI recommendations reduced overreliance, but participants preferred the more effective conditions less.3 We therefore need to establish both the moments that warrant review and the amount required. An experience designed to support understanding must also remain usable in practice.
Questions that reveal the value of an experience
If I could return to those meetings about session duration, I would ask different questions. What did the people who stayed longer accomplish? What did they understand? What would they be better able to do next time? We would examine those outcomes alongside the reasons their sessions grew longer.
The first experience would clarify the job to delegate. Continued use would make the reasons behind results understandable. People would see how their choices affected the work. When that experience leads to action, they can also explain why they use the product. An AI product that takes people from understanding to flow should help them do the work they care about more effectively.
Notes
[1] Mihaly Csikszentmihalyi, Flow: The Psychology of Optimal Experience, 1990. Publisher's book page. See also Flow 2.0, 2025, for a subsequent account of the conditions of flow. ↩
[2] Dorian Peters, Rafael A. Calvo, Richard M. Ryan, Designing for Motivation, Engagement and Wellbeing in Digital Experience, Frontiers in Psychology 9:797, 2018. METUX examines motivation, engagement, and wellbeing across levels of experience. ↩
[3] Zana Buçinca, Maja Barbara Malaya, Krzysztof Z. Gajos, To Trust or to Think, Proceedings of the ACM on Human-Computer Interaction 5, CSCW1, Article 188, 2021. Experiment with 199 participants examining overreliance and subjective evaluations. ↩