Finding the balance between structure and natural conversation
Conversational AI created an opportunity for learners to practice realistic conversations in a safe environment before applying language skills in the real world.
But designing the experience introduced a fundamental tension: language learning requires structure, guidance, and feedback, while natural conversations are open-ended and unpredictable.
The central question became: How do we create an experience that feels like talking to a real person while still helping users learn?
As Design Lead, I was responsible for design direction, stakeholder alignment, and design quality across the experience. Rather than designing the screens myself, I worked closely with a product designer to define the principles, frameworks, and key decisions that shaped the product.
A shift to high-level design directions
Rosetta Stone Sapphire Chat Missions was a high-priority initiative exploring how conversational AI could support language learning.
My director asked me to oversee the project because of the ambiguity around AI behavior, learning outcomes, and product direction. While the product designer owned day-to-day execution, I focused on defining the principles and frameworks that shaped the experience, aligning stakeholders around key decisions, and maintaining a high bar for design quality.
Defining AI behavior: conversation partner, not tutor
The first principle we needed to establish was the relationship between the learner and AI chatbot.
Was the AI chatbot a tutor? Or was it a conversation partner?
A tutor optimizes for instruction. A conversation partner optimizes for practice. Both are valuable, but combining them created tension. If the AI corrected every mistake as it happened, conversations became fragmented and started to feel like traditional lessons rather than immersive practice.
I advocated for a guiding principle:
Preserve the conversation first. Teach at the right moments.
This principle shaped how feedback, corrections, and AI behavior worked throughout the experience.
Creating a Feedback System That Supports Immersion
Learners needed corrections to improve, but different mistakes required different levels of intervention. A spelling mistake does not carry the same weight as misunderstanding a question or using language inappropriate for the situation. Instead of treating all feedback equally, I helped define a layered feedback model:
This allowed Chat Missions to provide meaningful learning moments without constantly interrupting the interaction.
Designing Guardrails for Open-Ended Conversation
Open-ended conversations also introduced unpredictable behaviors. Learners might switch back to their native language, misunderstand the scenario, or take the conversation in an unexpected direction. The challenge was maintaining structure without making the experience feel restrictive.
I helped define AI behavior patterns that guided learners back toward their goals through natural conversation rather than explicit error states. The AI could acknowledge unexpected responses, provide support, and redirect the conversation while preserving the feeling of talking with another person.
Defining when and how a mission should end
One of the hardest questions was deceptively simple:
When is a conversation complete?
The initial approach was to end the mission once the learner completed all required goals. But completing a checklist doesn't mean completing a conversation.
I proposed a different approach. Once learners completed their goals, we would acknowledge their progress and provide a path to review the conversation, while still allowing the interaction to continue. The AI would then guide the conversation toward a natural conclusion and automatically wrap up within a limited number of additional exchanges.
For example, imagine a scenario with three goals:
If the experience ended immediately after asking for the password, the learner technically succeeded but this ignores how real conversations work. The interaction needed a natural closing moment.
I proposed a framework that separated goal completion from conversation completion.
Once learners completed their objectives, we acknowledged their progress and gave them the option to review their conversation. If they continued chatting, the AI naturally guided the interaction toward a closing moment.
This maintained the authenticity of real conversation while giving the experience enough structure to support learning.
Positioning Chat Missions as a core part of the Sapphire experience
As the product evolved, Chat Missions shifted from a standalone AI experiment into a core part of the Rosetta Stone Sapphire learning journey. That shift changed how we thought about the experience.
Chat Missions wasn't simply another AI tool. It filled a specific role: Lessons build knowledge. → Chat Missions build confidence applying that knowledge.
I helped reposition Chat Missions as a companion to the core lesson experience, influencing decisions around navigation, interaction patterns, and integration with the broader Sapphire ecosystem.
When the team explored grouping Chat Missions with other AI-generated study tools, I pushed for dedicated visibility because conversation practice represented a fundamentally different learner need: strengthening language retention through active practice rather than generating learning resources.
This helped position Chat Missions as a continuation of learning rather than a separate AI feature.
Raising the quality bar
Alongside product strategy, I partnered with the product designer through critiques and working sessions to elevate the overall experience.
A key part of Chat Missions was helping learners imagine themselves in realistic scenarios, which made visual quality especially important.
I helped establish direction for AI-generated scenario imagery, focusing on consistency, realism, and avoiding the artificial qualities often associated with generated content. This also surfaced a larger product opportunity: truly immersive language practice requires cultural context.
Ordering food at a local restaurant in Seoul, Paris, and San Francisco shouldn't feel interchangeable. (Would you order bibimbap from a restaurant in Paris?) While deeper localization was outside the initial release scope, I helped identify cultural adaptation as an important future opportunity for making Chat Missions feel more personal and authentic.
Understanding how learners engaged with AI conversation practice
Following a soft launch, we analyzed early usage patterns to understand how learners naturally engaged with AI conversation practice.
Because Chat Missions introduced a new interaction model, our focus was not only adoption. We wanted to understand whether learners were engaging in ways that supported meaningful practice.
Discovery
In the first month since the soft launch, 611 learners organically started a Chat Mission. The most-started scenarios centered around practical real-world situations:
- Asking for directions: 571 starts
- Introducing yourself: 102 starts
- Bargaining at the farmers market: 81 starts
- Checking into your hotel: 69 starts
- Ordering a coffee: 65 starts
Looking deeper, we found that discovery patterns strongly influenced engagement. Scenario placement significantly impacted starts, while labels like "Intro" and "Popular" increased visibility even for scenarios placed lower in the experience.
This identified opportunities to better personalize recommendations based on learner goals, proficiency, and interests.
Understanding Engagement Quality
Initial data showed that 20% of started missions resulted in learners completing all three scenario goals.
Rather than viewing completion alone as success or failure, we used the data to identify the next set of product questions:
- Were learners dropping before meaningful interaction started?
- Were they having valuable conversations that didn't follow predefined goals?
- Did the experience provide enough guidance toward completion?
To answer these questions, we began evaluating additional signals including conversation length, message volume, first-message drop-off, and goal progression.
These insights continue to shape future improvements to onboarding, scenario design, and AI guidance.
The hardest design questions are about behavior, not interfaces
This project expanded my perspective on design leadership. My greatest contribution wasn't designing every screen. It was defining the principles that shaped the experience and creating alignment around ambiguous decisions.
With emerging technologies like AI, the hardest design questions often aren't about interfaces. They are about behavior:
- What role should the product play?
- How should it respond?
- What experience are we trying to create?
Answering those questions created the foundation for a learning experience that balances the freedom of conversation with the structure learners need to grow.