Skills
Product Design
User Research
User Journey
UX Writing
Timeline
May ‘22 - Aug ‘22
Team
Designer/Project Lead: Lily Hu
Research partner: Kieran Hatton
Advisors: Sierre Wolfkostin (design), Kaush Ganesh (research), Scott Henry (data & metrics)
My Role
I led the design and research of a lightweight feedback prompt to capture user sentiment during authentication. I also designed 3 new components aligned with the Duo design guidelines and collaborated closely with researchers and data scientists.
IMPACT
Responses collected
in 6 months
Increase in actionable responses
Survey complaints
2 users wanted to see the survey more frequently
New Authentication
OLD Authentication
Context
Cisco Duo provides two-factor authentication (2FA) for organizations to keep user logins secure. In 2023, they introduced the Universal Prompt, a more secure authentication flow. It was critical to migrate users quickly due to rising security threats, and as a security platform, Duo needed to test the experience with real users in real-world conditions. To ensure a smooth rollout, we needed a clear, measurable way to quickly identify bugs and usability issues—a challenge given the already high-friction nature of user authentication.
OlD Survey
Unable to collect feedback at scale
Initially, a quick, persistent survey above the prompt was used to discover-and-fix issues with the product experience.
However, organizations, our customers, felt it was too beta and unpolished for production use and it was deprecated in January 2022 as a result.
Analyzing the Persistent Survey
At the start of the project, I worked closely with the data science team to evaluate the 9,000+ survey responses collected and identify what they needed to extract more meaningful insights.
OlD Survey Responses Visual
Revised Problem Statement
How might we build a professional, low-friction survey that collects accurate and actionable user feedback?
Identifying Constraints
I first consulted with the engineers, data scientists, and accessibility experts to understand our constraints:
• No pop-ups (for accessibility and security)
• Authentication time is limited, so interaction had to be quick
• Survey must match Duo’s brand, visually and in tone
guiding principles
Get users in and out quickly
Avoid cognitive overhead
Collect accurate and actionable feedback
Narrowing down an idea
I explored many design ideas and everyone had their own idea. But how did we distill it down to one solution.
Where in the authentication process?
To better understand where the survey would be most effective in the user flow, I had to re-examine the thoughts and feelings of users at each step of the authentication process.
I reviewed 50+ authentication sessions and probed into how users interact with the login process to determine where the survey would be most effective.
How should the Survey Collect Data?
To answer this question, I looked to existing survey methods to better understand what they measure what best aligns with our goals:
• Report Forms
• Siskel and Ebert Approved Rating System
• Product Surveys
• Emoticon Ratings
• Star Ratings & Selection Tags
The smiley face rating survey is quickest and easiest since it relies on visuals over text. To reduce noise and collect more actionable feedback, I decided to explore more targeted questions, selection tags, and checkboxes.
Version B Performed the best
How can we design the survey’s feedback input to clearly capture user issues and ensure the most efficient interaction method?
An A/B Preference Test revealed 3 key findings:
Revised User FLow
How might we design a feedback channel that Lee can quickly understand and respond to during a time-sensitive moment?
In an unmoderated usability test, 20 users completed a typical authentication flow and issue reporting task.
Key findings:
Final Design
Final Usability Test
I ran one final usability test to validate the latest revisions. Users completed the standard authentication and issue reporting tasks. With updates to the copy, UI, and flow, the experience felt much more refined, and we were confident moving into the private preview phase.
How frequent should the survey be?
95% of participants preferred receiving the survey no more than once a month, which aligned with when they expected product updates.
To implement this, I worked with developers and the project manager to use a cookie-based system: fewer than 1% of authentications will trigger a check, and users who completed the survey within the past two months won’t be shown it again.
Design goes Public
Over a 6 month period, we collected nearly 32 thousand responses with a majority including freeform text that greatly helped the team identify several issues that the team was previously not aware about.
”By all measures, the survey was a success, with high and helpful levels of engagement, and very little negative pushback to the survey itself.” - Jake Ingman (Principal Product Designer at Cisco)
Traditional Prompt Retired
The volume and speed of feedback drove data-backed design decisions, directly shaping the Universal Prompt and helping retire a legacy system on March 30, 2024.
Reflection
As an intern, it was understandably unusual for me to lead a project, especially one involving a survey that could significantly disrupt the user experience. I received pushback from several team members who were unsure about the idea. To address their concerns, I actively listened and invited feedback through three design jams with designers, engineers, and researchers. This collaborative approach helped build trust and shaped the direction of the final solution.
Throughout the process, I learned how to weigh different perspectives, make tough calls when ideas had to be left behind, and support my design decisions through multiple rounds of testing and self-guided research. One area I’d improve in is being more comfortable with not resolving every concern, and recognizing that alignment is sometimes more important than agreement.
"Lily shows up everyday with a desire to learn. She proactively seeks knowledge both inside and outside of Duo to refine her designs. She solicits feedback early and often, and goes out of her way to seek multiple perspectives on her work."
- Sierre Wolfkostin, Design Manager