It's not AI! Designing for automated conversations
Designing for automated conversations with a few practical tips and real examples from shipping live to production
Below are a few practical things we learned as a design team when building automated conversations for our two products ReferralCandy and CandyBar.
A snapshot of CandyBar Assistant. It tells people how many rewards, stamps and points they have
Why automated conversations?
For us, it all started with an experiment.
We had an SMS being sent out, and we wanted to make it better i.e. more useful, helpful, more engaging and interesting.
Trying to make the SMS shorter, cleaner, with neater links
We did a ton of usability testing on the SMS flow. The results were not great. People didn’t notice the message. It didn’t feel relevant, engaging or interesting.
We had to figure out a better way of messaging people.
Our first MVP experiment with automated messaging was with Facebook Messenger.
Results looked good! Our engagement went up, our testing lead to better results, more people were giving feedback about the places they visited, and conversions looked good too.
So here’s what I learned from the last two years of building automated conversations with my team:
1. Don’t make it look like a human
Samantha is not a real person. Her photo is from unsplash.com, a website for free to use commercial photos.
Trust is fragile in an automated conversation.
Make sure your automation clearly looks like a bot. An automation cannot do human tasks.
Be honest and transparent that it’s just a bot and therefore it’s limited. Use non-human names and avatar images.
Our product is both merchant facing and customer facing. So our interface and bots speak to both end consumers and merchants who want to connect with their customers in a meaningful way. We also have an excellent customer support team talking to these folks.
In a complex situation where both people and bots are helping out, clearly differentiating between the two is the key between a good or effective experience and a bad or confusing experience.
2. Design basics are the same, but your tools are different
Stick to a typical iterative product design process.
Your qualitative and quantitative user data is in the center of your process as you ideate and test your ideas in the real world. Make sure you are measuring the right thing. Your success metrics should be chosen carefully.
We did a lot of testing especially for the first MVP automated conversation
But your tools are different
Sketch isn’t the best way to draw your MVP Conversation
Initially I tried to make detailed mockups but my team found it difficult to understand the MVP concept and design updates this way.
Some initial sketch mockups for the MVP
We were iterating too fast (every 2 days) for good looking mockups.
What actually worked:
Conversation tree for the MVP
Then, write an MVP script to test each flow
Imagine your MVP conversation like a movie script with characters, stages and scene changes. This will help you test each flow properly, so that it sounds as natural as possible.
Once you have your Conversation tree and MVP script, create some visual story elements:
A few story elements from CandyBar Assistant
Prototyping is the best way to share your work
I recommend Botsociety for quick prototyping for your MVP
So in conclusion, basic design process is the same, but the tools you use are different.
3. Become a really good writer OR get a really good writer
Can’t underline this enough. Your MVP has to have good writing. It’s not optional.
Getting a culture check is good too. Sitting in Singapore, we didn’t know this would happen with Americans using our bot. A surprisingly large number of people hate Jimmy Fallon
4. It’s ok to say sorry and I don’t know
Error handling is always important in any design process, it’s just way trickier in a conversation.
This person asks a pretty relevant question “How many points do I have?” but CandyBar Assistant doesn’t understand. It quickly apologizes, making it clear that it is limited, and provides a button so the person can find their answer anyway. By tapping on “View Rewards” this person can see how many points they have.
5. It’s not AI and it doesn’t have to be AI!
You are replacing a traditional interface of fields and labels with a rich conversation.
The bot reminds me of this olden days CLI
The bot reminds me a bit of this ancient command line interface I used to load up games when I was a kid. You put in queries, and the system did a thing. If you put in the wrong query, it failed horribly.
With an automated conversation, it’s pretty much the same Q&A format, just in a nicer, more human, friendlier wrapping. Maybe even some jokes, emoji’s and GIFS added to the mix!
You don’t have to have artificial intelligent to create a really really good experience for the people using your software that’s fun, enjoyable and also works.
TLDR:
Don’t make it look like a human.
Trust is fragile in an automated conversation
Design Basics are the same but your tools are different
Write your MVP script and test as much as possible
Get a really really good writer
It’s ok to say sorry and I don’t know
It’s not AI! It doesn’t have to be AI to be effective
Thank you for reading! This article is based on a talk I gave at UXSEA in Singapore, Nov 2018.
I’d love to hear your perspective on this, please add your comments below.
Design metrics for better design decisions
This quote by Spock explains what most designers instinctively understand. It’s easy for numbers to mislead a team down the wrong path. You need to be constantly vigilant that you are measuring the right things, for the right reasons, in the right way.
So how can metrics help designers in a consistent, practical way?
This quote by Spock explains what most designers instinctively understand. It’s easy for numbers to mislead a team down the wrong path. You need to be constantly vigilant that you are measuring the right things, for the right reasons, in the right way.
So how can metrics help designers in a consistent, practical way?
1. Funnels convert complex user behavior into neat little chunks of data
Let’s say you have an amazing number of 5000 signups/week. However this is not actionable to you a designer. Even if you have a trend on top saying signups are going up or down, that is still not helpful.
Funnels are a great way to identify starting points for a redesign. You can prioritize what needs to be fixed first by looking at conversions and drop-offs. You can even kill the entire funnel and make a new one.
2. Upgrade to a goal oriented design process (stop the opinions)
Awesome illustration by The Design Team
Design reviews often spiral into confusion and vagueness. People focus on fonts and colors when the real stuff is the interaction and user flow. Use metrics to keep everyone clear headed and on point. Especially for non-designers, it’s a great way to start learning about potential impact of good design.
Use metrics to deal with superficial stakeholder feedback like “Make the logo bigger.”
What I imagined when I read the amazing tweet above
3. Test and validate your designs
It’s not always possible to extensively test your design with user research. Especially for superficial changes, use metrics to validate your designs once they are already live in production. This is the fastest way to work in small teams. Keep checking and validating that you are on the right track with user interviews and numbers.
Any extensive research effort with both interviews and a/b testing may lead to a month of wasted time when you can always validate on the go.
Old hero tile
New hero tile
For example we recently redesigned this hero tile for our product. Instead of extensive internal reviews and testing for a relatively minor change, we went ahead and made the change in a day. Then we checked that this was working better than the older version and it was.
Conversions increased by 7%!
4. Setup your own design metrics stack
In about 3–4 weeks you should be up and running so that your design team can make more data based decisions. Depending on how your product architecture is structured, it may be easier or harder. But it’s always worth it.
There are a ton of amazing services out there that make it much easier today.
Fullstory has an amazing rage quit feature that is amazing for any designer. You can watch video session replay of people who were recently pissed off by your app or service.
Services we use for qualitative and quantitative data. User interviews happen in parallel.
So we use GA for basic numbers, Optimizely for A/B testing, Amazon QuickSight for dashboards, FullStory for qualitative and quantitative and Desk for customer support data.
Eventually it’s really cool to have your own dashboard with all the behavior data in one place.
5. Metrics for discovery and exploration
The way people use software and apps is constantly changing. Looking at trends helps you understand user behavior at a deeper level.
You can gain new insights and convert them into radical ideas and designs for the future of your product, six months or a year from now.
Discover new usability and functional bugs
Although the engineering team is always tracking performance, sometimes problems slip through the cracks. Mysterious changes and dips in numbers can be an insight into a new bug in your product.
6. Find your user centric metrics
Joel uses our new product CandyBar
Never lose sight of the person who uses your software. It’s crucial to listen to the rich stories behind the numbers so that there’s always context.
What is success for your customer?
For example at ReferralCandy our success metric is the amount of referral revenue we help our retailers earn.
For the consumer side of our service, its the reward the consumer gets. It could be a gift or cash reward.
So really think of the user centric metric that your entire product team can get behind and design for.
User centric metrics fits neatly into user centric design
Thank you for reading! This article is based on a talk I recently gave at Tech in Asia 2017.
You can also check it out on Medium.
Sequoia::Hack 2016
Super exciting to be part of the hackathon as a judge and mentor for the second year in a row. More than 5500 people applied this year to participate, and it was an amazing turnout. Inspiring to see so many teams come together to build something awesome in 24 hrs. And it is always a learning experience mentoring teams as they develop an idea into realization in a frantic rush to the finish line.
Super exciting to be part of the hackathon as a judge and mentor for the second year in a row. More than 5500 people applied this year to participate, and it was an amazing turnout. Inspiring to see so many teams come together to build something awesome in 24 hrs. And it is always a learning experience mentoring teams as they develop an idea into realization in a frantic rush to the finish line.
The "Slightly Mad" team talks to me about their foot scanner concept. Loved their design process and teamwork!
See this Instagram photo by @uxartist * 20 likes
By Day #2 everyone was sleep deprived, intense and focussed and the space started to reflect it :)
Only an hour left until judging begins...
It was a bit disappointing to see an all-male judging panel for the main development track in Seq Hack, but hopefully things will improve next year.
The day after the hackathon, there was terrible violence due to the Cauvery river water conflict. It was sad to see this especially just one day after the hackathon where so many futuristic concepts were shared, discussed and built. While a team in the hackathon won for building a robot that can help diagnose your illness, the streets the next day fought over water shortage, communal differences and drought issues.
