Make Samsung’s AR Emoji expressive with HelloMoji!
HelloMoji is an AR Emoji avatar app designed to enable users to express themselves through animated avatars using AI-driven emotion extraction, lip-sync, and body gestures. This project marked Samsung R&D's first consumer product launch in five years.
HelloMoji enables users to express themselves using AR Emoji avatars according to:
- Their own voice, which enables avatars to lip-sync
- AI extracted emotion based on the user's message, which enables facial and body gestures of avatars
Validating the initial prototype
A PoC effort was launched in 2022 to create an avatar video app where your voice could animate your avatar video. Upon PoC completion in Q4, I designed a user study to validate the technology readiness and concept adoption.
Goals of the discovery period
- Determine if the AI avatar technology the engineering team developed is accurate and realistic.
- Gauge our target demographic's sentiment towards an avatar video app
10 employees were selected based on their age group (preference for Gen Z and Millennials), their current messaging sticker usage, and their current voice message usage. They completed a screener survey and then completed an in-person 30 minute trial.
During the trial, they were asked to:
- View an avatar video created with HelloMoji
- Record their own HelloMoji video
- Edit their HelloMoji video
At each of these three steps, participants responded to questions about the quality of the avatar's lip-sync and expressions. When creating and editing their video, I also asked about the usability of key features like starting/stopping a recording, changing the emotion, changing the body animation, and gathered additional input for other needed features.
From the study, I derived three takeaways:
- Participants who were able to try out many different animations were more satisfied with their stickers. To enable people to enjoy the different face and body expressions available, users needed an easy way to preview body animations.
- Although 10 out of 10 users complained about low lip-sync accuracy, 9 out of 10 thought facial and body expression was more important. Only 2 out of 10 participants said that lip-sync accuracy was more important than naturalness. This meant that to create the most valuable product for users, we should focus on creating more entertaining body gestures instead of having perfect lip-sync.
- 6 out of 10 participants had a complaint about avatar eye movement. 3 participants thought the avatar was not looking at them, 2 thought the avatar was looking above them, and 4 thought the avatar eyes generally lacked movement.
Based on the survey results, features that would lead to the highest level of customer satisfaction were created and prioritized based on the number of people who complained about each issue.
Design Iterations
Based on the internal survey, I defined which features should be put into each sprint. The most critical features were designed first, while the last sprint was dedicated to polish. The design effort was broken down into eight sprints total lasting about 2 weeks each. Each sprint's deliverable was a UX guide and a redline document along with any UI assets (if applicable), these documents were reviewed and implemented by Engineering. I also collaborated with QA to validate their test cases and perform QA on the UX/UI components.
The final HelloMoji app contains the following core features:
Preset and custom avatars
Users can choose from preset avatars. The non-human avatars are re-used from the Samsung Kids app. I designed each human avatars based on trends observed with Gen Z and Millennial users in the target launch countries (US and Korea).
Users may also create a new avatar by pressing the '+' icon in the landing page.
Voice recording
Once an avatar is selected, the user may record a voice message under 30 seconds.
Emotion and body gesture selection
Emotion is extracted based on your voice message, and various emotions and body animations can be added to bring your avatar to life.
Background selection
Videos are enhanced by selecting the perfect background to match the animated avatar.
Saving and sharing
Once the video is complete, it can be saved or shared directly on your favorite apps.
App Launch
Launching to the Galaxy Store
App Icon
The HelloMoji app icon was initially designed with the goal of showcasing AR Emoji in a 3D manner, while also subtly showing that this was a voice-powered app. To validate this icon, I created two internal surveys sent to various departments at SRA to gauge preference for a female-led vs. male-led icon, and to determine a preferred color for the icon.
The icon was then reworked for V1.02.00 with the following goals:
- The color must be trendy and match Samsung's current branding
- The composition must be original, while showing avatars relevant to our demographic countries
- The design must clearly showcase the voice capture feature
To further select the color, I looked at the Pantone 2024 colors and selected rust, teal, and violet colors to ideate with as well.
After the colors and avatars were selected, I mixed and matched the different options to find the optimal combination. I relied on shadow and other effects to further polish the look of the app icon.
The final icon was selected from the results of a confidential internal survey to designers, engineering, and technical artists.
Galaxy Store Images
Impact
Despite launching with zero marketing spend, HelloMoji currently has a 4.5/5 star rating on the Galaxy Store. This success is a testament to the app's intuitive design and the joy it brings to users. To promote the app, I created all marketing materials, including email blasts to Samsung employees, Galaxy store banners, and social media posts using Adobe Photoshop, Adobe Premiere, and Figma.