IRONMAN AR
Instagram Filter
Spark AR, Blender, After Effects, Javascript
2020 | Fackbook AR Hackathon
IRONMAN AR is a humorous Instagram filter that enables users to wear the Ironman’s suit and shoot the laser.
Role
Concept and interaction design
Front-end development
VFX
Inspiration
Due to COVID-19 pandemic, more and more people are working from home and especially family members are having unexpectedly hard time as they have to deal with each other much more frequently than before. In this situation, I and my co-worker Vincent thought "how we can create a fun AR experience that people can relieve their stress from their counterpart?" This question led us to imagine wearing the Ironman suit through AR, and we wanted to give opportunities that users gain Ironman’s super power in their daily lives and have fun moments with their cute enemies.
What it does
Using face recognition on the face-facing camera, you are able to wear the Ironman’s suit that includes the shooting ability. Afterward, you need to use the backside camera by switching a camera in order to shoot the magic laser to another’s face. When you tap and hold on the screen, the laser will be launched on the detected face of the cute daily enemy. The embedded face landmark will recognize the detected face to apply magical effects through AR laser.
How we built it
As for the suit-wearing scene, we have imported a green screen processed png sequence and replaced its face part with the user's face which is animated along the sequence.
Next, we used body segmentation and made the background very dark in order to show the user's face is inside of the suit.
Lastly, we added a face tracked HUD (head-up display) and synchronized the rotation with the user's face. The laser of the back-facing camera is triggered by long-pressing anywhere on the screen and made with the built-in particle system. For the face dividing effect when the laser is hitting the target, we previously divided a typical face mesh into the top and bottom part in Blender and exported so that we ended up animating each part of the head differently and added supporting png sequences for the explosion effect.
Challenges we ran into
Since we were running hundreds of images and audio files like wearing the suit, HUD, and explosion, putting everything in a limited space was a big deal. We had to try using different PNG image optimizers several times for converting the PNG images into the smallest size and finally succeeded to modify the color bit precisely.
Accomplishments that we are proud of
We have been focusing on the storyline in a certain context, unlike the other AR function. Beyond the facial effect, our idea has a context that is broadly being used and the generally happening situation in our daily lives. We believe that those attempts make this service a more user empathic AR service with familiar hero characters. This sort of storytelling driven AR effect will facilitate users to use the service more enjoyable and richly with others in various situations.
Results
This filter recorded 65.2K Impressions, 35K Opens and 4.8K Captures over 2 years!