Prototyping Augmented Reality

Something that really frustrates me is that, while I’m excited about the potential AR has for storytelling, I don’t feel like I have really great AR experiences that I can point people to.

We know that AR is great for taking a selfie with a Pikachu and it’s pretty good at measuring spaces (as long as your room is really well lit and your phone is fully charged) but beyond that, we’re really still figuring it out. Because we’re in such an early stage of defining the paradigms and patterns that make compelling AR content, it’s important to have tools and techniques that allow you to quickly evaluate your ideas and iterate on them. At Knight Lab, we use a combination of 2D and 3D tools to test our AR design hypotheses. This gives us the opportunity to test concepts and refine them before we’ve invested significant resources.

A paper prototype is the easiest and most accessible way to begin bringing your ideas into the world. Paper is cheap and convenient. Collaborators don’t need to learn how to use a new tool or technology before contributing to a paper prototype. However, paper can only get you so far when you’re working in AR because it lacks the ability to convey information about depth and three dimensionality. Working on paper also encourages you to think in frames. You might think this is ok for mobile AR, since the AR content you’ll be looking at will be framed by your phone’s screen. But, the frame in mobile AR is kind of a lie. In mobile AR, the user decides where to put the camera. If you spend significant time prototyping with a static, framed tool like a piece of paper, it’s natural to fall into the trap of designing beautifully composed vignettes that might never be viewed from your intended angle. To encourage your team to think about an AR experience as something that can be approached from infinite perspectives, it’s sometimes helpful to make cut outs in the dimensions of common phones and look through them while walking around the space where you intend to generate your AR experience.

VR tools can also be useful for prototyping AR experiences, especially if you’re going to be modeling your own 3D content. Apps like Google Blocks and Tilt Brush allow you to quickly produce create 3D content and walk around it in 6 degrees of freedom. You can even take a 360° photo of the place where your content will be triggered and set it as your background in Tilt Brush. However, a VR prototype isn’t very easy to share and walking around with a heavy headset strapped to your face is a fundamentally different experience than being outside with your phone. VR is a great tool for experimenting with 3D assets, but it isn’t ideally suited for prototyping the overall vibe of a large scale AR project.

A screenshot of the Vuforia app triggering with a marker inside Knight Lab.
This 3D model might augment the succulent garden on the roof of the Ellis Arts and Recreation Center in Bronzeville. Here we use a VuMark marker to preview it in Knight Lab.
While working on a recent project we discovered that AR markers are a useful prototyping tool because they allow you to quickly add, delete and reorder scenes. Instead of using GPS, marker based AR relies on your phone scanning an image in order to generate an AR experience. While a GPS based experience might feel more “magic” to the user, it’s also a lot less reliable. This is especially true indoors.

This past spring, we collaborated with community members from Chicago’s Bronzeville and Pilsen neighborhoods to prototype a mobile AR app to help users discover the history, culture and environment of those neighborhoods. This prototype needed to fulfill many different functions for many different people. We needed to demonstrate the value of investing in Augmented Reality to potential funders with little experience with in immersive media. We also needed to represent the multilayered story our community partners who wanted to tell about their neighborhoods. Additionally, the project spanned a large geographic area--one that was about 18 miles from our lab in Evanston. Using markers allowed us to build an experience that was portable and gave a clear representation of both the possibilities of the technology and the content. Since you can recreate a marker based experience anywhere you have a set of markers, we could iterate on it from campus and it could also be recreated in a potential funder’s office. The lightweight nature of markers is useful for user studies, too. If a user’s attention is lagging, you can just pull markers out of their path. Think a certain AR event might be better in a different location? You can just swap the marker, there’s no need to open up Unity and build a new version of your app.

These are a few techniques that have worked for us but we’d love to know how you’re working in AR. Do you have a workflow that really works for you? Have you checked out any of the brand new prototyping tools? Send us an email at

About the author

Rebecca Poulson

Engineer, 2016-2020

Rebecca leads AR/VR projects at Knight Lab. In addition to being a software developer and playwright, Rebecca is an Oculus Launchpad Fellow and Mozilla Tech Speaker. She has taught WebVR workshops on three continents.

Latest Posts

Storytelling Tools

We build easy-to-use tools that can help you tell better stories.

View More