Prototyping Augmented Reality

Something that really frustrates me is that, while I’m excited about the potential AR has for storytelling, I don’t feel like I have really great AR experiences that I can point people to.

We know that AR is great for taking a selfie with a Pikachu and it’s pretty good at measuring spaces (as long as your room is really well lit and your phone is fully charged) but beyond that, we’re really still figuring it out. Because we’re in such an early stage of defining the paradigms and patterns that make compelling AR content, it’s important to have tools and techniques that allow you to quickly evaluate your ideas and iterate on them. At Knight Lab, we use a combination of 2D and 3D tools to test our AR design hypotheses. This gives us the opportunity to test concepts and refine them before we’ve invested significant resources.

A paper prototype is the easiest and most accessible way to begin bringing your ideas into the world. Paper is cheap and convenient. Collaborators don’t need to learn how to use a new tool or technology before contributing to a paper prototype. However, paper can only get you so far when you’re working in AR because it lacks the ability to convey information about depth and three dimensionality. Working on paper also encourages you to think in frames. You might think this is ok for mobile AR, since the AR content you’ll be looking at will be framed by your phone’s screen. But, the frame in mobile AR is kind of a lie. In mobile AR, the user decides where to put the camera. If you spend significant time prototyping with a static, framed tool like a piece of paper, it’s natural to fall into the trap of designing beautifully composed vignettes that might never be viewed from your intended angle. To encourage your team to think about an AR experience as something that can be approached from infinite perspectives, it’s sometimes helpful to make cut outs in the dimensions of common phones and look through them while walking around the space where you intend to generate your AR experience.

VR tools can also be useful for prototyping AR experiences, especially if you’re going to be modeling your own 3D content. Apps like Google Blocks and Tilt Brush allow you to quickly produce create 3D content and walk around it in 6 degrees of freedom. You can even take a 360° photo of the place where your content will be triggered and set it as your background in Tilt Brush. However, a VR prototype isn’t very easy to share and walking around with a heavy headset strapped to your face is a fundamentally different experience than being outside with your phone. VR is a great tool for experimenting with 3D assets, but it isn’t ideally suited for prototyping the overall vibe of a large scale AR project.

A screenshot of the Vuforia app triggering with a marker inside Knight Lab.
This 3D model might augment the succulent garden on the roof of the Ellis Arts and Recreation Center in Bronzeville. Here we use a VuMark marker to preview it in Knight Lab.
While working on a recent project we discovered that AR markers are a useful prototyping tool because they allow you to quickly add, delete and reorder scenes. Instead of using GPS, marker based AR relies on your phone scanning an image in order to generate an AR experience. While a GPS based experience might feel more “magic” to the user, it’s also a lot less reliable. This is especially true indoors.

This past spring, we collaborated with community members from Chicago’s Bronzeville and Pilsen neighborhoods to prototype a mobile AR app to help users discover the history, culture and environment of those neighborhoods. This prototype needed to fulfill many different functions for many different people. We needed to demonstrate the value of investing in Augmented Reality to potential funders with little experience with in immersive media. We also needed to represent the multilayered story our community partners who wanted to tell about their neighborhoods. Additionally, the project spanned a large geographic area--one that was about 18 miles from our lab in Evanston. Using markers allowed us to build an experience that was portable and gave a clear representation of both the possibilities of the technology and the content. Since you can recreate a marker based experience anywhere you have a set of markers, we could iterate on it from campus and it could also be recreated in a potential funder’s office. The lightweight nature of markers is useful for user studies, too. If a user’s attention is lagging, you can just pull markers out of their path. Think a certain AR event might be better in a different location? You can just swap the marker, there’s no need to open up Unity and build a new version of your app.

These are a few techniques that have worked for us but we’d love to know how you’re working in AR. Do you have a workflow that really works for you? Have you checked out any of the brand new prototyping tools? Send us an email at knightlab@northwestern.edu.

About the author

Rebecca Poulson

Engineer, 2016-2020

Rebecca leads AR/VR projects at Knight Lab. In addition to being a software developer and playwright, Rebecca is an Oculus Launchpad Fellow and Mozilla Tech Speaker. She has taught WebVR workshops on three continents.

Latest Posts

  • A Big Change That Will Probably Affect Your Storymaps

    A big change is coming to StoryMapJS, and it will affect many, if not most existing storymaps. When making a storymap, one way to set a style and tone for your project is to set the "map type," also known as the "basemap." When we launched StoryMapJS, it included options for a few basemaps created by Stamen Design. These included the "watercolor" style, as well as the default style for new storymaps, "Toner Lite." Stamen...

    Continue Reading

  • Introducing AmyJo Brown, Knight Lab Professional Fellow

    AmyJo Brown, a veteran journalist passionate about supporting and reshaping local political journalism and who it engages, has joined the Knight Lab as a 2022-2023 professional fellow. Her focus is on building The Public Ledger, a data tool structured from local campaign finance data that is designed to track connections and make local political relationships – and their influence – more visible. “Campaign finance data has more stories to tell – if we follow the...

    Continue Reading

  • Interactive Entertainment: How UX Design Shapes Streaming Platforms

    As streaming develops into the latest age of entertainment, how are interfaces and layouts being designed to prioritize user experience and accessibility? The Covid-19 pandemic accelerated streaming services becoming the dominant form of entertainment. There are a handful of new platforms, each with thousands of hours of content, but not much change or differentiation in the user journeys. For the most part, everywhere from Netflix to illegal streaming platforms use similar video streaming UX standards, and...

    Continue Reading

  • Innovation with collaborationExperimenting with AI and investigative journalism in the Americas.

    Lee este artículo en español. How might we use AI technologies to innovate newsgathering and investigative reporting techniques? This was the question we posed to a group of seven newsrooms in Latin America and the US as part of the Americas Cohort during the 2021 JournalismAI Collab Challenges. The Collab is an initiative that brings together media organizations to experiment with AI technologies and journalism. This year,  JournalismAI, a project of Polis, the journalism think-tank at...

    Continue Reading

  • Innovación con colaboraciónCuando el periodismo de investigación experimenta con inteligencia artificial.

    Read this article in English. ¿Cómo podemos usar la inteligencia artificial para innovar las técnicas de reporteo y de periodismo de investigación? Esta es la pregunta que convocó a un grupo de siete organizaciones periodísticas en América Latina y Estados Unidos, el grupo de las Américas del 2021 JournalismAI Collab Challenges. Esta iniciativa de colaboración reúne a medios para experimentar con inteligencia artificial y periodismo. Este año, JournalismAI, un proyecto de Polis, la think-tank de periodismo...

    Continue Reading

  • AI, Automation, and Newsrooms: Finding Fitting Tools for Your Organization

    If you’d like to use technology to make your newsroom more efficient, you’ve come to the right place. Tools exist that can help you find news, manage your work in progress, and distribute your content more effectively than ever before, and we’re here to help you find the ones that are right for you. As part of the Knight Foundation’s AI for Local News program, we worked with the Associated Press to interview dozens of......

    Continue Reading

Storytelling Tools

We build easy-to-use tools that can help you tell better stories.

View More