How OurWorlds Uses RICOH THETA to Extend Reality

We met Catherine Eng, OurWorlds Co-Founder and CTO. OurWorlds is a communication platform using extended reality to transform public and private spaces into interactive environments. Catherine is a designer and filmmaker by training, and learned to code early on. She loves to teach kids how to code and believes that with a strong design and a cool project in mind, coding is even more amazing! Please check out how she uses the RICOH THETA V and RICOH THETA Z1 for her work!

Can you tell us how you prefer to use 360-degree photography in Our World's projects?

We see 360-degree videography as an important part of the sets of narratives recorded by tribal communities, or by us, all over the country. 360-degree videography is an integral way to capture the context of the places where these narratives come from. Sometimes within the media we superimpose on top of the video, providing even more meaning and context. Place-based, narrative traditions are a strong part of indigenous storytelling and the way the land looks, feels, and sounds, is what we try to capture with 360-degree photography. We see all of our recordings, augmented reality objects, volumetric video holograms, and 360-degree video - as an integral part of the record of Native America.

One of the things we are striving to do is to work with foundations who can help us find a way to preserve these, for future generations to see. The content we capture are, in themselves cultural objects, and worthy of being maintained, and future-proofed, much in the way that works of art in a museum are kept in climate-controlled environments. We think that the types of images, videos, and recordings we are making are critical to understanding culture. Why are there so many objects in museums? It's because in many cases, that is all that is left of a culture. Historians, archaeologists, and even geologists have to guess at what the environments and contexts were, in which these objects were used. Here, we have the opportunity to capture those very contexts using 360-degrees; along with the objects themselves, while also using augmented reality and narratives from the knowledge-keepers who know about them in volumetric video. It's imperative that we find ways to preserve and future-proof these digital media, so that they can keep being viewed on the next = generations of platforms. 

Our work with Dr. Rodriguez, Advisory Board on the OurWorlds, gives us insights about how to work with cultural leaders, what the technology means to them, and what their needs and ideas about the technology are, so that as we reach out to more communities and leaders around the country, we can better serve them to help them preserve and share their stories. We are pioneering technology and workflows that are sustainable, using devices and cameras like RICOH THETA that are affordable, sharing these technologies and workflows with the community. We never want to silo our technology from indigenous creators, or be gatekeepers of the stories. We want others to tell their own stories - and share them on our platform, OurWorlds, where we have a patent-pending technology that can present these multi-dimensional, cultural stories in XR in place-based contexts. We want to make meaningful XR storytelling as accessible as possible. The ease of use and affordability of the 360-degree cameras are an important part of that.

We love how the RICOH THETA can be used to tell the stories of so many. How did you discover RICOH THETA? 

I first learned about the RICOH THETA at the Reality Virtually Hackathon at MIT in 2017. I worked with a team that created a 360-degree project called "Homeward" to help first responders, and we won the Ricoh THETA prize at that event. Later, after Kilma, the co-founder of OurWorlds, and I began collaborating on OurWorlds, we had built a platform that allowed us to drop various stories, videos, photos, and AR objects through a map-based interface to a particular location, based on the historical culture of that place. Sort of a "reward" for people who took the time to go out into nature, to different locations, to discover history. But then we thought, what if people couldn't get there, practically-speaking? It became clear that the magic of some of these places could be captured by the 360-degree camera. RICOH THETA had come a long way, resolution-wise, since 2017... I ran out to BestBuy that night, and bought a THETA V, and the rest is history.

We’re so glad that you discovered RICOH THETA! What do you like about the THETA V and Z1? Did you find any differences between the two? 

While we loved the RICOH THETA V, the THETA Z1 was an improvement for our field work, and our ultimate mission - which is to preserve these places for posterity, so that all can see them. Our platform works on mobile, desktop, and headsets - and we know those resolutions are just going to keep getting better and better. We strive to capture in the highest resolution possible so these recordings remain crisp, and beautiful, even viewed on future platforms. The RICOH THETA Z1 is great because the battery life is solid and has fast transfer speeds too.

We agree - we love the RICOH THETA Z1! Were you able to identify which scenes are best for the THETA? How did you photograph these scenes before you began using 360 cameras? 

The vast majority of the footage we have so far is recorded in nature on a tripod. It is a great way to preserve the way places in nature looked and felt at a particular time. For OurWorlds, 360-degree video has always been a part of our platform. Before we realized all the ways we could use this footage, I would say our platform would have relied upon telling history with 2D / 3D holograms of people and objects - rather than also providing immersive 360-degree video. Some of these places are hard to get to.. imagine being inside of a 12-foot high stand of reeds, in a forest, or in a traditional Kumeyaay village surrounded by willow shelters!

We can only imagine how tough that used to be! What has been useful for your workflow when using THETA for Our Worlds?

The iPhone app for the RICOH THETA cameras is super useful. The app helps us with checking recordings and setup, giving us time to get out of the way and ensuring the photo will be correct. Right now our needs are simple because we know more about how people use OurWorlds. They are looking for quick experiences that enhance their presence in an environment or they want a quick answer about a particular place. Because of this, we have been shooting shorter form videos which is easy on both cameras - though the THETA Z1 holds more. We always carry a laptop and G-Drives* when we go out and shoot in nature as well in case we have to do quick downloads to free up space on the cameras.

‍

*G-Drives are made by SanDisk and are super durable, storage-intensive drives with high-speed data transfer rates which make them great for audio/video editing on location.

We also love the RICOH THETA apps! They make using the cameras so simple. Do you use the app for any photo editing? If not, how do you edit your photos? 

We mostly use the cameras for video rather than 360-degree photo experiences, though OurWorlds can display both kinds of experiences. For video editing, the first step of course is to stitch the hemispheres, which we do with the THETA app. When we use Adobe After Effects and Adobe Premiere to edit, with Boris FX plugins - mostly Mocha and Continuum. After we grab the cuts we want, we touch out the monopods and camera people. On some of our activations, we layer in 3D objects and volumetric videos we have exported from Unity through a proprietary process we have created which we will share soon! This makes it possible for people anywhere in the world to experience what it is like being in a particular place, and seeing the holograms which we have pinned there. It is really nice to have the audio recording feature from the cameras as well. To clean up the audio, we use Audacity and a couple of filter sets we have configured just for this purpose through trial and error.

Excellent! Building on that - how do you use the images to create a VR experience that tells Native American history? 

We sent the camera out to Choctaw Nation and Judy Allen, their Historical Projects Manager, was able to use the camera and tripod to record about 10 minutes of footage at the Choctaw Veterans Memorial. It is a beautiful, peaceful place. At the center of it is a large commemorative marker with the veterans' names inscribed. We have an activation in our app called "Hear Choctaw Code Talkers", which gives viewers the experience in 360-degree video of being at the present-day memorial site in Oklahoma superimposed with our holograms of the soldiers as they would have appeared and sounded in WWI France in 1918. 

The beauty of using 360-degree cameras like the RICOH THETA V and THETA Z1 in our workflow is that the footage they capture drives home the point that Native American culture is still here with us today - the history, the stories, and the values, are woven into our modern landscapes. The 360-degree recordings and stories we create help to unlock that, so that everyone can find deeper meaning in the places where they live, work, and play. We capture these modern-day landscapes on the Ricoh THETA cameras; and superimpose narratives from tribal leaders, elders, language experts, artists, and storytellers, in holographic form to provide these additional layers of meaning. These locations don't even have to be just in the U.S.! Kilma recently went to Algonquin territory up in Ontario, in Canada. Some of the 360-degree footage we shot for Choctaw Code Talkers 1918, which is a story that takes place in France, was shot on a RICOH THETA V in the forests and places where one of the last battles of WWI took place. Native Americans contributed to culture and history all over the world. We have several additional stories planned that will take place in Europe; following the story of a young Payòmkawichum priest named Pablo Tac; and the story of the Iroquois and Navajo soldiers serving in the U.S. military in WWII Europe. 

More recently, Dr. Stan had a series of reed canoe launches here in coastal San Diego - the last one was at Silver Strand State Beach on Coronado, and featured 50 traditional "ha kwaiyo" or tule reed boats, made by the community! You will see footage of me out in the water - and also some of the footage I shot with a monopod, with the THETA Z1 mounted on top - while paddling the canoe with one arm! I am a rower, so that training came in super handy here. There is really something magical about being amidst so many canoes, out in the water, and the energy of how excited and happy everyone was to be able to participate. 

That sounds incredible. Are there any exciting uses that you’ve found while using RICOH THETA for your work?

One of the most exciting uses of the technology is combining the 360-degree videos of places with Native language - you will see in some of the stills, we have the word for "sky" or a definition of "Kumeyaay.” A big part of what we do is using these emerging technologies for language revitalization, so you will start seeing a lot of that in the immersive 360-degree videos we capture of landscapes. It is a great way for people to learn about these places, see and hear the indigenous languages of these places, and to understand these languages as part of the everyday experience of being in these locations.

It’s great that 360-degree videography has the ability to capture so much of a landscape and share with others to make them feel like they’re right there in the moment. When you’re shooting, are there any accessories you use with RICOH THETA?

I mainly use monopods. I also love that the cameras fit inside a snug neoprene sleeve. I just toss the cameras into my backpack and run out to go film at the beach, in the mountains, etc. They are the one type of camera I have that I worry about the least - even less (or maybe especially less) than my iPhone.

Yes - it is quite a simple camera! What do you think about the design of RICOH THETA? 

It is great that the RICOH THETA cameras are super easy to use and portable. The big button on the front makes it simple to just push “record!” When we are working with more tribal community members around the country mailing out our loaner THETA Vs with a monopod, it’s simple to explain the shooting process to them and provide them with self-addressed packaging so they can mail back the camera once they've made their recordings. This is a great way to get multi-generational buy-in - for people of all ages to feel like they can participate and do something cool even if technology workflows aren't a part of their everyday life. More tech-savvy users would be able to download the app, and get the footage onto a computer and a cloud drive, but we don't require it. We are working with a few younger people, who share our vision of using technology to advance their culture. It is our goal to establish a technology fellowship where we provide our fellows a set of hardware and software tools, mentorship, access to our proprietary backend, and access to support from a Creator Fund to help support indigenous storytellers who are interested in developing their XR craft. This is an important part of our mission. We are a benefit corporation, and it is written into our company charter to uplift and amplify indigenous voices using technology. 

‍

You can follow OurWorlds on Facebook, Twitter, LinkedIn, and Instagram to stay up to date on the exciting projects they’re creating with RICOH THETA!
‍

Catherine's 360-degree videos: https://ourworlds.io/putting-code-to-culture-and-rediscovering-san-diego/