Using AI to enhance tourism

These images are no longer just fleeting memories of a glorious gap year but valuable material for tourism management and anthropological scholars. That’s thanks to AI built by researchers at Facebook and the University of Texas at Austin that analyzes thousands of public photos of various archaeological wonders in and around Cuzco, the historic capital of the Inca Empire, to explain how tourists engage with these sites and how that engagement changes over time.

It’s the first time that AI has been used at the intersection of anthropology and heritage tourism in this way, explains Kristen Grauman, a research scientist at Facebook AI Research and professor of computer science at the University of Texas at Austin. “We use a statistical model and computer vision to unlock what’s happening on the ground in terms of how tourists are experiencing a site,” she says. “It tells us how popular the sites are, how people are transitioning between them, and how much time they spend there.”

But the novel AI approach does not just measure footfall and activity along the Inca Trail. It also explains how tourists shape the special aesthetic legacy of cultural heritage sites. “It tells us about visuality — how tourists see a place,” says Grauman. For example, anthropologists could examine those Sun Gate shots in greater detail to see whether tourists are copying similar images they have seen on social media or consciously paying homage to Hiram Bingham, a famous American explorer who wore a wide-brimmed fedora. Over a century ago, Bingham photographed Machu Picchu from exactly the same spot using a special panoramic camera.

These kinds of ground-level insights might one day inform and influence policy decisions about tourism management in Cuzco or any of the world’s cultural heritage sites. This can help enhance tourist experiences, maximize the economic potential of the sites for the local region, and preserve places like Machu Picchu for future visitors, says Grauman. But this AI will also serve academics well. “It also gives anthropologists much more evidence for deeper scientific inquiry,” she explains. 

Cuzco offered the perfect blend of topography, archeology, and popular interest for this kind of project. Around 90 percent of all tourists that travel to Peru visit the city, which was declared a UNESCO World Heritage Site in 1983. Around half of these visitors fit in a trip to Machu Picchu, which is 73 kilometers away from Cuzco. Annual visits to the Cuzco region now top 3 million, an increase of more than 500 percent since 2000.

The rapid growth of tourism in the region led UNESCO to threaten to place Machu Picchu on the List of World Heritage in Danger over concerns of overcrowding and potential site damage, notes Nicole Payntar, a PhD candidate in the Department of Anthropology at the University of Texas at Austin and first author of a recent journal article covering the team’s results. “Knowing how tourists experience and move across heritage landscapes can translate into policy solutions that help ease issues of overcrowding and over-commercialization at popular heritage sites,” explains Payntar. These insights are especially timely as new infrastructure and economic recovery plans could increase the influx of tourists to Machu Picchu and Cuzco.

Machu Picchu may be the crown jewel in Cuzco’s archaeological heritage tourism circuit, but there are monumental Inca sites studded all over the city and its surroundings. Many tourists buy the Boleto Turístico del Cusco (BTC), a multisite pass that grants access to 10 other sites: Sacsayhuaman, Tipón, Tambomachay, Puca Pucara, Chinchero, Ollantaytambo, Pisac, Moray, Qenqo, and Pikillacta (a Wari site).

BTC sales, together with surveys, offer a snapshot of the volume of tourists across Cuzco’s main sites, but they cannot provide detailed, ground-level information about how long tourists spend at sites and how they move among them. Recent computational approaches that analyze archaeological spaces have revolved around remote sensing, which mainly uses aerial photography and satellite imagery to monitor sites. But these techniques generally capture landscape and site characteristics rather than explain the nature of tourist activity. 

But there is a medium that offers up context and insight about the tourist experience: social media. That is because there are mountains of visual, spatial, and temporal data in the billions of images shared by tourists on photo-sharing sites such as Flickr and available through their public API, the software that relays and retrieves user requests. This includes a wealth of metadata, such as URLs, geotags, and timestamps.

Grauman concedes that AI, social media, and the Incas may sound like an incongruous marriage. “Anthropologists and computer scientists do not come together often in their research goals, but here the match is essential to the interdisciplinary work,” she says. But she explains that these nuggets of data are fertile territory for AI that helps us make sense of the real world. “The way that tourists capture their photos also reveals consistent trends about the way that they see a site.” 

The research team (which also includes Alan Covey of the University of Texas at Austin and Kimberly Hsiao, affiliated with both the University of Texas at Austin and Facebook AI), built their data set by calculating the GPS coordinates of 12 sites: the 10 BTC sites, Machu Picchu, and the city of Cuzco. They plugged these coordinates into Flickr and used their public API to download the top 4,000 photos from each of the 12 sites over the 2004–2019 time period. The researchers expanded and organized the data set by querying all the unique user IDs in this initial batch of 48,000 photos and then retrieving the photos those users took during the time they visited any of the 12 sites. This approach gave them 57,804 images from 2,261 users. 

The geotags and timestamps in these neat photo albums don’t just allow the AI model to identify the relative popularity of each site by how frequently each one is photographed; they also build a picture of the way tourists move around the sites. The researchers used a Markov model to identify these patterns and the way they change over time.

“Aggregating over thousands of photos, the model captures the probability that tourists tend to move to site A given they were previously at site B, independent of the places they visited before,” explains Grauman. For example, tourists might start in Cuzco, then go to nearby Sacsayhuaman, and finally to Machu Picchu, while fewer visit the Wari site of Pikillacta. This transition probability matrix also allowed them to explore how transitions were affected by policies that regulated the heritage landscape, such as the release of new BTC packages in 2008, and new entry rules in 2017. 

The AI generated some insights that would not have been possible through an analysis of BTC sales. Visitors sometimes spent 10 hours at Machu Picchu but less than an hour at Tambomachay, Pisac, Chinchero, and Puca Pucara. They also tended to spend time at the same few sites before departing for Machu Picchu. The introduction of ticket packages in 2008 encouraged tourists to explore more sites through condensed day-trip itineraries. “These insights can inform policy, whether that’s about preservation or economic concerns,” says Grauman. “In the future, the AI could identify patterns not just between sites but within individual ones too.” This type of data may also provide new solutions for heritage managers and local communities looking to protect the past while maintaining the socioeconomic benefits of tourism, adds Payntar.

AI can also generate rich anthropological insights from the photos. The researchers adopted a deep convolutional neural network (CNN) that was pre trained on ImageNet, a database with more than 10 million images labeled with 1,000 common object categories. The CNN extracted semantic features from images, such as color and gradient orientation, before using a clustering algorithm to discover common themes at each site: grassy terraces, blue sky, drystone walls, alpacas, and mountains.

These aesthetic preferences are valuable to anthropological scholars, who study the way that people engage with heritage. Knowledge, wisdom, and ideas can of course be passed on via oral traditions in the form of folktales, songs, and poetry, but heritage is also shaped by the way we see, a theory that anthropologists call visuality. “The clustering approach allows us to quantify the extent of homogeneity or variety in the way people are seeing sites,” says Grauman.  

In other words, the common themes at heritage sites will tell anthropologists whether tourists are imitating the curated images of alpacas they have seen on social media, or replicating the experience of the original Machu Picchu influencer: Bingham and his iconic shot from the Sun Gate. Either way, the AI is generating new insights about Cuzco’s constantly evolving aesthetic legacy, and what Machu Picchu means to the world over time. 

That is not something that human anthropologists can easily measure. “Modeling and measuring visuality, or the way that iconic photos are being captured and framed, is a real challenge,” says Grauman. “People can’t analyze tens of thousands of photos in a consistent, objective way, but machines can do this pretty easily,” she adds.  

Tourism management might seem like unfamiliar territory for Facebook AI researchers, but it’s part of a wider goal to build AI systems that enhance our lives.  While many of Facebook’s research initiatives are deployed to products — such as using AI to connect blood donors to blood banks in dire need — they also leverage scientific breakthroughs to help people in the real world. This includes generating detailed maps of the world so nonprofits can deliver aid in rural areas, and working to help advance the United Nations’ sustainable development goals. 

“This is similar,” says Grauman. “We’re leveraging online information to help us understand the offline world.”