Mapping the Future with Big Data
By Patrick Tucker
A little-known California company called Esri offers a “Facebook for Maps” that promises to change the way we interact with our environment, predict behavior, and make decisions in the decades ahead.
The setting is central California’s Yosemite National Park. A hiker, let’s call him Steve Clark, has gone missing on one of the trails. As the head park ranger, your job is to lead a search-and-rescue mission to find him. All you have to go on is the point where he was last seen, your training, and a computer; from this, you have to predict the behavior of a lost hiker. Sunset is approaching, and in some parts of the park the temperature will be below freezing in a matter of hours. What do you do?
Many experienced hikers know that the recommended course of action when lost is to follow a stream downhill and this will eventually lead to civilization. But you can’t assume that Steve Clark is aware of this, or that he’s even seen the Discovery Channel. He might elect to stay put, or, if he has a cell phone, he might be moving uphill to find a signal. You also don’t know if he’s injured. A person with a sprained ankle is less likely to walk up, but he may not move down, either.
You go to your computer and open ArcGIS.com. A computer map of Yosemite that you’ve made and uploaded appears on the screen. Let’s say you also have access to a “big data” database of records from 30,000 lost hiker search-and-rescue missions and you can query this database with key words.
You soon learn that 66% of lost hikers are found within two miles of the spot last seen. You impose a ring over your map reflecting this two-mile perimeter. You then learn that 52% of lost hikers are found downhill, only 32% go up, and 16% keep walking at the same elevation. You impose an elevation layer on the area with all the land above the last point seen shaded one color and the land beneath it shaded another. You can even impose a new lens depicting tree and plant cover and open fields, and one depicting linear objects like trails, roads, power lines, and streams, knowing that the vast majority of lost hikers follow some sort of linear marker to avoid going in circles.
Querying the big database of records, you discover that wayward hikers usually quit trekking after about three hours. You can now create a predictive model. It’s not an absolute location but a priority list of places to check for the lost hiker. When you run the model through ArcGIS, you get three new concentric circles depicting Steve’s most likely location based on a statistical analysis of what’s in the database. But the possible area is still too big.
You need more help, so you call a group of retired Yosemite park rangers—men and women who know the back trails of that area of the park better than anyone, including the wooded nooks and crannies where hikers are most likely to get lost. But the trails and twists through the dense forest are nameless and hard to describe; you just have to know them. You share the map with your ranger network and activate a feature that lets them color the portions that they would hit first. The potential search area has now shrunk down considerably.
Next, you share the map with the public, targeting people who mentioned on Twitter and Facebook that they would be hiking in Yosemite that day. You enable these people to place points on the map where they saw something that might be a clue, like a shoelace or an article of clothing. You take every piece of information you receive and add it to your predictive model.
A new map emerges. The area you now have to search is several times smaller than what you were looking at an hour ago. You’ve successfully predicted where your hiker is going to be by the time you catch up with him. You’re going to find Steve Clark before the sun goes down.
The above hiker scenario is one that Esri (originally Environmental Systems Research Institute Inc.) demonstrates at conferences, such as its Federal GIS user conference that took place in February. It is, in many ways, a snapshot of the way that statistical data from databases, user data from multiple participants, and social network data from the public will change the nature of rapid decision making in the years ahead. It’s a very big change, and Esri is at the forefront of the way big data and geography will merge in the future.
You’ve probably never heard of Esri, but if you work for a large enough institution, a company that changes or manages the physical landscape in any way, your boss has heard of it. To understand what this change means, you have to take a journey about 60 miles east of Los Angeles, to a little town called Redlands.
If you were to meet the man who happens to occupy number #554 on the Forbes list of richest people in the world, you might easily mistake him for a librarian. Jack Dangermond is a very untypical billionaire. Although his fortune comes from software, he’s escaped the notoriety of Mark Zuckerberg or Peter Thiel and has none of the egoistical flamboyance of, say, Larry Ellison. He’s quiet, grandfatherly, unpolished, and will address a room full of people in the same way he speaks to individuals—with a simple, authentic attentiveness.
Although his company, Esri, is in the background of the way governments, major corporations, and NGOs around the world plan and understand physical space, few know he exists. Atlantic writer James Fallows, who has known Dangermond his whole life, has called him “one of the world’s secret plutocrats.”
Dangermond just calls himself a landscape architect.
Esri helps its clients make sense of physical space with computational assets. “When we conceived of this organization, the question was, Can we apply computer techniques and systems techniques to mapping techniques?” Dangermond told me when I visited the Esri campus in the fall of 2012. “Mapping is thought of by many people as what we do, but it’s really just the facade. It’s the modeling, the digitizing of the planet, and building of geographic relationships—that’s the real purpose of the company. We thought of that years ago when we started. We said, ‘Can we build this into a business so that it becomes a kind of organism that pushes towards improving human understanding of the environment?’”
Computerized mapping may not sound terribly exciting until you consider just how pervasive that need is and how big a player Esri is in it. With revenue of $818 million a year, it owns 47% of a $1.65 billion-a-year market for what is commonly called geographic information services, or GIS. Esri helps companies like Capital One and Starbucks plot where to place new store locations on the basis of current and future demographics, and do so on a block-by-block level. Every major U.S. oil company uses Esri to plan oil operations, which helps them act more efficiently. Environmental groups like the Nature Conservancy also use Esri to track wildlife patterns and map nature preserves.
Esri’s biggest client, representing about half of its business, is the U.S. federal government. The EPA, HHS, DHS, DOD, DOI, and 25 other departments use the company’s software for location analysis and real-time information gathering. Commanders in the U.S. Marines use Esri products to plan and coordinate troop movements, and Arlington National Cemetery uses Esri maps to help relatives of deceased solders find their loved ones in Arlington’s 400,000-plot burial park. But this is just a small sampling. Some 350,000 organizations around the world use Esri software to make maps for thousands of purposes.
“One of them is USGS. One of them is National Geographic,” says Dangermond. “One of them is the state of Maryland. The city of Los Angeles is one, [and] the city of Beijing [and] the city of Abu Dhabi. You go through those places, and millions of people are doing that every day.”
When you move through a planned government facility, through an airport, through a shopping center, when you pass a strip mall on your way out of town and then climb a peak in your favorite nearby national park, the influence of Esri is literally all around you.
In the nascent era of big data, Esri is poised to become much more significant as we incorporate computerized sensing and broadcasting abilities into our physical environment, creating what is sometimes called an “Internet of things.” Data from sensor networks, RFID tags, surveillance cameras, unmanned aerial vehicles, and geotagged social-media posts all have geographical components to them. After decades of quietly serving the computer mapping and modeling needs of its clients, Esri has suddenly found itself in a new field, using geo-specific data to reveal how businesses, institutions, populations, and entire nations are changing—or being changed by—the physical world, in real time.
Disaster management, in particular, is an area where maps updated with live, streaming data can make a big difference. The Pacific Disaster Center (PDC) in Hawaii used the ArcGIS platform to build a smartphone app called DisasterAWARE that “integrates near-real time hazard information with infrastructure and population data in a geospatial environment to allow decision makers to quickly assess and react to disasters,” as PDC head Chris Chiesa explained at a 2011 conference. The app offers a hazards and dangers snapshot of the world as it exists in the moment. It’s the sort of tool that a manager charged with having to evacuate a town in the path of a tsunami or hurricane—or deploy a relief team to recently hit area that might still be in the line of danger—would find invaluable.
A military expression for this sort of capability is “situational awareness,” a term that engineer Mica Endsley coined in 1995 to refer to “the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future.” In techno-thriller television shows like NCIS, Person of Interest, and Criminal Minds, situational awareness is the command center view that the stereotypical “geek” character employs to solve the crime, find the bad guy, and provide live back-up while the good-looking actors slink around with their pistols drawn. It serves as a sort of superpower, one that’s showing up in real command centers, on tablet PCs, and on the smartphones of Esri’s customers.
“Web maps, basically, are a visual expression of all this information of change,” says Dangermond. “I’m a bit biased in this respect; I think actually all information is geographic in one way or the other.”
There’s a reason Google Maps is a household name but GIS is still obscure, even though it provides much more location-based context than does most of the map data we access from our smartphones.
“We’re not a consumer-facing company,” says Dangermond. When people pull out their smartphones, most are only looking for enough location information to get to their appointments on time or select among competing coffee shops or hotels. GIS is for planning, which, in the minds of most people, is something to be done by government, by companies, by institutions with the resources to hire experts. What Esri is doing, quietly, is allowing these experts across fields to look over one another’s shoulder.
The Facebook for Maps
In 2010, Esri launched a new Web site, ArcGIS.com, which allows its users to share their maps in a cloud environment with anyone they choose. The maps become available across devices, from desktop to smartphone, and can be programmed and customized by outside developers and users the same way developers make their own apps through application programming interfaces, or APIs.
Importantly, unlike GPS maps that you might access through Google, the user-generated ArcGIS maps aren’t just about location; they’re about context. A conservationist working in Northwest China and a political scientist in Washington, D.C., can access aquifer, census, and historical maps created by other users around the world. The result is a sort of social-networking hub for geographic information.
“This is kind of like Facebook for geography, where our users are sharing their maps, or connecting their maps, into a cloud environment,” says Dangermond.
It’s a huge step forward in terms of the way most people understand place. But that doesn’t mean that expanded GIS awareness, especially in the hands of a few, is without controversy. Putting items on a map can make complex situations appear more simple than they in fact are. Here’s a case in point.
X Marks the Scene of Future Crime
In 2008, Atlantic writer Hanna Rosin wrote an article discussing the role that Section 8 housing, which is government-subsidized housing for individuals at a low income level, seemed to be having on crime in the city of Memphis. As part of her research, she interviewed Memphis crime expert Richard Janikowski and housing expert Phyllis Betts. Janikowski had been working with people in the local Memphis government to develop more data-driven approaches to crime fighting, and so he had a big crime hotspot map.
With Rosin looking on, Janikowski and Betts superimposed the Section 8 housing map, showing the areas where people lived in federally subsidized housing, on top of the crime map. Here’s how Rosin described it: “On the merged map, dense violent-crime areas are shaded dark blue, and Section 8 addresses are represented by little red dots. All of the dark-blue areas are covered in little red dots, like bursts of gunfire. The rest of the city has almost no dots.”
The conclusion, and the provocative suggestion that runs throughout the article, is that two of the most important antipoverty programs ever enacted, HOPE VI and Section 8, were causing crime. The rising Memphis murder rate “implicated” Section 8 in particular.
Scholars in the field of public policy were quick to point out that Rosin was mistaking correlation—two things happening at the same time—with causation, or one phenomenon causing another. In a scalding critique of the article, planning experts Xavier de Souza Briggs and Peter Dreier noted, “The most casual and unfortunate part of Rosin’s analysis of crime and public housing relocation is her assertion that there must be a direct causal link somehow mirrored in the maps she discusses and shows, between a federal program and the patterns of crime in one city. She indicts a program, without any hint of direct or clear evidence, using the simple version of an ongoing mapping project by two University of Memphis researchers.”
Today, we know that the relationship between crime and federally subsidized housing is actually reversed: Someone who lives in Section 8 housing is more likely to be the victim of crime than is the average person, not necessarily a perpetrator. Later, Janikowski realized that part of the reason for the crime increase in those areas was that some very important Section 8 housing units had been torn down and a vulnerable population was suddenly placed on the street. But none of that vital information was available or came across in the little-red-dot map passage in Rosin’s article.
“Rosin heard these ideas when she interviewed a number of the nation’s top housing researchers, but she chose to misconstrue what they said in order to produce the dramatic, but misleading, conclusion that low-income housing programs had the unintended consequence of driving up crime, including murder—all the way out in the once-healthy Memphis suburbs and city neighborhoods too. We have not claimed that this effect is impossible, only that she never presents adequate evidence for it,” note Briggs and Dreier.
Esri, along with IBM, didn’t write the article or lead Rosin to her conclusions, but Esri software played a role in developing the Memphis hotspot map. There’s a cautionary takeaway from the story: Items on a map, like correlations across big data sets, can suggest relationships that don’t exist or misrepresent relationships as they are. Because an interactive map can communicate so much so very quickly, it works to fuel faster decision making, and faster is not always better.
Memphis was one of the first cities in the United States to attempt to deploy police to designated areas in anticipation of future incidents (based on statistical analysis). This practice is more commonly known as predictive policing and is a key area of growth for Esri.
It’s also controversial. Some civil-liberties advocates worry that predictive policing tactics could be used to preempt peaceful civil demonstrations, as some claimed happened in Miami in the 2003 World Trade Organization protests. In New York, predictive policing is considered a key component of the city’s zero-tolerance approach to crime, and it factors in the use of stop-and-frisk tactics that have repeatedly been challenged in court as discriminatory and potentially unconstitutional.
None of these problems is the fault of Esri or interactive maps, but they do highlight the dangers of looking at a chart of blighted areas, adding a new map of bright red dots, and suddenly seeing only a field of potential criminals instead of the full picture. As these GIS systems become more robust and make their way from the command center to phones that individual patrol units carry with them, the powers of law enforcement will expand even in places where the relationship between law enforcement and the community is sour.
This speaks to one of the big worries that privacy advocates have about big data: that large institutions will use new capabilities against the interests of consumers, citizens, and communities. When a bank takes a data snapshot of a neighborhood, sees that a majority of the residents lives paycheck to paycheck, and puts in a high-interest payday loan office, they’re using people’s data against them in a way that’s perfectly legal but not ethical, and likely to become more common.
To Janikowski, the best way to avoid these traps is to involve more data sources and more experts and institutions, to use the power of the collaborative computer mapping to its fullest potential. This, he says, is what Betts’s Center for Community Building is attempting in Memphis.
“One of the things [the Center] is doing is working with [the Memphis Police Department] to integrate with MPD all of those kinds of layers, from foreclosures to blight, to population transitions as neighborhoods change [and] as populations are moving,” says Janikowski. This involves not just “analysis of risk factors, but also all kinds of information on assets in the community that you can leverage and build with. Because it’s not just, ‘what are the risks?’ but also, ‘what are your available assets right now?’ that you can build on.”
Every map is only as good as the data that built it and the understanding of the map maker. When we look at the Lenox Globe, considered the cutting edge of mapmaking in the early sixteenth century, and we discover in the portion of the globe represented by East Asia the Latin inscription Hc Svnt Dracones or “here are dragons,” we see a ridiculous attempt on the part of the mapmaker to disguise his ignorance through deceit. But we’re prone to the same intellectual traps. Our inclination is to treat maps as absolute and infallible instructions. You follow the dotted line to the big X and you reach your goal.
If we can avoid the temptation to view any map as complete, if we can remind ourselves not to simply layer a map of housing subsidies on top of the crime map and call it a day, if we can find the energy to instead go one map further, and then another, and then another, then perhaps GIS will live up to its fullest potential. It will become a tool to take knowledge that’s been accumulated across disciplines and recombine it in a way that’s useful to an ever-growing sphere of people.
Uniting the world and all the data we’ve gathered about it through a shared geographical understanding, and then creating maps that go backward and forward in time: This is the promise Dangermond sees in the future of Esri.
“We’ve got all of these ‘-ologies’—biology, archaeology, cultural geography, and physical geography. We have all the different sciences that we’ve have dichotomized, or broken down; we’ve dissected our world into specialists of science. But how do you put it all back together again? That,” he says, “is the interesting part.”❑
About the Author
Patrick Tucker is the deputy editor of THE FUTURIST magazine and director of communications for the World Future Society. His first book, The Naked Future: What Happens in a World That Anticipates Your Every Move?, is forthcoming from Current in 2014.
Free Email Newsletter
To sign up for Futurist Update, our free monthly email newsletter, enter your email in the box below and click Save.
September 21, 2014 - No part of our highly technological economy today is free of being rethought by applying automation in the form of robots. In the past I have written about the use of field robots in agriculture.
On two sides of the planet, two countries describe approaches to dealing with a consequence of climate change - rising sea levels. One has unveiled a multi-decade plan to counter this coastal threat. The other isn't even acknowledging the issue and instead an independent, not-for-profit, crowd-funded group is issuing independent findings.
The first country is The Netherlands.
Quietly lost in the background of this week's NASA announcements is the news that Jeff Bezos' company, Blue Origin, is to provide United Launch Alliance, owners of the Atlas V rocket, with a next generation engine technology for a next generation rocket. Although Boeing and SpaceX got the major headlines as the companies selected to provide crew-manned capsules for the International Space Station (ISS), Blue Origin may have the last laugh.
Recent reports on areas north of 60 degrees latitude point to rapid climate change in this region of the planet. What's happening?
Every Government Official Should Read This: Choosing Between Economic Growth and Fighting Climate Change Is No Longer an Issue
In today's Financial Times, Pilita Clark has written an article titled "Growth and fighting global climate change not incompatible." Did you hear that? Economic growth, the creation of jobs, increased Gross National Product - you in government no longer have to concern yourselves that enacting climate change mitigation strategies is going to drive your country into depression or worse.
One of my readers shared the following infographic, titled NASA Spinning Off Since 1962. It highlights the inventions, discoveries and economic return on investment resulting from NASA. For every dollar invested by the government the American economy and other countries economies have seen $7 to $14 in new revenue, all from spinoffs and licensing arrangements. That amounts to in $17.6 billion current NASA dollars spent to an economic boost worth as much as $246.4 billion annually.
Think of human services as something you guide through collaborative Governances that share ideas and advances at the speed of networks. When we work together vendors will learn to design the products and services you want.
America appears to have reached the Singles' Singularity—and one of the reasons may be the proliferation of dating sites. According to the Bureau of Labor Statistics, 50.2% of the adult U.S. population are unmarried, up from 22% in 1950.