Visualizing SMS hotspots in days following the earthquake in Palantir.
[Editor’s Note: an edited version of this post first appeared on O’Reilly’s Radar blog.]
The prologue was an earthquake of unexpected magnitude and location that left 250,000 dead.
As computer scientists and technologists, we’re used to dealing with large numbers in the abstract. Expressed in human terms, the mind-boggling numbers of 250,000 dead, 300,000 injured and over 1 million people left homeless are hard to comprehend.
Hit the link to read more about how effective data management and analysis is crucial to recovery efforts and see specific examples of data about the situation in Haiti modeled in Palantir Government.
There was one glimmer of hope in this sea of tragedy: the world’s reaction. In the early hours and days after the quake, the focus was on pinpointing, triaging, and rescuing those in grave danger. Since those first harrowing hours, the world has made plain its willingness to help the people of Haiti. Supplies of money, food, medicine, fresh water, and volunteers have been pouring into Haiti and fundraising efforts are on-going around the world.
Technology also played an early, crucial role, with Mission 4636, InSTEDD and Ushahidi reacting lighting-fast to create a data collection system that enabled people in trouble to quickly communicate their urgent needs to rescuers and relief workers . If you haven’t already read it, Lukas Biewald’s piece, How crowdsourcing helped Haiti’s relief efforts, is a great look at those first, and most urgent efforts to collect data and synthesize information about the situation on the ground.
The extent of the devastation in Haiti. Photo courtesy of Marko Kokic / ICRC / American Red Cross.
Unfortunately, even partial recovery in Haiti will take years at the bare minimum. U.S. Vice President Joe Biden stated on 16 January that President Obama “does not view this as a humanitarian mission with a life cycle of a month. This will still be on our radar screen long after it’s off the crawler at CNN. This is going to be a long slog.”
The recovery from a disaster of this magnitude presents some important tasks in the sphere of information technology: coordination of effort, triaging those most in need, and getting good data into the hands of decision makers and aid workers.
Here’s a partial list of aid, relief, and rescue organizations currently in Haiti, gleaned from Wikipedia:
A wealth of collaborators like this presents some unique challenges around information fusion: unlike business competitors or opposing sides of a war, the different groups want to share as much information as possible to achieve their common goal. A unified organization, like a single national military will have pre-existing methods to model and share their situational awareness. That is not the case in Haiti: this is a collection of groups coming together to form an ad-hoc relief force. Everything from differences in human languages, database schema, collection methodology, and problem domain make most of the datasets seemingly disjoint from the others.
However, each organization has a produced a fairly detailed picture of the parts of Haiti that they are interacting with. Each organization also wants to consume every other’s organization’s detailed knowledge of the situation. To act effectively, they need to integrate that knowledge into a common operating picture that accurately models the situation on the ground yesterday, today, and tomorrow.
Our reaction to the earthquake was to try to help in the best way we knew how. We set up a publicly available instance of our Palantir Government product, already loaded with relevant data, for use by aid workers and organizations working in Haiti. Using relevant, open-source data we’ve started modeling a picture of what’s going in Haiti.
Our first cut was to include the locations and names of collapsed buildings, Internally Displaced People (IDP) camps, and Misson 4636 SMS messages, among others. We also added in map layers that let us see what administrative zone any point on the map is located in.
Having mapped the data into this model, users have access to it through a suite of visualization, analysis, querying, and collaboration tools that allow them to get useful answers to practical questions. Here are some examples:
We’ve created a video showing all the pieces put together into a seamless whole, using live data in our publicly available Haiti instance:
A view of the water point at the Citee Renault camp in Port-au-Prince, Haitii.Photo courtesy of Joe Lowry / IFRC / American Red Cross
From the Red Cross website:
“We’re racing against the clock with hurricane season just around the corner,” said Jean Pierre Taschereau, a Red Cross disaster expert just back from Haiti. “Getting semi-permanent structures in place as well as trenches for sanitation latrines will be critically important.”
The UN wants to move 200,000 people out of overcrowded camps like this one. The Haitian government is trying to find land. It’s identified five sites outside of the Haitian capital, but those five sites are about 200 hectares and by the UN’s estimates 600 hectares will be needed to house the people it plans to move safely to have proper drainage when the rainy season finally arrives.
Haiti’s rainy season is notorious for causing flooding. Now, with the infrastructure of the country destroyed, flood season will be more dangerous than usual. Not only are the normal structures that protect people from the waters gone, but they’ve moved out of the ruins of Port-au-Prince to hastily constructed IDP camps, some of which are sitting in the flood plains of Haiti’s waterways.
The essential question facing relief workers: Which of the approximately 2500 IDP camps are most at risk from flooding?
In a place like the United States, an earthquake response and recovery team could engage the services and expertise of the US Geological Survey, which maintains the National Water Information System, a warehouse of detailed information about all things water in this country. No such luck in Haiti, where the closest thing to the USGS is the Centre National de l’Information Géo-Spatiale. A quick look at their website shows that they didn’t really make it through the earthquake. (In the video, we feature a picture of what’s left of their facility — it’s not pretty).
Since we’re starting from square one we put together data from the Army Geospatial Center, the UN, NOAA, Haiti-based NGOs, a number of academic papers, and even geo-tagged photos from Flickr. The time it took to integrate this data? About six hours. Time it took to do the analysis? About seven minutes. Amount of that work that is reusable? All of it.
Check out this video for a walk-through of the analysis:
The best way to improve this analysis will be to add more detailed information about flooding, gathered from the field. We’re looking into getting new conduits of information into the Haiti instance to make this a reality as the rains really pick up.
If you’d like to help us, we’re accepting new data sources, analyses, and contact with relief organizations.
Volunteers, supplies, and goodwill are only the raw ingredients to recovery; it’s the efficient and timely application of those resources to Haiti’s most pressing problems that will change lives and make recovery a reality instead of just a good intention.