Zombie Lunchpocalypse: An Iron Sheep 2012 Entry

Thanks to Matthew Zook and the folks at Floating Sheep for organizing the Iron Sheep 2012 open data hack in NYC. While some of the teams were tempted to go with a serious analysis of social/spatial issues, most of us couldn't resist running with the frequency of Twitter and Google uses of words like "zombies" and "aliens". Our entry to the competition:




"The humans are gone. Long past is the zombie apocalypse. No human brains remain for feasting. So what is a hungry zombie to do but seek out sheep brains, according to the dietary preferences they might have once held as a living hipster in Williamsburg? Lunchtime finds discerning zombies wandering the five boroughs in search of organic sheep brains . . . 

The Lunchpocalypse tool is meant to aid the enterprising zombie restaurateur in locating and exploiting key demographic clusters of organic-preferring zombie consumers. Simply plug in a NYC address, and you'll be able to see what the local preference is, leveraging a geodemographic segmentation analysis of the frequency of internet searches of the words "zombie" and "organic". Maximize your zombie profit and shuffle along with your organic herd to where the customers are! Because the dead are damn hungry!"

Read more ...

SOPA 2014

While I'm aware that dystopian fiction is more the field of Newt Gingrich, I had a few thoughts today as the SOPA/PIPA battle continued to rage across the interwebs and traditional media both. You see, I'm a cartographer, and visualization is my stock in trade. As a cartographer with a particular interest in Crisis Mapping, I realized that the bills now before the House and Senate could have some as-yet-unexplored consequences. Let's explore some of them:

**************************
January 19th, 2014

• At 8AM, the shelling begins in Acarigua. Venezuelan government forces are trying to push their way into the center of the city, where pro-devolution demonstrators had clashed with - and ultimately expelled - the federal police the previous day. Now, as the troops begin to retake Acarigua block by block, panicked residents scramble for cover while others erect barricades on the main streets.
• Eustacio Torres was living in Pittsburgh on a guest worker visa when the news arrived that Acarigua had rebelled against federal Venezuelan control; he was immediately worried for the safety of his mother, who still lived there. At 9:10AM, some mobile phone providers' lines are jammed with traffic and others have obeyed a shutdown order issued by the government. Eustacio is not alone in his concern; thousands of others in Venezuela and beyond are looking for news from Acarigua.
• At 9:15AM, Belen Diaz - a software developer in Bogota - registers the domain name ataqueacarigua.org.co and hosts it to her server on a local ISP. She deploys the Ushahidi crisis monitoring platform and attaches it to that domain. By 9:55, after some calls to news organizations and social media outlets, the site begins to see significant traffic. Hundreds of users post updates and comments as news appears. A small but key number of reports filed to the site come from residents of Acarigua, sent by SMS on the spottily-available signals to a dedicated shortcode registered by Belen.
• At 11:43AM, a freelance journalist working for News Corp. takes 17 seconds of footage showing soldiers firing live rounds on a group of apparently-unarmed young men in Acarigua. Unseen in the confusion, he manages to quickly offload the footage to New York via satellite uplink.
• At 12:34PM, as the shelling tapers off and automatic weapons can be heard closer by, Anamaria Torres asks her nephew Jaime to call her son Eustacio, saying that she's safe for the moment but scared. Jaime tries, but can't connect the international call. He then posts the message by SMS to the Ushahidi shortcode. The message reaches ataqueacarigua.org.co.
• At 1:12PM, Belen sees the message posted by Jaime, marks it as plausible but unconfirmed, and posts it to the site for all to see.
• At 1:14PM, Jacob Crenshaw in Dallas sees a grainy clip of soldiers firing on civilians in Acarigua, as part of a Fox News analysis segment. He records it off the TV with his phone camera and uploads the footage directly to ataqueacarigua.org.co. A volunteer working with Belen sees it and posts it to the public view.
• At 1:56PM, a Fox web technician is alerted to the unauthorized rebroadcast of footage on ataqueacarigua.org.co. The News Corp. copyright-dedicated legal team submits an urgent takedown request to the U.S. Justice Department. The A.G.'s SOPA-compliance division grants the request, noting that the appropriate boxes are checked to meet the burden of proof and that the paperwork is in order.
• By 3:45PM, the domain ataqueacarigua.org.co has been blocked and all search engines have removed it from their listings. Attempts to broadcast the direct IP address are similarly blocked and sites sharing it are added to the injunction.
• At 5:22PM, reports are still coming in to the server, but less than 5% of visitors are able to view the site.
• At 6:40PM, Eustacio is able to access the internet using a friend's laptop. There is no communication from his mother, and the front page of ataqueacarigua.org.co reads "This domain has been blocked pending legal action as per the provisions of H.R. 3261, the Stop Online Piracy Act" . . .

********************
So this is nuts, right? But I've grown a bit weary of hearing that SOPA and PIPA will "Only impact foreign sites dedicated to infringment." Many of us work across borders, and our world is interconnected in ways that our Congress doesn't seem to understand. But more specifically, does anyone believe that the Justice Department will get right on SOPA compliance requests with lots of funding and an adequate staff? Or that they won't be carroted and sticked into pro-forma approval of most requests the same way the major banks bulldozed the OTS in the run-up to the 2008 economic crash?

These bills are a real threat to freedom, at home and abroad. We should be concerned about both. This has been a whimsical case study, but it's because these bills show a lack of imagination.

Read more ...

Crowdmapping Breadcrumbs

Just had a great chat with the folks at Kelliher Samets Volk about the broad range of applications for crowdsourced mapping. My experience so far has been in the context of disaster response, but there's nothing like a good conversation to open up my brain to broader possibilities. 

The talk inspired some good linkbait, so here are some sites that have mattered to me as I navigated the waters of new technology after TS Irene (flooding pun regrettably intended).

Sites I actively manage:

- VTIrene Crowdmap
- VTIrene High Water Assessment
- Twitter activity during the storm

Ushahidi-related sites:

- Oveview (video)
- Deployment examples
- Roll your own crowdmap

SM Data interpretation - potential partners and tools:

- metaLayer Inc.
- SwiftRiver (to be released at the end of January)

And lastly a great read from Matthew Ericson on when data is really geographic and when it
isn't

As a whackjob storm system rolls through Texas, it's nice to know that these tools are being put to use as I write: https://houweather.crowdmap.com/reports/view/404



Keep it up, people!

Read more ...

Puzzler

Update 1/11/12: We have a winner over in Facebook-land, and the answer is Coconuts! You can tell by the faint serrations of the fronds, the long shadows and the widely-spaced planting pattern. Thanks to all who participated!

Happy New Year! We're deploying a mind-numbing array of triggers using ifttt. Here's an imagery puzzler while you wait out this demo.

What crop type are we looking at in this QuickBird Satellite image from somewhere in the caribbean?

Leave your guess in the comments section. The first correct answer will receive a glossy GeoSprocket bumper sticker designed by Cranewalk studio. Pimp your refurbed macbook or whatnot.

Read more ...

GeoSprocket 2011 in Review

It's been a good year. Lots of new open-source tools for geospatial applications have come on-line; large organizations have been opening up their data stores and letting us run wild with the possibilities. It's hard to keep up with everything - to leverage every new technology - but here are a few projects we've been occupied with this year:

Read more ...

Place and Sentiment From Many Voices

Place is powerful in a crisis.

Place brings the responder to the emergency. Place measures risk and damage. Place is often the most important piece of situational context, particularly when a hurricane takes a wrong turn on the New Jersey Turnpike at the end of an otherwise-unremarkable August.

Crisis maps were invaluable in the reaction and recovery surrounding the drunken lope of Hurricane Irene up the eastern seaboard a few months ago. The rapid deployment of tools like Ushahidi, Tweak the Tweet and #VTResponse created a centralized, crowdsourced voice for the crisis. Google's Crisis Landing was used to augment the capabilities of local, state and federal agencies as they organized reports of damage and assessed priorities, disseminating the information directly to the public. [Patrice Cloutier has exhaustively documented the crowdsourced Irene response in this report] All of these tools involved a place component -  "Where is this happening?", "Where is this needed?" - and could draw on semi-precise geolocation assets like online address searches and smartphone GPS sensors.

However, there was also a huge dialog occurring without real geolocation, on Twitter, Facebook and any number of smaller social media outlets. These messages numbered in the hundreds of thousands, but on their own were lacking the context of place.

This is where new developments in "Big Data" analytics come in handy. With some computational heavy lifting from Kate Starbird at the University of Colorado and Chris Danforth at the University of Vermont, Geosprocket was able to bank millions of Twitter posts from the days surrounding the storm. Then the assistance of metaLayer Inc. was instrumental in putting Irene-related tweets on the map. Using a series of digital sifting processes, they were able to mine the archived Twitter data for placenames and keywords. Where a town or street name was included in a post, that post could be placed at a set of geographic coordinates. Were words like "washout" and "devastated" were used, fine-tuned algorithms could assign a scaled value for the sentiment of the post.

Add a bit of cartographic styling and serving with the open-source MapBox toolkit, and we've got an interactive mapped timeline of Hurricane Irene, as told by Twitter:

No analysis of this type is going to be perfect - the data is inherently messy, noisy, inconsistent - but now that the storm is long past this kind of view can tell us a lot about how we responded to the crisis. It can give us ideas for preparedness. And as platforms like metaLayer and Swiftriver become more robust, we can apply the power of place and more in real-time when the next crisis hits.

Read more ...

Creating Geographic Variation Where it Doesn't Exist . . .

Info-Vortex Over Burlington

Cartography has a well-earned but still-largely unrecognized reputation for messing with human perceptions of real phenomena. Got heathens on the other side of your national border? Just shade them out and pretend they don't exist. Want to make it seem like you're a global underdog? Show yourself surrounded and menaced by powerful foes. This sort of crap makes cartography - and geography more broadly - look like something practiced by schemers in tophats and handlebar mustaches.

But manipulation of space is the bread and butter of cartography, stemming from some central questions: How do you make round things flat? How do you make the impossibly-large small enough to see on a canvas or on a glowing smartphone screen? Mapping inherently involves reduction and obfuscation, so the problem I recently came across is a novel one:

How do you expand an effect on a map?

When Hurricane Irene hit the U.S. Northeast, hundreds of thousands of people took to the Twitterverse (and to Facebook and traditional civil response agencies, not to discount them) to report damage and request help. A few thousand folks mostly in the NYC area sent their tweets with geolocation included. Good on them, but they represented somewhere in the neighborhood of 1% of the hurricane-related traffic. Tens of thousands more sent messages with placenames in the text, some more specific than others.

Modern geocoding services like Google, Yahoo and Nominatim provide great geolocation based on addresses and fragments, but what happens when you've got a single placename representing a general area? What if it recurs thousands of times? Do you map everything at a single location, the centroid of a state boundary?

The word "Vermont" appeared in over 6,000 Twitter posts in the four days immediately after Irene washed through the state. With nothing else to add specificity, this means that over 6,000 events occurred at 43.8717 N, 72.4517 W: the geographic center of the state. While this is a bit whimsical, it still presents the problem of showing a huge number of unique messages at a single place.



A spiral seems to fit the bill; the points spill outward and occupy their own space, but they clearly originate from the central coordinates they share. It's not spatially exact, but it shows both association and information volume. Smaller spirals begin everywhere multiple posts share the same placename, for example in Rochester and South Burlington.

Using LibreOffice Calc (or Excel if you must) and CartoDB, this visualization is actually a snap. It begins with a set of Twitter posts and Lat (Y1) & Long (X1) coordinates taken from placenames in the text (a service provided by metaLayer - more on that in the next posting).

1. Sort the table by unique location
2. Add fields representing:
1. Location-magnitude (i)
2. Bearing of spiral point (ang)
3. New Lat (Y2)
4. New Long (X2)
3. Calculate as follows:

a = 0.005

b = 0.009

i = if X1 is not unique

  i + 1

 else 0

ang = (Pi/45)*i

Y2 = Y1 - (a * SIN(ang)) * (2.718281828 ^ (b * ang))

X2 = X1 + (a * COS(ang)) * (2.718281828 ^ (b * ang)) 

This may seem a bit obtuse, but it can be adapted to VBA, Python, JavaScript or whatever suits your purpose. And the 'a' and 'b' variables can be tweaked to make a spiral to your own specifications; 'a' controls the distance offset from the central point, and 'b' controls the rate at which the spiral expands. It works best if both are kept between 0.001 and 0.1. In any case, once the table has the new coordinates, upload it to your CartoDB account and render some tiles! 

There are certainly a host of other ways to attack this problem, but a spiral is a basic solution in a world where crowdsourced data increasingly muscle into the geographic sphere, begging to be placed on a map.

Read more ...

CartoDB and the Manifold Uses of Tiled Maps

Javier de la Torre was kind enough to give me a demo account for the new CartoDB project from Vizzuality. Described by some as the "Open-source fusion tables", CartoDB is interesting to me for its novel tiling functionality. Usually, tiled maps are lightning-fast on the web, but they're set in stone and take forever to render on the server side. CartoDB uses some cool tweaks to serve map tiles fast as usual, but with added feature editing capability. When a feature is selected from the tiles, CartoDB instantaneously serves up a vector version for editing.

While this can lead to some interesting hiccups - multi-feature editing, for example - this is still a huge step forward for web cartography and GIS. It's got the speed of tiles and the interation of desktop vector. The future looks bright . . .

Here's an example from CartoDB, serving up a Geosprocket database of ecoregions clustered into Biodiversity Driver Zones (BDZs):

Read more ...