FOSS4G - North America 2012: Observations

I had the good fortune to make it to this year's iteration of the Free and Open Source for Geospatial (FOSS4G) tech conference - North American edition (other sections of it will be occurring in Germany and China). Having missed the O'Reilly festival of place, it was a great opportunity to catch up with - as one attendee put it - "The real makers in the world of maps". [Sorry, ESRI, you've actually been standing in the way of cartographic explanation for about a decade now]

While there was more afoot than I could ever have followed entirely, I took note of two items:

1.) People are really jazzed about PostGIS. Core architect Paul Ramsey kicked off what would be three days of love for the new release (version 2.0), and we were treated to all sorts of examples on how you can use it. Raster support, TileMill optimization, simple SQL strings replacing reams of jagged python for geoprocessing workflows - the possibilities seem endless.

I really like that such a great, extensible project is at the heart of open-source geographic technologies, but I still feel left out of the party. Basically, I am intimidated by the first hurdle of PostGIS: PostgreSQL at the command line. This is not that big of a deal for true developers, but I've been repeatedly burned by the wonkiness of database administration, permissions and security in postgres. Once I can get to the actual SQL, the birds start singing, the sun comes out and the world is full of possibilities, which is what makes client-focused tools like CartoDB and QGIS so enjoyable.

2.) Web maps are the home of cartographic design now. This may have been the purview of National Geographic even five years ago, but static maps are lacking the possibilities laid out - for example - by AJ Ashton and Nathaniel Von Kelso in sessions that touched on using OpenStreetMap as both art and function.

MapBox "Lacquer" Tiles

Stamen "Watercolor" Tiles

Take this with a word of warning issued by Schuyler Earle during a panel session: Software developers think they are natural designers. They're not. They need to hire designers, who can - like sober friends at 2AM - shepherd us away from bad decisions. Users will benefit if we can all get a design perspective on "I think this thing needs another button on the sidebar" before we act.

Many thanks to the organizers of FOSS4G-NA. It was a rockin' good time, full of productivity and great encounters. And for those who need a lingering shot of natural-disasterage, here is my ignite talk from the first night. Heavy on pictures, low on explanation, much the way it should be:

Read more ...

OpenstreetMap and the Bulk Battle

I like building footprints. As a cartographer, I find that they bring important context to a certain scale of map. In the same way that I look for patterns and paths in the contour lines of a national park, I see explanations of how we live and work in the layout of buildings in a city. Basemaps for the web applications I write always benefit from having structures as a ground reference.

Over the past few weeks I've been working to add about half a million building footprints to OpenStreetMap, mostly in the Mid-Atlantic region of the United States. The excellent data is the result of years of work - ongoing in many sites - by the Spatial Analysis Lab at the University of Vermont. They've fused LIDAR data with multispectral aerial imagery in an object-based classification of land use for their client municipalities, and they're happy to offer the results to the OSM open license.

Here's the current OSM Mapnik render in Maryland Park, just outside of D.C. -

And here's the same area with UVM-SAL building footprints added -

I've been careful to discard footprints that overlap with an existing OSM line or polygon feature, since user-traced content takes priority over bulk imported content. But having taken this data to the OSM community for review, I am now aware that it's a complicated thing to "Add data" in bulk to a crowdsourced database. Users have eloquently argued both for and against the import of these building footprints, and I find it boils down to a pair of inextricably-linked perspectives on OSM:

1.) OpenStreetMap is a community of individuals. The interaction between the user and the map is most valuable when one user "owns" their offering, and a dozen building-footprint-tracing contributors from Baltimore will feel greater ownership of their neighborhood than one guy in Vermont who uploads the whole county in one batch.

2.) OpenStreetMap is a tool. It is becoming a basemap of record in the GeoSpace precisely because that's what we wanted to do with it. OSM competes admirably with proprietary datasets, and we can use it for beautiful cartography and complex analysis. Any scale of addition is valid, as long as it is offered on the same open license.

The question that lies between these two perspectives: Can we have a quality basemap while maintaining a strong commitment to individual user engagement?

I think the answer is yes. I'm going to proceed with my buildings import on an assumption that was echoed by Kate Chapman: If a user in Bethesda finds that the buildings in her neighborhood are "already there", maybe instead of losing a sense of ownership she'll take it to the next level by adding identities to those structures.

Because I'm just adding the outlines. I can't tell you which one of them is a Bodega. There are many layers of value yet to be added.

Read more ...

WhateverConf 2012 - Notes from a Nonparticipant

So this O'Reilly chap evidently has some soiree planned every year this time, somewhere on the West coast. In the past, I've been able to blithely ignore the proceedings and hope that someone else would eventually point me toward anything useful that arises. Basically the same attitude I have toward SXSW.

Regrettably, this year I'm being forced to perk up from my West-coast-of-New-England-afternoon-coffee stupor and pay the hell attention. Three items caught my eye on Twitter today and yesterday:

1.) Custom cartography for Mapbox Streets - The Development Seed team clearly proceeds from day to day in a Red-Bull-fueled tornado of productivity. They've caught on that the momentum is away from the ubiquitous Google Maps API and base layers, and they're doing everything they can to offer a replacement for the whole stack. In particular, they've done Google one better here on cartographic quality. MapBox's OpenstreetMap-based tiles are uniquely beautiful, and sufficiently-varied that we users might now be able to escape the homogenization of web map styles over the past few years (Yes, Microsoft, even your basemaps look a lot like Google's). Cartographic design just won big. Here's some Tilemill-styled twitter data of mine over the new Mapbox "Lacquer" Base tiles:



2.) Foursquare has a new geocoding engine - Saints be praised, another way to escape Google. I don't yet know how solid Foursquare's placename database - called TwoFishes- will be for everyday applications. However, my current reality has me tethering whole projects to the GMaps API specifically because the Google geocoder is the only one to consistently find the correct location when a user searches for an address. I need alternatives, and they are particularly welcome from a company that is already showing a dedication to open geodata.

3.) CartoDB murders the middle of the GeoStack - The team at Vizzuality has been intimidatingly-busy as well. Though it may have been overlooked in the WhereConf hoopla, yesterday they teased out a project called VECNIK; at the moment it consists of a series of HTML5-ready libraries for fast vector styling directly in the browser. This includes pulling the parsed vector from the already-smooth CartoDB SQL API, compiling a stylesheet in the super-flexible Carto language, rendering the whole thing with Mapnik and serving it into the ModestMaps library - entirely in the browser. This happens at speeds that ten months ago would have been achievable only with cached tilesets. As with basically everything Vizzuality does, it's going to take me a few more months to grasp the full implications of this advance; I'm psyched enough for the moment though. Here's an example showing "Hand-sketched" parcel boundaries in my neighborhood (Modern browser required):



So to anyone else who's planning to drop major innovation bombs on the GeoSpace at WhereConf, $#@%in' stop. I've had enough for this week. Don't even get me started on PostGIS 2.0. Save it for FOSS4G-NA in D.C. next week, by which time I hope to have recovered enough spare neurons to take it all in. See you there.

Read more ...

Zombie Lunchpocalypse: An Iron Sheep 2012 Entry

Thanks to Matthew Zook and the folks at Floating Sheep for organizing the Iron Sheep 2012 open data hack in NYC. While some of the teams were tempted to go with a serious analysis of social/spatial issues, most of us couldn't resist running with the frequency of Twitter and Google uses of words like "zombies" and "aliens". Our entry to the competition:




"The humans are gone. Long past is the zombie apocalypse. No human brains remain for feasting. So what is a hungry zombie to do but seek out sheep brains, according to the dietary preferences they might have once held as a living hipster in Williamsburg? Lunchtime finds discerning zombies wandering the five boroughs in search of organic sheep brains . . . 

The Lunchpocalypse tool is meant to aid the enterprising zombie restaurateur in locating and exploiting key demographic clusters of organic-preferring zombie consumers. Simply plug in a NYC address, and you'll be able to see what the local preference is, leveraging a geodemographic segmentation analysis of the frequency of internet searches of the words "zombie" and "organic". Maximize your zombie profit and shuffle along with your organic herd to where the customers are! Because the dead are damn hungry!"

Read more ...

SOPA 2014

While I'm aware that dystopian fiction is more the field of Newt Gingrich, I had a few thoughts today as the SOPA/PIPA battle continued to rage across the interwebs and traditional media both. You see, I'm a cartographer, and visualization is my stock in trade. As a cartographer with a particular interest in Crisis Mapping, I realized that the bills now before the House and Senate could have some as-yet-unexplored consequences. Let's explore some of them:

**************************
January 19th, 2014

• At 8AM, the shelling begins in Acarigua. Venezuelan government forces are trying to push their way into the center of the city, where pro-devolution demonstrators had clashed with - and ultimately expelled - the federal police the previous day. Now, as the troops begin to retake Acarigua block by block, panicked residents scramble for cover while others erect barricades on the main streets.
• Eustacio Torres was living in Pittsburgh on a guest worker visa when the news arrived that Acarigua had rebelled against federal Venezuelan control; he was immediately worried for the safety of his mother, who still lived there. At 9:10AM, some mobile phone providers' lines are jammed with traffic and others have obeyed a shutdown order issued by the government. Eustacio is not alone in his concern; thousands of others in Venezuela and beyond are looking for news from Acarigua.
• At 9:15AM, Belen Diaz - a software developer in Bogota - registers the domain name ataqueacarigua.org.co and hosts it to her server on a local ISP. She deploys the Ushahidi crisis monitoring platform and attaches it to that domain. By 9:55, after some calls to news organizations and social media outlets, the site begins to see significant traffic. Hundreds of users post updates and comments as news appears. A small but key number of reports filed to the site come from residents of Acarigua, sent by SMS on the spottily-available signals to a dedicated shortcode registered by Belen.
• At 11:43AM, a freelance journalist working for News Corp. takes 17 seconds of footage showing soldiers firing live rounds on a group of apparently-unarmed young men in Acarigua. Unseen in the confusion, he manages to quickly offload the footage to New York via satellite uplink.
• At 12:34PM, as the shelling tapers off and automatic weapons can be heard closer by, Anamaria Torres asks her nephew Jaime to call her son Eustacio, saying that she's safe for the moment but scared. Jaime tries, but can't connect the international call. He then posts the message by SMS to the Ushahidi shortcode. The message reaches ataqueacarigua.org.co.
• At 1:12PM, Belen sees the message posted by Jaime, marks it as plausible but unconfirmed, and posts it to the site for all to see.
• At 1:14PM, Jacob Crenshaw in Dallas sees a grainy clip of soldiers firing on civilians in Acarigua, as part of a Fox News analysis segment. He records it off the TV with his phone camera and uploads the footage directly to ataqueacarigua.org.co. A volunteer working with Belen sees it and posts it to the public view.
• At 1:56PM, a Fox web technician is alerted to the unauthorized rebroadcast of footage on ataqueacarigua.org.co. The News Corp. copyright-dedicated legal team submits an urgent takedown request to the U.S. Justice Department. The A.G.'s SOPA-compliance division grants the request, noting that the appropriate boxes are checked to meet the burden of proof and that the paperwork is in order.
• By 3:45PM, the domain ataqueacarigua.org.co has been blocked and all search engines have removed it from their listings. Attempts to broadcast the direct IP address are similarly blocked and sites sharing it are added to the injunction.
• At 5:22PM, reports are still coming in to the server, but less than 5% of visitors are able to view the site.
• At 6:40PM, Eustacio is able to access the internet using a friend's laptop. There is no communication from his mother, and the front page of ataqueacarigua.org.co reads "This domain has been blocked pending legal action as per the provisions of H.R. 3261, the Stop Online Piracy Act" . . .

********************
So this is nuts, right? But I've grown a bit weary of hearing that SOPA and PIPA will "Only impact foreign sites dedicated to infringment." Many of us work across borders, and our world is interconnected in ways that our Congress doesn't seem to understand. But more specifically, does anyone believe that the Justice Department will get right on SOPA compliance requests with lots of funding and an adequate staff? Or that they won't be carroted and sticked into pro-forma approval of most requests the same way the major banks bulldozed the OTS in the run-up to the 2008 economic crash?

These bills are a real threat to freedom, at home and abroad. We should be concerned about both. This has been a whimsical case study, but it's because these bills show a lack of imagination.

Read more ...

Crowdmapping Breadcrumbs

Just had a great chat with the folks at Kelliher Samets Volk about the broad range of applications for crowdsourced mapping. My experience so far has been in the context of disaster response, but there's nothing like a good conversation to open up my brain to broader possibilities. 

The talk inspired some good linkbait, so here are some sites that have mattered to me as I navigated the waters of new technology after TS Irene (flooding pun regrettably intended).

Sites I actively manage:

- VTIrene Crowdmap
- VTIrene High Water Assessment
- Twitter activity during the storm

Ushahidi-related sites:

- Oveview (video)
- Deployment examples
- Roll your own crowdmap

SM Data interpretation - potential partners and tools:

- metaLayer Inc.
- SwiftRiver (to be released at the end of January)

And lastly a great read from Matthew Ericson on when data is really geographic and when it
isn't

As a whackjob storm system rolls through Texas, it's nice to know that these tools are being put to use as I write: https://houweather.crowdmap.com/reports/view/404



Keep it up, people!

Read more ...

Puzzler

Update 1/11/12: We have a winner over in Facebook-land, and the answer is Coconuts! You can tell by the faint serrations of the fronds, the long shadows and the widely-spaced planting pattern. Thanks to all who participated!

Happy New Year! We're deploying a mind-numbing array of triggers using ifttt. Here's an imagery puzzler while you wait out this demo.

What crop type are we looking at in this QuickBird Satellite image from somewhere in the caribbean?

Leave your guess in the comments section. The first correct answer will receive a glossy GeoSprocket bumper sticker designed by Cranewalk studio. Pimp your refurbed macbook or whatnot.

Read more ...

GeoSprocket 2011 in Review

It's been a good year. Lots of new open-source tools for geospatial applications have come on-line; large organizations have been opening up their data stores and letting us run wild with the possibilities. It's hard to keep up with everything - to leverage every new technology - but here are a few projects we've been occupied with this year:

Read more ...