Pages

American Agriculture on the Ground

Monday, July 2, 2012
The day following my cousin's wedding in Geneva Illinois, I climbed into my rental car and cruised West a few miles. I had some time before my flight back East and realized I didn't know what a real cornfield looked like. Not from the ground, anyway. Though I'm working on a PhD in agricultural remote sensing, I spend my days analyzing crops from a vertical perspective of several hundred miles up - from high-resolution satellite instruments that can tell me a lot about cultivation patterns, vegetation health and potential crop yields. I can do this without leaving my office, and the Vermont landscape where I've lived most of my life is a hilly mosaic of field and forest. The largest corn plot in my neck of the woods is about 50 acres.

Where I was in Illinois, on the other had - seen from the air - could be plausibly called the start of the cornbelt. It's where the Chicago exurbs come to a surprisingly-abrupt halt and are replaced by a gradient of endless agriculture heading to the rockies - first the large grains, then wheat and hay, then rangeland, and not a lot of people. So I went to have a look. It only took a few minutes' drive to get beyond the housing developments and among some of the broadest agricultural expanses I'd ever seen live. I pulled off the road at one spot that looked fallow - like I could walk freely there - and got out of the car.



The line is drawn outside of Geneva, IL
10AM and already over 90 degrees, it was not a hospitable environment for a northern desk jockey. But shimmering just beyond the fallow was an easy 500 acres of cultivated land, split evenly between recognizable corn and alien-to-me soy. Flat as the griddle before the pancake batter hits it. I broke a sweat and trudged out into the fields.

A few things were noticeable. First that the fringes of the field had smaller plants; this is nothing unique to the midwest, and usually due to some tricks of drainage and fertilizer-scattering angles. But the corn plants suffering on the edges here were still twice as tall as the healthiest of those I'd seen the week before in Massachusetts, burdened by draught upon heatwave upon hail upon late frost. The soy plants here I at first mistook for some monstrous cover crop, with leaves like clover growing a foot up out of the ground. 
Soy


Corn
There were distinct drill-planting scars in their rows, cracked wider in the heat and the dry. The two crops ran into each other in a perfect line, like allied brigades marching in formation rather than opposing armies that have crashed together. In most of the country outside of the Northeast, corn and soy are planted in an annual rotation. They both serve as cattle feed and compliment each other - soy brings nitrogen to the top soil horizon for the corn to thrive on the following year. The scattered cornstalks serve as fertilizer and mulch for the soy sprouts the year after that. On and on, with fallow making its way into the lineup less and less frequently.

I stooped to take some pictures of the rows, and I held a soy plant in my hand. The heat was impossible. I felt myself wilting, desiccating with no shade in sight. The leaves of the plant, however, were thick and healthy, bouncing a bit in the breeze. The robust copse of the stalk was covered thickly with a soft, spiny fuzz, and a few flowers were just peeking out of their exploratory growth. The corn as well seemed to be taunting the sun, suggesting it could do worse.

These were genetically-modified plants. There were no tags and no one to ask, but I could say it with 97% certainty. Most likely from a family of seeds that Monsanto brands "Roundup Ready" for their ability to convert sunlight and soil to biomass at a ferocious rate while being regularly doused with a proprietary insect-and-weed-killing cocktail that could plausibly be used to kill humans with the right concentrations.

I walked across to the other side and touched a corn broadleaf. It was already wider than my hand, and the whole plant came to the height of my nose, with two months yet to go. One ladybug reclined on the leaf, which was otherwise completely devoid of noticeable insect life. What seemed like translucent hairs covered the green surface, projecting the picture of vegetative health. In working with farmers in Central America and the caribbean, I had learned that the interesting stuff was always underneath - turn over a corn leaf and you find a merry band of aphids or a spider with a nest to escape the sun and prey on the rest of the locals. But here the bottom of the leaf was as clean as the top - more of a matte finish than the gloss of the upper surface. Empty as a new house with the plaster still drying.
 I shot some more photos before the sun bested me and I trudged back to the car.  

Corn and soy grown are grown in the Midwest at a scale that matters, in a way that draws the attention and support of the drafters of a five-year farm bill. Grown toward a subsidized ideal that is more efficient than many steel machines. Extending from where I stood West into Nebraska, North into Canada and South to the Gulf.

Awesome to behold.
Looking West

Read more ...

Browser Cartography: Some Safehouses for ESRI Refugees

Wednesday, May 30, 2012
"Island" by Konstantin Kafer

A Primer for Getting Started With Open-Source Web Maps


Now that you know why I care about telling compelling stories with widely-distributed maps, let's look at a few of the many tools that are out there to help the process. I confess to narrow experience here; I use MapBox and CartoDB for the majority of my projects, and there are plenty of alternatives to those. But as a starting point I think that these open-source web map design platforms are perfect - they minimize the amount of code required, they use the best graphic rendering engine in the field, and they are extremely cheap (or free) to use, even in an enterprise or high-traffic environment. I'm avoiding ESRI's "ArcWhatever Online for Server" options because of a.) the high price tag, and b.) the actual user-facing sites are only as robust as the javascript or flash developer who builds them. My preferred options give you a lot more to work with out of the box, for free.

I am not going to walk you step-by-step through the process below - I'll point you to resources that will - but rather I mean to sketch the structure that can get your map applications up and running. If you're stumbling and need some help, drop me a line through the GeoSprocket contact page or on Twitter

Step 1: Data


Data in this context can mean a lot of different things. The tools we're using aren't picky, so this includes:

  • Shapefiles (but this will need to be in a compressed folder)
  • Spreadsheets (CSV, XLS, DBF, you name it)
  • KML or KMZ
  • GPX or in many cases raw text from various GPS units

Desktop GIS isn't dead. Those who say so aren't paying attention. You still need some form of spatial data manipulator to analyze and prepare your source information, and if you can do that with pure GDAL hacking at the command line of a virtual machine, you have no need for my advice. Yes, "Desktop Platform" includes ArcMap, but if you want to make a clean break I recommend Quantum GIS for robust, open-source geoprocessing potential.


Step 2: Choose a Platform and Get a Hosting Account

Your data should live on the web somewhere - where, exactly?
  • Option 1: MapBox - Most advantageous for its speed and complete cartographic design potential. However, once you've rendered your map into tiles (in step 3), there's not much you can do to change them on the fly. Use this option if you have a specific vision of symbology in mind to present to your users, if you have a lot of information to present interactively (i.e. text/charts/images that pop up when a feature is clicked), and also if the underlying data will be accurate for more than a few weeks. Sign up here for the free starter account.
  • Option 2: CartoDB - This is a user-friendly, cloud-based adaptation of the popular PostGIS database architecture. It can run a bit slower than MapBox (since it's rendering on the go), and the cartography options are more limited. However, the provided SQL API means that you can sort, filter and process your data basically in the browser. Go with this option if your application will have a lot of user-generated queries (i.e. "how many points are within 50 miles of this one?"), or if you regularly update the data being mapped. Sign up here for the free starter account (notice the pattern of freeness).


Step 3: Mapping Your Data




At this point using either option, you should now have maps that are ready to be launched.

Step 4: Serving Your Map to the World




Step 4a: Embedding - By far the easiest way to get a URL that you can distribute to your audience. Both Mapbox and Cartodb have fantastically-easy embed interfaces for plugging maps into your blog or website content management system, and in each case they essentially host a full-page map website for you as well. Examples:


Racial Breakdown of Census Blocks in Burlington, VT Using Mapbox


Participatory Farm Mapping in Vermont with CartoDB



Step 4b: The World of Pages, Javascript and Beyond - This creeps a bit further beyond the scope of what I hope to cover here, but this is the ultimate destination for most of my map applications. There is simply greater flexibility in a website or mobile application that you can manage yourself, though it requires some knowledge of HTML, CSS and Javascript.

I say "some" knowledge - I didn't know the first thing about code when I got into this world, but it was amazing how easy it was to adapt a little bit. The resources available in the open-source software community are spectacular; the help and guidance offered to me by experienced developers out of sheer goodwill has been uniformly superior to the high-priced support of proprietary vendors (Trimble and ESRI, I'm looking at you). I am no developer, but with community help I've been able to explore and utilize some of the most exciting tools available to cartographers in our time. And things are only getting better from here . . .

Here are a smattering of places to start for customizing a page of your own:

  • Mapbox Templates - these require a bit of github knowledge, but otherwise they are as close to plug-and-play map sites as you'll find. 
  • CartoDB Examples - Plenty of code available for re-purposing and adapting. In many cases this just involves changing a line or two to point to your own data.
  • Codecademy - Might as well learn how to do this stuff for real . . .


Read more ...

Browser Cartography: A Manifesto

Tuesday, May 29, 2012
Hear me out for a minute . . .


"Toner" tiles by Stamen Design

. . . I'd like you to make maps intended for online viewing. 

This is partially a selfish impulse; I'm going to throw a rock through the window if I see another "PDF Download" link masquerading as a web map. I'd also like you to do this to save a few trees from their 24"-by-36"-poster fate, but mostly it's because we're at an inflection point in cartography . . .


View Larger Map
Routing application with Google Maps

. . . I think the public - the folks who show up to input forums for development projects, and who want to know how far it is to the next lean-to on the trail - are now fully-literate in maps. Mapquest started this process and Google advanced it, but now there are tools beyond those to help you engage your audience on computers and mobile devices (No, Trimble, I am not referring to the Juno. Screw that.). Many of these tools happen to be open-source . . .


Ecoregions of New Zealand, hosted by Github

. . . If you currently describe yourself as a "GIS Analyst/Technician/Monkey" - as I did not long ago - your day largely consists of desktop GIS, and 95% of you are working entirely with some combination of ESRI products. I frequently slag ESRI for bugginess and closed-ness, but they do just fine making a desktop platform for geospatial analysis. However they have never known what "Cartography" means (that's right, you take it to Illustrator if you want your map to look good), and they're just starting to realize that they don't know what "Web Mapping" means either. . .


Population Change in South American Cities, using Mapbox 

. . . The "open-source alternatives" are not replacements for ArcGIS. But through advances in cartography and and browser-based distribution, open tools are unearthing the truth that the rest of us forgot about while sorting through ArcToolBox: that maps tell stories. That your data can be weighted, decoupled, buffered and regressed twenty ways and it means nothing if your cartography is crap. That you can drop cash on a printed poster with a sweet imagery background and it's lost to the ether if two people show up to your hearing, then you post it publicly as a PDF.

These are issues that bedevil public and private sectors, academic and amateur alike. But I say that you can make accurate yet artistic maps and deliver them in a way that is almost universally accessible (at least in the developed world). Let's make maps that people care about.

With the 'Why' of it behind, my next post will be an introduction to the 'How.'

Read more ...

FOSS4G - North America 2012: Observations

Saturday, April 14, 2012

I had the good fortune to make it to this year's iteration of the Free and Open Source for Geospatial (FOSS4G) tech conference - North American edition (other sections of it will be occurring in Germany and China). Having missed the O'Reilly festival of place, it was a great opportunity to catch up with - as one attendee put it - "The real makers in the world of maps". [Sorry, ESRI, you've actually been standing in the way of cartographic explanation for about a decade now]

While there was more afoot than I could ever have followed entirely, I took note of two items:

1.) People are really jazzed about PostGIS. Core architect Paul Ramsey kicked off what would be three days of love for the new release (version 2.0), and we were treated to all sorts of examples on how you can use it. Raster support, TileMill optimization, simple SQL strings replacing reams of jagged python for geoprocessing workflows - the possibilities seem endless.

I really like that such a great, extensible project is at the heart of open-source geographic technologies, but I still feel left out of the party. Basically, I am intimidated by the first hurdle of PostGIS: PostgreSQL at the command line. This is not that big of a deal for true developers, but I've been repeatedly burned by the wonkiness of database administration, permissions and security in postgres. Once I can get to the actual SQL, the birds start singing, the sun comes out and the world is full of possibilities, which is what makes client-focused tools like CartoDB and QGIS so enjoyable.

2.) Web maps are the home of cartographic design now. This may have been the purview of National Geographic even five years ago, but static maps are lacking the possibilities laid out - for example - by AJ Ashton and Nathaniel Von Kelso in sessions that touched on using OpenStreetMap as both art and function.

MapBox "Lacquer" Tiles

Stamen "Watercolor" Tiles
Take this with a word of warning issued by Schuyler Earle during a panel session: Software developers think they are natural designers. They're not. They need to hire designers, who can - like sober friends at 2AM - shepherd us away from bad decisions. Users will benefit if we can all get a design perspective on "I think this thing needs another button on the sidebar" before we act.

Many thanks to the organizers of FOSS4G-NA. It was a rockin' good time, full of productivity and great encounters. And for those who need a lingering shot of natural-disasterage, here is my ignite talk from the first night. Heavy on pictures, low on explanation, much the way it should be:

Read more ...

OpenstreetMap and the Bulk Battle

Wednesday, April 4, 2012

I like building footprints. As a cartographer, I find that they bring important context to a certain scale of map. In the same way that I look for patterns and paths in the contour lines of a national park, I see explanations of how we live and work in the layout of buildings in a city. Basemaps for the web applications I write always benefit from having structures as a ground reference.

Over the past few weeks I've been working to add about half a million building footprints to OpenStreetMap, mostly in the Mid-Atlantic region of the United States. The excellent data is the result of years of work - ongoing in many sites - by the Spatial Analysis Lab at the University of Vermont. They've fused LIDAR data with multispectral aerial imagery in an object-based classification of land use for their client municipalities, and they're happy to offer the results to the OSM open license.

Here's the current OSM Mapnik render in Maryland Park, just outside of D.C. -

And here's the same area with UVM-SAL building footprints added -

I've been careful to discard footprints that overlap with an existing OSM line or polygon feature, since user-traced content takes priority over bulk imported content. But having taken this data to the OSM community for review, I am now aware that it's a complicated thing to "Add data" in bulk to a crowdsourced database. Users have eloquently argued both for and against the import of these building footprints, and I find it boils down to a pair of inextricably-linked perspectives on OSM:

1.) OpenStreetMap is a community of individuals. The interaction between the user and the map is most valuable when one user "owns" their offering, and a dozen building-footprint-tracing contributors from Baltimore will feel greater ownership of their neighborhood than one guy in Vermont who uploads the whole county in one batch.

2.) OpenStreetMap is a tool. It is becoming a basemap of record in the GeoSpace precisely because that's what we wanted to do with it. OSM competes admirably with proprietary datasets, and we can use it for beautiful cartography and complex analysis. Any scale of addition is valid, as long as it is offered on the same open license.

The question that lies between these two perspectives: Can we have a quality basemap while maintaining a strong commitment to individual user engagement?

I think the answer is yes. I'm going to proceed with my buildings import on an assumption that was echoed by Kate Chapman: If a user in Bethesda finds that the buildings in her neighborhood are "already there", maybe instead of losing a sense of ownership she'll take it to the next level by adding identities to those structures.

Because I'm just adding the outlines. I can't tell you which one of them is a Bodega. There are many layers of value yet to be added.
Read more ...

WhateverConf 2012 - Notes from a Nonparticipant

Tuesday, April 3, 2012

So this O'Reilly chap evidently has some soiree planned every year this time, somewhere on the West coast. In the past, I've been able to blithely ignore the proceedings and hope that someone else would eventually point me toward anything useful that arises. Basically the same attitude I have toward SXSW.

Regrettably, this year I'm being forced to perk up from my West-coast-of-New-England-afternoon-coffee stupor and pay the hell attention. Three items caught my eye on Twitter today and yesterday:

1.) Custom cartography for Mapbox Streets - The Development Seed team clearly proceeds from day to day in a Red-Bull-fueled tornado of productivity. They've caught on that the momentum is away from the ubiquitous Google Maps API and base layers, and they're doing everything they can to offer a replacement for the whole stack. In particular, they've done Google one better here on cartographic quality. MapBox's OpenstreetMap-based tiles are uniquely beautiful, and sufficiently-varied that we users might now be able to escape the homogenization of web map styles over the past few years (Yes, Microsoft, even your basemaps look a lot like Google's). Cartographic design just won big. Here's some Tilemill-styled twitter data of mine over the new Mapbox "Lacquer" Base tiles:



2.) Foursquare has a new geocoding engine - Saints be praised, another way to escape Google. I don't yet know how solid Foursquare's placename database - called TwoFishes- will be for everyday applications. However, my current reality has me tethering whole projects to the GMaps API specifically because the Google geocoder is the only one to consistently find the correct location when a user searches for an address. I need alternatives, and they are particularly welcome from a company that is already showing a dedication to open geodata.

3.) CartoDB murders the middle of the GeoStack - The team at Vizzuality has been intimidatingly-busy as well. Though it may have been overlooked in the WhereConf hoopla, yesterday they teased out a project called VECNIK; at the moment it consists of a series of HTML5-ready libraries for fast vector styling directly in the browser. This includes pulling the parsed vector from the already-smooth CartoDB SQL API, compiling a stylesheet in the super-flexible Carto language, rendering the whole thing with Mapnik and serving it into the ModestMaps library - entirely in the browser. This happens at speeds that ten months ago would have been achievable only with cached tilesets. As with basically everything Vizzuality does, it's going to take me a few more months to grasp the full implications of this advance; I'm psyched enough for the moment though. Here's an example showing "Hand-sketched" parcel boundaries in my neighborhood (Modern browser required):



So to anyone else who's planning to drop major innovation bombs on the GeoSpace at WhereConf, $#@%in' stop. I've had enough for this week. Don't even get me started on PostGIS 2.0. Save it for FOSS4G-NA in D.C. next week, by which time I hope to have recovered enough spare neurons to take it all in. See you there.



Read more ...

Zombie Lunchpocalypse: An Iron Sheep 2012 Entry

Monday, March 12, 2012
Thanks to Matthew Zook and the folks at Floating Sheep for organizing the Iron Sheep 2012 open data hack in NYC. While some of the teams were tempted to go with a serious analysis of social/spatial issues, most of us couldn't resist running with the frequency of Twitter and Google uses of words like "zombies" and "aliens". Our entry to the competition:




"The humans are gone. Long past is the zombie apocalypse. No human brains remain for feasting. So what is a hungry zombie to do but seek out sheep brains, according to the dietary preferences they might have once held as a living hipster in Williamsburg? Lunchtime finds discerning zombies wandering the five boroughs in search of organic sheep brains . . . 

The Lunchpocalypse tool is meant to aid the enterprising zombie restaurateur in locating and exploiting key demographic clusters of organic-preferring zombie consumers. Simply plug in a NYC address, and you'll be able to see what the local preference is, leveraging a geodemographic segmentation analysis of the frequency of internet searches of the words "zombie" and "organic". Maximize your zombie profit and shuffle along with your organic herd to where the customers are! Because the dead are damn hungry!"
Read more ...

SOPA 2014

Wednesday, January 18, 2012

While I'm aware that dystopian fiction is more the field of Newt Gingrich, I had a few thoughts today as the SOPA/PIPA battle continued to rage across the interwebs and traditional media both. You see, I'm a cartographer, and visualization is my stock in trade. As a cartographer with a particular interest in Crisis Mapping, I realized that the bills now before the House and Senate could have some as-yet-unexplored consequences. Let's explore some of them:

**************************
January 19th, 2014
  • At 8AM, the shelling begins in Acarigua. Venezuelan government forces are trying to push their way into the center of the city, where pro-devolution demonstrators had clashed with - and ultimately expelled - the federal police the previous day. Now, as the troops begin to retake Acarigua block by block, panicked residents scramble for cover while others erect barricades on the main streets.
  • Eustacio Torres was living in Pittsburgh on a guest worker visa when the news arrived that Acarigua had rebelled against federal Venezuelan control; he was immediately worried for the safety of his mother, who still lived there. At 9:10AM, some mobile phone providers' lines are jammed with traffic and others have obeyed a shutdown order issued by the government. Eustacio is not alone in his concern; thousands of others in Venezuela and beyond are looking for news from Acarigua.
  • At 9:15AM, Belen Diaz - a software developer in Bogota - registers the domain name ataqueacarigua.org.co and hosts it to her server on a local ISP. She deploys the Ushahidi crisis monitoring platform and attaches it to that domain. By 9:55, after some calls to news organizations and social media outlets, the site begins to see significant traffic. Hundreds of users post updates and comments as news appears. A small but key number of reports filed to the site come from residents of Acarigua, sent by SMS on the spottily-available signals to a dedicated shortcode registered by Belen.
  • At 11:43AM, a freelance journalist working for News Corp. takes 17 seconds of footage showing soldiers firing live rounds on a group of apparently-unarmed young men in Acarigua. Unseen in the confusion, he manages to quickly offload the footage to New York via satellite uplink.
  • At 12:34PM, as the shelling tapers off and automatic weapons can be heard closer by, Anamaria Torres asks her nephew Jaime to call her son Eustacio, saying that she's safe for the moment but scared. Jaime tries, but can't connect the international call. He then posts the message by SMS to the Ushahidi shortcode. The message reaches ataqueacarigua.org.co.
  • At 1:12PM, Belen sees the message posted by Jaime, marks it as plausible but unconfirmed, and posts it to the site for all to see.
  • At 1:14PM, Jacob Crenshaw in Dallas sees a grainy clip of soldiers firing on civilians in Acarigua, as part of a Fox News analysis segment. He records it off the TV with his phone camera and uploads the footage directly to ataqueacarigua.org.co. A volunteer working with Belen sees it and posts it to the public view.
  • At 1:56PM, a Fox web technician is alerted to the unauthorized rebroadcast of footage on ataqueacarigua.org.co. The News Corp. copyright-dedicated legal team submits an urgent takedown request to the U.S. Justice Department. The A.G.'s SOPA-compliance division grants the request, noting that the appropriate boxes are checked to meet the burden of proof and that the paperwork is in order.
  • By 3:45PM, the domain ataqueacarigua.org.co has been blocked and all search engines have removed it from their listings. Attempts to broadcast the direct IP address are similarly blocked and sites sharing it are added to the injunction.
  • At 5:22PM, reports are still coming in to the server, but less than 5% of visitors are able to view the site.
  • At 6:40PM, Eustacio is able to access the internet using a friend's laptop. There is no communication from his mother, and the front page of ataqueacarigua.org.co reads "This domain has been blocked pending legal action as per the provisions of H.R. 3261, the Stop Online Piracy Act" . . .

********************
So this is nuts, right? But I've grown a bit weary of hearing that SOPA and PIPA will "Only impact foreign sites dedicated to infringment." Many of us work across borders, and our world is interconnected in ways that our Congress doesn't seem to understand. But more specifically, does anyone believe that the Justice Department will get right on SOPA compliance requests with lots of funding and an adequate staff? Or that they won't be carroted and sticked into pro-forma approval of most requests the same way the major banks bulldozed the OTS in the run-up to the 2008 economic crash?

These bills are a real threat to freedom, at home and abroad. We should be concerned about both. This has been a whimsical case study, but it's because these bills show a lack of imagination.
Read more ...