Pages

Showing posts with label web map. Show all posts
Showing posts with label web map. Show all posts

The District Plows Tonight

Thursday, January 2, 2014
It will surprise no one in Vermont if I say it's cold tonight.
So I'm indoors, and bemused to note that it's also snowing down in the nation's capital, where such things tend to bring out the crazy in the more troglodyte-like elected representatives.

On the fun side, during the last storm in D.C. Jeremy Bowers of NPR wrote a handy script ("Mister Plow") to scrape location data for the city's plow fleet and offer it up as a live feed. Since this sort of screamed for a map, I built one, and now it can serve for this storm:



It's lacking a good temporal symbology, but it'll do to see if the plow has been by your street recently. The mechanism is Leaflet, Mapbox Satellite, and a dose of PHP to parse the feed. Grab it here if you dare.

I FOUND THEIR NEST! DEAR GOD, THE PLOWS ARE SWARMING!
Read more ...

How to build a Mobile-First Map Form

Friday, November 8, 2013

Find me, small screen.

I've spent a lot of time thinking about the incredibly rich experiences we're getting these days from a digital map paradigm basically invented by Google in 2005.

A rich experience everywhere but on mobile devices, that is.

The dichotomy is pretty intense - on our larger screens, we have data visualizations of artful grandeur. We have amazing cartography in tiles and vectors, and we have smooth analytical tools. We're even approaching "GIS on the Web" that doesn't suck.

But then you turn on your phone and the best applications use oblique-angle maps the same way they did in Car SatNavs a decade ago.

I don't have an answer to this problem, I just mean to say I've been thinking about it. And it turns out that a bunch of the technologies I use regularly can be pretty easily combined to offer some possible fixes for how a user experiences maps on mobile. I'll walk you through them while trying to tackle a basic application: a location-based survey.

1.) Bootstrap

I get it. I overuse Bootstrap for the architecture of my sites. But in the latest release (3.0) they went "Mobile First", so it makes many of my other tasks easy by providing the responsive classes out of the box:


2.) Google Forms

I still think of Google Forms as an underutilized gem for easy data collection: pipe data from a form to a spreadsheet with an API; one barely needs to try anymore :) However, tying a custom Bootstrap page to a specific google form requires a few things available in the source code of the input form's page:
  • Form identifiers:
  • Field-specific IDs:
  • A Redirection iframe:


3.) Mapbox.js

Most of the necessary functions here are actually native to Leaflet, but I like the look of the Mapbox implementation, and they provide "Retina Tiles" that scale better for mobile. The basic tasks for the map in this application are:
  • Figure out user location without asking
  • Let the user fix the inevitably-incorrect positioning
This works with a quick call to a free IP registry, centering the map on the result:


Then the user can drag the map around under the crosshair to where they actually are - this is a tactic I borrowed from OpenPlans after I got a reminder:




4.) CartoDB

Using a new tool they've made available to sync tables, there's almost no code involved in this step. All I needed to do was go to "File-->Publish" in the form target spreadsheet, then copy the .csv URL into the "New Table" creation dialog in CartoDB. Then a bit of easy styling created a custom-mapped results page:


The page itself is pretty simple, and you can grab it as a gist here. It'd be pretty easy to fork it and replace the form parameters with one of your own. As always, please share if you use it; I'd love to see where others take this.

The Caveats and Alternatives:

  • Geolocation by IP is pretty awful, unless you happen to be Google (but in that case you're currently being sued for the method by which you collected your registry). The service I used put me in the correct city on my laptop, the correct state on my tablet, but according to my phone I'm in Buffalo. There are better ways to get location, but they require user permission, which is a UX hurdle I didn't want to jump for this.
  • I subscribe to the gospel of responsive design because I'm lazy. It would probably be smoother to build this as a native Android/iOS app, but no way would it be this easy.
  • Since location is being passed to the table from the initial map, there's no geocoding and you could theoretically just tap the Google Drive API to put points on a results map. But again: EASY. CartoDB is that :)

#Facepalm

In the first run I forgot the mobile-specific meta tag. I was reminded :) Thanks, Vladimir!




Read more ...

5 Reference Recipes

Thursday, October 17, 2013


At the risk of beating a dead horse, I'm coming back to reference overlays. I've noticed that there a still a fair number of interactive maps online that permit a bullying thematic layer to stomp all over the poor, defenseless map labels and context information. This is not cool:



So I've hashed out the code for getting past this using five common basemap services:

Feel free to reuse them as you see fit. All but the Google example use Leaflet, but the z-index and canvas manipulation tactics are applicable to other APIs as well. If anyone wants to add adaptations for OpenLayers or whatnot I welcome pull requests. Also worth a note is that - in the leaflet examples - the thematic layer is rendered as vector from topojson, which is arguably the fastest vector format available to us today. Give it a shot as a thematic mapping tool.

Go forth and bring context back to your interactive maps!


Note: Thanks to Bobby Sudekum, Josh Livni and Jason Sanford for the starting blocks.

Read more ...

Tiled Basemaps Survey 2013 - Results!

Tuesday, August 6, 2013
From Stamen, OSM Contributors and National Geographic

The results are in!

123 helpful participants submitted their thoughts on their favorite basemap services, and without further ado I present some of the results:

Which tiled basemap is most cartographically appealing to you?



Google, National Geographic (served via ESRI), and Mapbox are the winners here, with support for Stamen's options as well. "Other" in this case included OSU's Crinkled Watercolor and OpenCycleMap. Between the Stamen, OSM-Mapnik and Mapbox selections here, tiles based on OpenStreetmap data were ahead of the others. But this is a popularity contest based on looks. Moving on to substance . . .

Which tiled basemap do you use most frequently in your projects?

Google wins on ubiquity. Almost half of the respondent pool uses Google's tiles first; though given the widespread adoption of the API this is not a great surprise. More interesting is that about half of these Google users find a different basemap to be more attractive. Which brings us to . . .

What factors drive the choice of basemap for your projects?


Respondents could choose more than one here, but it's clear that not many balked at the technical challenge or vendor lock-in concerns. Mostly it seems like choices were driven by a balance between aesthetics and quality/detail, with cost and licensing hovering in the back of respondents' minds.

What do you like best about your "cartographic favorite"?

Unpacking these results is best done with a series of [gasp!] word clouds for the top three choices. You might call it a word storm . . .

National Geographic: 

Mapbox: 

Google: 

Interpret these as you will - subjective assessments have a way of bleeding from one choice to another. For instance, it looks like everyone likes the "colors".

Whoa there . . .

This is not a scientific survey. And despite my best efforts to reach out to all the communities involved, it's probably not even terribly representative. But it's interesting to inspect the choices made by this particular group of mappers, and fantastic that so many were willing to share their thoughts. Check out the unvarnished results here - mostly interesting to see the full explanations of what folks like about their preferred aesthetic.

As always, drop me a line if you have any questions or concerns about these results. Thanks to all who participated!


Read more ...

Tiled Basemaps: the Survey

Monday, July 8, 2013


The time has come once again for a taking of the geopulse! In years past I've asked broad questions about platform-level technology use, but this year I'm getting more specific:


Yes, I realize that the very concept of tiles is about to go the way of the Avenue script, but the present moment is still flush with amazing cartography served up in square chunks. For this survey I've chosen a selection of "Streetmap" styles from the major players, since imagery is another can of worms entirely.

I've built a comparison app here (with thanks to Dair Grant for the idea). Check it out and vote for your favorite.

Note: It's not possible to get a perfect comparison, since some maps tiles are locked to a particular API (what up, Apple), and others were too hard to parse into a single web map (Was a quadkey scheme really necessary, Bing?). Feel free to add these in if they serve as your go-to basemaps anyway.

Thanks for participating, and check back here for results in a few weeks!
Read more ...

5 Examples of an Inflection Point for Maps on the Web

Monday, June 3, 2013
Some recent thoughts by Brandon Rosage reminded me that webcartographics (I should trademark that one) are the midst of a change. Since the dawn of Mapserver, we've been building maps into our websites in a segregated fashion, often treating the map as little more than an <iframe> keyed to a CMS. This works well enough in a short blog or location description, but there are more engaging stories to tell with cartography, and better ways to deal with UX complexity than just adding more markers.

Fortunately some great developers are steadily building tighter integration between geography and web design. My favorite recent examples of this follow:

1. Chasing the Boston Bombers

A maps-in-the-background piece on the Boston bombings from the New York Times Graphics Department uses a Google static map as a location-based table of contents to a scrollable storyline highlighted with more maps and photos.

2. The Royal Navy in WWI

From Vizzuality, a moving record of Allied ship traffic in WWI was derived from their work crowdsourcing the transcription of old ships logs. This is more of a show than an interactive piece, which works perfectly to illustrate the flurry of activity around major geopolitical events.

3. The Adventure of the Bruce-Partington Plans

Young Hahn's Mapbox-based adaptation of a Sherlock Holmes storyline changes the click-and-drag paradigm of the web map by tying map navigation exlusively to a scrolling text storyline. Unlike the NYTimes piece above, the map does move - but only as the narrative progresses. Bonus points that it's responsive, making this an equally-rich read on mobile.

4. Crowdmap's New Layout

Crowdmap has some great potential here, though it's still being pulled in several directions by varying stakeholders. The previous version was a codeless window into Ushahidi's functionality, and that was great for easy access to a map-within-a-CMS. But with the current iteration they've tried to bring the focus to a map driven by its content, rather than content accessed via a map.

5. Hurricane Sandy Coastal Impacts

Another entry in the the journalism category, this tour-de-force from Derek Watkins treats the coastal axis (North-to-South) as a narrative, attaching aerial photos seamlessly to a thematic map built with the D3 javascript library. Perhaps the coolest part of this one is what it implies - D3 is one of the best emerging tools for breaking maps out of "Plugin" status and letting them interact seamlessly with the architecture of web design. This is looking more and more like a window into the future of maps on the web.


Read more ...

We Are Still Talking About These Things: Dispatches from the Dawn of Crowdsourcing

Thursday, May 23, 2013


I was just having a conversation with some colleagues about the nuts and bolts of participatory mapping with some local farmers. We were all:
"Man, it totally depends on if you have internet at the site. If you do, just pull up Google Maps or CartoDB and have them digitize right into the iPad. If not it's a huge hassle; you'd have to annotate a PDF and georeference it later."
"Yeah, but either way the stylus is key. Non-technicians always want to use a pen and paper."
This talk of getting a skeumorphic data-entry device in the hands of the common folk reminded me of something from the past. This, specifically:
Even taking as a fact of life that community GIS will always require some mediation by the more technically skilled, Al-Kodmany's Chicago neighbourhood projects are not a little bit extreme. In fact, by the time he describes the community members being given 'coloring the map' participation exercises as a way to actually participate, there is a rather patronizing air about the project. 'Participants were broken into small groups and were given a map and felt-tipped markers,' he writes. Felt so as not to inadvertently poke out their own eyeballs, no doubt.
This from Christopher Miller, writing in 2006 about how participatory GIS (nee "GIS/2") was being hamstrung by academic condescension, cultural barriers and the failure of imagination that is traditional GIS ("GIS/1"). Man, 2006 sounds like a long time ago from here, but we're still spinning our wheels in some ways on this stuff.


Based on Andy Woodruff's Boston-oriented project, last year I built an app to get locals defining the boundaries of neighborhoods in the city of Burlington. When I showed up to the first hearings on city redistricting to present the results, their credibility was questioned by some who hadn't heard of the project, even though it had been announced in every local news outlet as well as digital channels. These were people who showed up to meetings, because that was how you participated. Not learning my lesson, I adapted Azavea's Districtbuilder app for the redistricting process and made it public, hoping "participatory technology" would open a traditionally-closed political world. It turns out that the technology wasn't really the problem.


In his snark-laden, brilliant 2006 paper, Miller goes on to describe this neat little app called Scipionus, built in the aftermath of Hurricane Katrina on the back of a newly-opened Google Maps API. Here's the thing: it let anyone with a web connection report their location and status on a Google Map! Fast-forward three years and Ushahidi comes along. Fast-forward another three years and Google Maps gets collaborative, Crowdmap is everywhere and Twitter becomes a news source. Fast-forward to last month and the most reliable information on a civil war is from a participatory map, and Ushahidi can take in reports from every technology short of smoke signals.

We still don't have good retorts to Miller's challenges. We still haven't made GIS accessible to the public, and I'm not really sure that's a valid goal. More importantly, we seem to be driving deeper into the gulch between Internet-Worshippers and Tech-Hostile Curmudgeons that Miller warned us about. Whether we're community-building, solving disputes or reporting the news, we've got to do a better job building bridges between the public and our maps.

In many cases, "Paticipation" is still a felt-tipped marker jabbed into the eyeball.

Read more ...

Toward an Ideal Geoportal

Thursday, April 18, 2013

A Geoportal Identity Crisis

A city wants to open its geodata for public use. An NGO wants to spark a transparency initiative. A regional planning commission wants to stop emailing zipped shapefiles when pestered. They want to deliver two contrasting products - raw data and parsed themes - to as many as a dozen different audiences: policymakers, technical service providers, the press, professional curmudgeons, etc.

In the past decade, the solution to this nearly-impossible balancing act has been to build a Geoportal, and we've seen some pretty memorable misfires. Most of the trouble can be chalked up to the good intentions of the technicians who produce the data and the tools to distribute it; accustomed to a desktop GIS environment for exploration and analysis, they have built and re-built "GIS on the Web" (Chris Herwig documented his travels around a broad selection of them). They've done this under the assumption that users will be filtering, buffering, selecting by location and overlaying, and as it happens that's not really true. This approach fails all sectors of the public.

For many months now, Brian Timoney has been championing sanity in the wasteland of geoportals. He expertly trolls the mapjunk and the faustian UX, but he's also offered an analytics-based selection of "Best practices" for getting geodata to the public. Beyond how users actually interact with online maps, he's drawn attention to the subtle distinction between "open data" and "useable open data". But he is still something of a voice in the wilderness, as we all nod vigorously in agreement and go back to downloading file geodatabases from the USGS. We work with the system we have, because it's hard to envision the details of the alternative.

A Template

I've been trying to envision such an alternative, spurred on by some adventurous clients. Specifically, I wanted to see if it was possible to crack open a public dataset in a way that was compelling to technicians as well as to the lay public - something that would adhere to emerging best practices as well as to my own bias toward open architecture for open data. Timoney himself has already taken a stab at this, but we have different styles. So I cobbled together a template geoportal, wrapped around a standard multipurpose bit of public data: building records.

Give it a spin here.

I assumed two audiences: 1.) citizens who want their own zoning information and permit history fast, and 2.) analysts who want to grab bulk chunks of building footprint geodata for urban planning, disaster response or just noodling cartography. The former group can search for their address, see their building in context, and get the basic info before heading off to print their fact sheet. One minute or less.


The GIS specialists of the world can hit the download link and vacuum in all the features within the current view extent, in the interoperable format flavor that suits them (+TopoJSON for the bleeding-edge types).


This is just a page for a single dataset, but I think it meets the needs of both audiences and doesn't suck to look at or navigate. A few of the other features I wanted to include:
  • Lightweight and Javascript-based - less than 1MB before the tiles show up.
  • Shareable URLS - the root URL is subject specific, and the location hash allows users to pass around a focused view of a neighborhood or house.
  • Traditional search (in the hanging dialog box) is prominent without obscuring the map, since I'm of the opinion that a map is better as a page canvas than as a tiny sidebar window.
  • Very few visible bells and whistles - This is built to some narrow workflows with no mission creep and no toolbars. A bunch of info is socked away in a modal popup for the intrepid.
  • Cartography - I'm done with auto-pixelated graphics and bad default symbology; this is 2013 and we should all be using Mapnik for the web.
  • Open-source undercarriage - this is a combo of Bootstrap, Leaflet, CartoDB and Mapbox.
  • A note on CartoDB and Mapbox - their server code is legit open-source (meaning I could just run it my own damn self), but I've used their hosted services here since it's just - ack - easier to let them handle the services and flexibility thereof. And still cheaper than the competition. 
It's true that much of my requirements were about what to omit, but it is truly a difficult task to limit the scope of an application that is being driven to omni-functionality by stakeholders. Resistance is part of the process.

Here's the code on github. The index.html is commented liberally, so you should be able to tell where to swap things in and out. Scaling this approach to an entire suite of geodata could be as easy as forking the repo for every dataset you want to present to the public. Single-theme maps will see the most use in the long run, so keep it simple.

The To-Do List

This app needs typeahead in the search box to really deliver options. Andrew Hill at Vizzuality shows how easy this is when pulling from a CartoDB address column, but I'm still trying to bolt it onto the partially-abstracted leaflet geosearch module with my meager javascript skills. I will gleefully accept pull requests. Additionally, you may note that each building's "Fact Sheet" points to the same place. The city of Burlington is doing great things with its data, but we don't have building URLs keyed to parcel IDs or addresses yet :)

The bigger question is one of data discovery; if we're going to limit geoportals to one theme at a time, how do users get where they want to go? I hate being directed to a silverlight-slinging "Map Gallery" when I'm looking for info, but I'm not sure what the alternative is for top-level geodata search. Is it Chicago's spare text-based search? Is it the 300-button web GIS that requires training to use? Is it good SEO?

We know something about how users behave once they find the map they want, but how do we get them there in the first place?

Listocracy from Chicago, GIS-in-a-browser from VT Nat. Resources
Many thanks to Jay Appleton from the city of Burlington for being the single-handed support structure of open data here, including emailing the occasional zipped shapefile :)


Read more ...

Navigation on Planks

Wednesday, February 6, 2013
We're not there yet . . . (Skier dude by Saman Bemel Benrud)
The news yesterday was that Google has added the trails of 38 ski resorts in the US and Canada to its Maps database. Here in Vermont, my stashes are safe from further encroachment, since the only two Green Mountain resorts on the list are perennial tourist-bait favorites Okemo and Stowe. That said, this is a cool move by Google in its desire to permeate our mobile world. I'm used to leaving my phone behind while I ski, but this sort of information would be useful to have at my fingertips if I were in unfamiliar terrain at a new resort. It could also be useful in tandem with the ski-run-tracking apps that are proliferating these days.

However, one item is missing from the new functionality: ski navigation. You can't request the shortest (or gnarliest) route from the top of the Sensation Quad to the bottom of the Alpine Double. These new lines - while color-coded according to difficulty - are just background images in Google's otherwise-rich network of information. The navigation engine suggests that the shortest route down the Liftline at Stowe is to pop off my cables and trudge down the Toll Road or the Long Trail.

The longest way around
I can see why Google would be nervous about offering directions in this context - There are two problems that are unique to alpine skiing as a navigation paradigm:

Obey the Rope.

Trail conditions (especially in the East) change more rapidly than road conditions. However, they don't change more rapidly than traffic conditions, and Google's got that covered in near-realtime in major metro areas. 
Solution: GTFS for ski resorts - While it's not immediately realistic to ask every resort to come up with a trails API, it's possible to work out a common format for trail reports (Y'know, those things you look at in the morning to see where it's not bulletproof) much like Google does for public transit. It would be an afternoon task for a data ninja at Mountain View to figure out a way of scraping and parsing that information on a daily basis.

Gravity's a . . . Precondition.


Leaving nordic and backcountry skiing aside, ski navigation would require consideration of elevation change. No uphill turns allowed. 
Solution: Spatial Analysis -  This is one of those bread-and-butter geoprocessing problems that GIS undergrads are given: calculate a flow accumulation surface, add turn attributes to the trail data accordingly. This is actually kind of a fun one.

Lots of folks are never going to look at a mobile device while skiing, maybe myself included. But for the ones who do, it'd be pretty danged cool to have navigation assistance built into the vertical territory that Google is now adding to its already-thorough map system. Just a few hitches to overcome, and I think they're up to it :)

"If only there were some way I could technify this experience . . ."

Read more ...

Context For Cheap - The Map Reference Overlay

Wednesday, January 23, 2013
I used to hate building reference layers in my maps. Labeling placenames was painful, but nothing compared to the chest-hair-waxing misery of scaling transportation symbology by road class. No out-of-the-box defaults were ever cartographically pleasing, and hours were incinerated in the fires of annotation placement. [Sorry, QGIS and Illustrator - you're just as much to blame here as the Redlands upstart.]

But that's basically all in the past. Not long after I slogged my way into web interactive maps, the crew at Mapbox released the finest preconfigured basemap I've ever seen, "Mapbox Streets". They quickly followed with a dozen attractive starter styles and an impressive customization pallette.

Mapbox Streets

The idea of the pre-baked map was nothing new - Google, ESRI, GeoIQ and others had entered the web map age with variations on the Reference-Canvas idea: "You provide the overlay data and the story to tell, we'll provide the geographic context." And this works really well for monodimensional Point of Interest (POI) data, as demonstrated by the Google pushpins that these days rain from the sky to skewer every interesting set of coordinates on the planet. Pins and icons don't get in the way of labels and reference features.

The siren call of the martini glass . . .

But what about polygons and the world of the choropleth? Not that it stopped anyone from trying, but polygons on top of a reference canvas either obscure the features beneath or require too much transparency to make a thematic point. I bombed out on early attempts to work with this essential truth:

NYC Metro, covered in bubble bath

The solution to this problem is under our noses. I first noticed this technique when John Keefe and Steven Melendez at the WNYC Data Desk posted a Mapbox-based interactive looking at NYC's proposed wards; the streets were curiously visible above the color-coded ward polygons.

They had introduced me to the reference overlay.

Leveraging the customization options of Mapbox streets, they had
  • Winnowed out the layers representing land and water from their basemap, leaving just roads, land use and text,
  • Set these to a modestly transparent level (maybe 30-40%),
  • Using the compositing of the Mapbox API, laid this semi-transparent layer on top of the thematic polygon layer, inverting the standard reference canvas model
After I recovered the pieces of my brain that had exploded out my ears (maybe I'm easily impressed), I set to applying this tactic to my own maps. I also realized that this could be expanded to allow for a sort of map sandwich, with land and water below, thematic data next and reference data on top:


And it's not just a Mapbox thing:
Hell, if you can do better than AJ Ashton and company, build your own mostly-transparent reference overlay and cache it in a tile server for future projects.

Subtle Context on the Census Dotmap

While I realize this is all old hat in the GIS world (yes, of course you place your labels above your polygons dude), it's usefulness in web mapping can't be overstated. The reference overlay saves us serious time, solves the "mashup" problem, and lets us focus on our data and what it has to say.

Update, Wednesday Night:
After conversations with some of the Stamen and former-GeoIQ folks, I figured it'd be worth comparing what happens when three different teams build reference overlays from the same (OpenStreetmap) data. Check it out here.







Read more ...

HTML5 and the Limits of Imagination

Tuesday, December 11, 2012
I just (days later than everyone else) came across this awesome Google Maps application built by Darren Wiens using the buckled-on graphics capabilities of paper.js. It strikes me as a perfect example of good imagination at work on complex geographic problems. Specifically, this is a mixture of pure javascript, HTML5 browser-boosting and advanced trip routing, all to make a cool visualization of where traffic bottlenecks will arise.

The application calls on paper.js to generate an endless series of random point-to-point trips within the map view, then passes them to Google's routing API and renders the result visually as a trip on the local road network. Over a short time, the traces of the trips build up into a density map of highest traffic:


So as is my wont, I grabbed the code (Thanks again to Mr. Wiens) and adapted it to my home territory of Burlington, VT, with a bit of map styling for contrast (modern browser required):



I intend to spend a lot more time figuring out the ways that new graphics modules can interact with the maps I build, but as always I'm limited by my own imagination. It seems like the technology is increasingly unlimited in capability.

Read more ...

The Buffalo in the Room Part 2: Fade Out

Sunday, September 16, 2012
In the last post we looked at the many difficult paths that can take us to the summit of cartographic nirvana known as the "Buffalo Tint", as rocked by National Geographic Maps and others

As I noted, this effect has traditionally been impossible to pull off in a GIS platform like ArcMap or QGIS. Tilemill initially got us a little bit closer by giving us full control over styling possibilities with CartoCSS code.

But now, as of Tilemill 0.10.0, compositing functions make this kind of effect a snap. Let's look at making a full Buffalo Fade, still using South Sudan as an example. Specifically, we're going to make a fade mask in Tilemill that can be laid over some Mapbox base layers in a web map.


Step 1: Preprocessing a Mask

This step - preprocessing in a GIS platform - is optional, it just depends on where you want the fade to begin. The purpose of preprocessing is to create a fixed feature mask;

  • QGIS: Run a buffer on your focal feature, larger than the convex hull of the feature for good measure. Then run a difference process between the buffer and the focal feature. 



Either way you're aiming for a feature mask that looks like this:


Step 2: Into Tilemill


Then import that feature mask into Tilemill and style it with what might be the most efficient piece of code I've ever cobbled together, compositing the feature mask to fade inward from its border:



[Alternately, in this case you can do it by just compositing every country that isn't South Sudan and eliminate the buffer processing above. Here's the CartoCSS to do that]

That's it. Export to MBTiles format and drop it on top of a base map of your choice. You're off to the races:



There are still a few bugs when using this for dynamic tiles, notably some tile-edge artifacts that break up the smoothness. But overall I'm looking forward to messing around with these new compositing capabilities


Read more ...

The Buffalo in the Room Part 1: Fade In

Wednesday, September 12, 2012
Since the ancient days - well, since late 1993 or so - production cartographers have been been stuck in an awkward technical limbo between GIS and art. Two platforms were required to get a map from vector geoprocessing to publication-quality graphics: Mostly ESRI's ArcGIS for the former and mostly Adobe's Creative Suite for the latter. Sure the two tried to overlap each other as time went on ("Seven hours to export a 900dpi TIFF and I can still see the pixels? Thanks ArcMap!"), but the basic math was tough to overcome: with finite memory on a workstation, ArcGIS focuses its resources on geoprocessing at the expense of the graphic outputs, and vice-versa.

This was the way of it when I started mapping. And the classic example of "You can't do that in GIS" is the buffalo tint popularized and used to wicked effect by National Geographic Maps. Basically it's a targeted feature fade, meant to draw attention to a focal point or to one side of a divide. And pretty hot too.

And it's not really possible in ArcMap. Here, let's try doing the inverse of a fade, which is easier to envision. This is more of a halo, and it's theoretically possible to do this by adding line layer after line layer, each offset and transparent-ed a bit more than the last:

Buffalo Halo a la ArcMap
Not too shabby, I suppose. A fade out from a clear focal feature. Maybe a bit heavy-handed, but it gets the message across. Too bad it took 30 minutes of clicking into five successive sub-menus on each of ten layers to get it done. And since ArcMap isn't a graphic engine, there's no anti-aliasing, and pixels are visible in every feature. This is not a production-quality graphic.

Let's try that again with Tilemill. I know I know, it's not a GIS engine, but it's a lot closer to one than Adobe Illustrator is, try as they might. Tilemill has full support for operations like selecting and styling by attributes as well as basic geoprocessing if the data is tied to a source like PostGIS, Google's data API or CartoDB's SQL API. Also it's free and open-source (I love that such news is ancillary to my point here. Woot!). As I've mentioned before, Tilemill brings the efficiency of CSS code to the map styling process, and it pushes everything through the sophisticated Mapnik graphic driver to look damn pretty for web or print.

Code will save us, right? Here, check it out:

Buffalo Halo a la Tilemill
This is a more subtle effect, with no striping artifacts, and all the linework is anti-aliased for smoothness. Bonus points for also providing an interactive output where the halo scales dynamically.

So what kind of Carto CSS went into that? Oh, just more than three hundred lines of recursively offset style code. Oy. It's true that it's portable (feel free to plug the code into your own project), but it's not ideal. Definitely not for fast projects under a deadline.

This is where compositing comes in. Last month, the indefatigable Mapnik team added support for the graphical magic that underpins programs like illustrator. This is part of a long-running effort by cartographic designers at Stamen and Development Seed to get out from under the Iron Adobe boot. (or the supple GIMP moccasin, I suppose). With compositing, all sorts of things become a lot easier to do in Tilemill, for instance what we've been trying above is now about 30 lines of CartoCSS, and much richer:

Buffalo Halo with Mapnik Compositing Mojo in Tilemill
The possibilities are sort of mind-boggling, and I invite all the actual graphic designers of the world to figure them out (The composite parameter alone in CartoCSS has 35 options). In the meantime I'll continue to look for ways to enhance my mapping toolkit; the next post will focus on reversing the direction of this effect, like in the NatGeo example linked above.

Free-Range Buffalo Halo, Thriving in its Natural Environment.
Thanks to Dane Springmeyer for pointing out the time-saving parameters on this one.

Read more ...

The Official Takeover

Wednesday, August 29, 2012
High Seas by AJ Ashton. Pirate by Lego, clearly.
ESRI has made some interesting moves in the web mapping space in the past year. I don't blame them for being desperate to get a grappling hook up on a ship dominated by a combination of Google Maps and an open-source polyglot, but at this point the attitude is starting to border on dickish:

  • Step 1: ESRI rumbles toward a unified way of getting licenseholders' data online, while pulling a group of early innovators - GeoIQ - onboard to help. Admirable.
  • Step 2: ESRI adopts the term "Web Map" along with "Story Map" and a few other items that they clearly stole. No worries; a bit of rebranding and we're all one big-happy-web-mappy family, ESRI, Google, FOSS4G and your grandmother with her Bing API side project.
  • Step 3: ESRI kicks everyone else off the boat they just boarded. Now only ESRI makes "official" web maps, and clients should be wary of cut-rate imitators. Specifically a "Web Service" is only an ESRI REST service, and a "Web Map" is "[Like an] .mxd file, but for the web." 
Okay, this is an oversimplification - particularly that step 3 doesn't come from ESRI directly, but from a well-intentioned partner. Well-intentioned because the author clearly wants the "Average GIS professional" to have access to the brave new world of the cartointerwebs and that path is not currently an easy one. But this attitude doesn't arrive in a vacuum; ESRI has cultivated it in the hope that no one will notice they didn't innovate the web mapping space. Web developers did and still do. Folks from tiny open-source shops all the way up to search giants.

Fair play to the big guy with the marketing budget, you say. But here's why I want the developers that built this ship to retain control of it: THEY'RE BETTER AT IT. The user experience is uniformly superior in non-ESRI web maps, the implementation costs are lower and the data is faster. Though this will change and ESRI will catch up if new hires like Sean Gorman have anything to say about it.

But for now I would love to see a bit of humility and willingness to listen on the part of the GIS giant, instead of taking the ship by storm and kicking off everyone who knows how to steer it.
Read more ...

Results of the Geospatial Technology Users' Poll 2012

Friday, July 20, 2012
Update 7/23/12:  The poll is now reopened and live results are appearing at a new post. The figures and discussion below should be considered preliminary

Thanks to all those who hit yesterday's poll of technologies at work in the geospatial field. I've got some interesting results below.

First a note on experimental design: This crap is not scientific. First I tweeted, facebook-posted and Google+'d, so I got in contact with the core community of geogeeks with whom I regularly interact. Then I sent it out via the Vermont GIS listserv, the ESRI user conference hashtag and the O'Reilly open-source conference hashtag, hoping for balance. There is surely a geographic skew toward the U.S. Northeast, but I'm pleased with the general distribution of respondents. n = 117, which seems pretty good to me. Hit me on Twitter or on the GeoSprocket contact page if you'd like a copy of the raw survey results.

Here's a look at the participants using the generalized locations of reported companies/institutions (lots were left blank, so who knows):


The results of question 1:

Note: Some of the technologies that went into the "Other" column include FME, MicroStation, ENVI/IDL, GIS Cloud, AutoDesk, Maptitude, Idrisi, Mapserver and Geocortex. Sorry to have ignored those, but it's a big ecosystem out there.

And the results of question 2:
Ayup, ESRI Desktop is the big winner in this circle. But a surprising number of Google Maps folks there too. Also intriguing is the even split among the open-source toolset types, contrasting with the topheavy ESRI lean toward desktop.

Here is primary toolset use by overarching category:


Things get interesting when we parse out some conditional results:
  • 40% of users whose primary tool is an ESRI product have also used an open-source geo platform in the past year.
  • But a whopping 80% of users whose primary tool is open-source (desktop, web or DB) have also used an ESRI product in the past year.
  • Same with Google - 80% of respondents who primarily use Google Maps have also used an ESRI product in the past year.
  • That favor is largely returned - 75% of primary-ESRI users have used Google Maps.
  • OpenStreetmap and GeoCommons had plenty of casual users, but very few used them/built them as their primary tool (1% each).
      There's a venn diagram to be had in there somewhere, but I'm not up to it.

Without leaping to conclusions, I would say that it's still an ESRI world. Even the folks whose day-to-day revolves around open-source or Google tools still fire up an Arc license every now and then. The converse is not equivalent; fewer than half of ArcJockeys use any of the open-source tools, though they are partial to Google Maps.

There are a lot of potential reasons for that, but it seems safe to say that open-source geo is still developers' territory, and Google mapmaking tools are more comfortable ground for ESRI's users. I recall that specific path when I was making my own way from ArcGIS to GDAL and Javascript.

There's a lot to read here; what are your thoughts? Anything surprising?


Read more ...

Browser Cartography: Some Safehouses for ESRI Refugees

Wednesday, May 30, 2012
"Island" by Konstantin Kafer

A Primer for Getting Started With Open-Source Web Maps


Now that you know why I care about telling compelling stories with widely-distributed maps, let's look at a few of the many tools that are out there to help the process. I confess to narrow experience here; I use MapBox and CartoDB for the majority of my projects, and there are plenty of alternatives to those. But as a starting point I think that these open-source web map design platforms are perfect - they minimize the amount of code required, they use the best graphic rendering engine in the field, and they are extremely cheap (or free) to use, even in an enterprise or high-traffic environment. I'm avoiding ESRI's "ArcWhatever Online for Server" options because of a.) the high price tag, and b.) the actual user-facing sites are only as robust as the javascript or flash developer who builds them. My preferred options give you a lot more to work with out of the box, for free.

I am not going to walk you step-by-step through the process below - I'll point you to resources that will - but rather I mean to sketch the structure that can get your map applications up and running. If you're stumbling and need some help, drop me a line through the GeoSprocket contact page or on Twitter

Step 1: Data


Data in this context can mean a lot of different things. The tools we're using aren't picky, so this includes:

  • Shapefiles (but this will need to be in a compressed folder)
  • Spreadsheets (CSV, XLS, DBF, you name it)
  • KML or KMZ
  • GPX or in many cases raw text from various GPS units

Desktop GIS isn't dead. Those who say so aren't paying attention. You still need some form of spatial data manipulator to analyze and prepare your source information, and if you can do that with pure GDAL hacking at the command line of a virtual machine, you have no need for my advice. Yes, "Desktop Platform" includes ArcMap, but if you want to make a clean break I recommend Quantum GIS for robust, open-source geoprocessing potential.


Step 2: Choose a Platform and Get a Hosting Account

Your data should live on the web somewhere - where, exactly?
  • Option 1: MapBox - Most advantageous for its speed and complete cartographic design potential. However, once you've rendered your map into tiles (in step 3), there's not much you can do to change them on the fly. Use this option if you have a specific vision of symbology in mind to present to your users, if you have a lot of information to present interactively (i.e. text/charts/images that pop up when a feature is clicked), and also if the underlying data will be accurate for more than a few weeks. Sign up here for the free starter account.
  • Option 2: CartoDB - This is a user-friendly, cloud-based adaptation of the popular PostGIS database architecture. It can run a bit slower than MapBox (since it's rendering on the go), and the cartography options are more limited. However, the provided SQL API means that you can sort, filter and process your data basically in the browser. Go with this option if your application will have a lot of user-generated queries (i.e. "how many points are within 50 miles of this one?"), or if you regularly update the data being mapped. Sign up here for the free starter account (notice the pattern of freeness).


Step 3: Mapping Your Data




At this point using either option, you should now have maps that are ready to be launched.

Step 4: Serving Your Map to the World




Step 4a: Embedding - By far the easiest way to get a URL that you can distribute to your audience. Both Mapbox and Cartodb have fantastically-easy embed interfaces for plugging maps into your blog or website content management system, and in each case they essentially host a full-page map website for you as well. Examples:


Racial Breakdown of Census Blocks in Burlington, VT Using Mapbox


Participatory Farm Mapping in Vermont with CartoDB



Step 4b: The World of Pages, Javascript and Beyond - This creeps a bit further beyond the scope of what I hope to cover here, but this is the ultimate destination for most of my map applications. There is simply greater flexibility in a website or mobile application that you can manage yourself, though it requires some knowledge of HTML, CSS and Javascript.

I say "some" knowledge - I didn't know the first thing about code when I got into this world, but it was amazing how easy it was to adapt a little bit. The resources available in the open-source software community are spectacular; the help and guidance offered to me by experienced developers out of sheer goodwill has been uniformly superior to the high-priced support of proprietary vendors (Trimble and ESRI, I'm looking at you). I am no developer, but with community help I've been able to explore and utilize some of the most exciting tools available to cartographers in our time. And things are only getting better from here . . .

Here are a smattering of places to start for customizing a page of your own:

  • Mapbox Templates - these require a bit of github knowledge, but otherwise they are as close to plug-and-play map sites as you'll find. 
  • CartoDB Examples - Plenty of code available for re-purposing and adapting. In many cases this just involves changing a line or two to point to your own data.
  • Codecademy - Might as well learn how to do this stuff for real . . .


Read more ...