Pages

Showing posts with label MapBox. Show all posts
Showing posts with label MapBox. Show all posts

On Reverse-Engineering a Map Stack

Saturday, January 11, 2014


I'm going to come right out and say this:

You should probably just use Mapbox.

How I came to that conclusion is a bit of a longer story.

The Scene

As a cartographer, I am an unabashed fan of Mapbox. I've been using Tilemill for years, and I love the fully-realized design of the Mapbox Streets basemaps. Even before Google Maps brought the paywall hammer down I was already migrating my clients' projects to the open-source ecosystem based out of a D.C. garage. Controlling so much of the stack in an open-source environment meant less risk to my clients, and in most cases it came out a lot cheaper. The choice was pretty easy, actually. 

Looking around this section of the market I don't believe that Mapbox and Google Maps have anyone to compete with but each other (mobile is another story). They are the only two companies currently offering a rigorous javascript API tightly integrated with attractive map services built for the web. (Debate this in the comments, ESRI, Nokia, and Bing users) There are plenty of a la carte options out there - Stamen's wonderful map tiles, the sheer power of the OpenLayers API - but as a lazy developer I've come to really like working with an integrated, open stack. Mapbox is my current choice, though many use cases all but require Google's services.

The Challenge

An open-source stack doesn't mean a free stack, and Mapbox's map tile charges can rack up quickly if you're not paying attention. Exhibit A: I offered Brandon Martin-Anderson one of my Mapbox tilesets to use as a reference for his Census Dotmap. Several viral weeks later I was looking at overages the size of my annual budget.

An open-source stack may not always be free, but in theory it can be copied and hosted by others. I've done this quite a bit on the javascript API side, mixing and matching tile providers with various client libraries. However, the costs that had begun to concern me were on the basemap end. So I set out to do what Mapbox flat-out encourages you to do with its wide-open codebase:

I would make and serve my own damn basemap. How hard could it be?

The Tools


I started with the open-source Tilemill template OSM-Bright, noting that there are some good examples of it in use out there. I grabbed the current OSM data, piped it into my local PostGIS database (note the hazards of adding many extracts) and spent some time turning it into "Geosprocket-Bright" in Tilemill:


Next came the heavy lifting. I exported a slew of regions to .mbtiles format; I would have loved to build a map of the entire world down to street zoom level, but I thought I'd start more realistically with a global map down to zoom level 9, then a handful of cities down to zoom level 17. I planned to put them on an Amazon S3 instance and tap them directly from the client library, doing an end-run around Mapbox hosting. Sounds pretty smooth, right?

Let's keep track of the time investment, shall we?

  • 4 hours to export everything from Tilemill
  • 6 hours to chop the .mbtiles into 1.6 million PNGs
  • 3 hours to consolidate them all into a single directory structure (because I was too dumb to do that in the last step)
  • 56 hours to push the tiles to an S3 bucket on a 20mbps connection
Obviously these are CPU hours, not billable hours - but it was still more than two days between when my map style was ready and when I could actually hook the map up to a browser.


The Verdict

This is a narrow case where I needed to roll my own tiles and serve them. My map included texture and custom fonts, which is beyond the reach of Mapbox Streets. The total filesize of my exports - even as .mbtiles - was 15GB; that translates to the Premium Mapbox hosting plan, and a whopping $6,000 flat fee per year. That's for five cities - the tiniest fraction of a world of tiles. It's not quite Google Maps Enterprise money, but damn. By contrast, it cost me nine bucks to get all of my map tiles into an S3 bucket, where I'll get billed something like half a cent per 1,000 map views.

But this process will be moot pretty soon anyway; the promise of Tilemill 2 is that you only need to bring your cartography to the table - Mapbox will do the rest of the work to get your style to the browser with vector tiles. They've only done this for a few testers while they're hashing out the details, but odds are there will be a public version in the first half of 2014. 

In keeping with other examples of open-source underpinning software-as-a-service, Mapbox has a sound business model. Despite the huge amount of intellectual capital they've open-sourced, it is still easier and - when time spent is considered - VASTLY cheaper to just use their hosted map services. I suspect they've been totally aware of this even as they release service-liberating tools like Tilemill, OSM-Bright and mbutil. The scale and efficiency of Mapbox make their hosted maps too good to avoid.

I hope my experience here has been instructive to others.

The Product

I did get a map out of my experiment, and it's free to use. If you happen to be mapping in Warsaw, Sochi, Santo Domingo, LA or the Bay Area, I hope it proves useful. Just use this XYZ tile scheme in your client implementation:

http://s3.amazonaws.com/geosprocket/tiles/{z}/{x}/{y}.png

As per the usual OSM license, be sure to include "© OpenStreetMap contributors". Happy Mapping!

Read more ...

The District Plows Tonight

Thursday, January 2, 2014
It will surprise no one in Vermont if I say it's cold tonight.
So I'm indoors, and bemused to note that it's also snowing down in the nation's capital, where such things tend to bring out the crazy in the more troglodyte-like elected representatives.

On the fun side, during the last storm in D.C. Jeremy Bowers of NPR wrote a handy script ("Mister Plow") to scrape location data for the city's plow fleet and offer it up as a live feed. Since this sort of screamed for a map, I built one, and now it can serve for this storm:



It's lacking a good temporal symbology, but it'll do to see if the plow has been by your street recently. The mechanism is Leaflet, Mapbox Satellite, and a dose of PHP to parse the feed. Grab it here if you dare.

I FOUND THEIR NEST! DEAR GOD, THE PLOWS ARE SWARMING!
Read more ...

What a difference a decade makes in satellite imaging

Tuesday, November 26, 2013
It seems like there's an armada of new earth-imaging satellites headed for orbit every day. As we consider the changes afoot in an industry where entry costs have traditionally been measured as a percent of GDP, it's worth a look at the tools of today's remote sensors.

This is a commercial, high-resolution earth imaging satellite before launch in 2001:

QuickBird 1 being prepped in 2001. Image courtesy of DigitalGlobe



And this is a roomful of them in 2013:

The PlanetLabs crew and their fleet of Doves, 2013.

Something has sure as hell been disrupted here. We're about to find out what.
Read more ...

How to build a Mobile-First Map Form

Friday, November 8, 2013

Find me, small screen.

I've spent a lot of time thinking about the incredibly rich experiences we're getting these days from a digital map paradigm basically invented by Google in 2005.

A rich experience everywhere but on mobile devices, that is.

The dichotomy is pretty intense - on our larger screens, we have data visualizations of artful grandeur. We have amazing cartography in tiles and vectors, and we have smooth analytical tools. We're even approaching "GIS on the Web" that doesn't suck.

But then you turn on your phone and the best applications use oblique-angle maps the same way they did in Car SatNavs a decade ago.

I don't have an answer to this problem, I just mean to say I've been thinking about it. And it turns out that a bunch of the technologies I use regularly can be pretty easily combined to offer some possible fixes for how a user experiences maps on mobile. I'll walk you through them while trying to tackle a basic application: a location-based survey.

1.) Bootstrap

I get it. I overuse Bootstrap for the architecture of my sites. But in the latest release (3.0) they went "Mobile First", so it makes many of my other tasks easy by providing the responsive classes out of the box:


2.) Google Forms

I still think of Google Forms as an underutilized gem for easy data collection: pipe data from a form to a spreadsheet with an API; one barely needs to try anymore :) However, tying a custom Bootstrap page to a specific google form requires a few things available in the source code of the input form's page:
  • Form identifiers:
  • Field-specific IDs:
  • A Redirection iframe:


3.) Mapbox.js

Most of the necessary functions here are actually native to Leaflet, but I like the look of the Mapbox implementation, and they provide "Retina Tiles" that scale better for mobile. The basic tasks for the map in this application are:
  • Figure out user location without asking
  • Let the user fix the inevitably-incorrect positioning
This works with a quick call to a free IP registry, centering the map on the result:


Then the user can drag the map around under the crosshair to where they actually are - this is a tactic I borrowed from OpenPlans after I got a reminder:




4.) CartoDB

Using a new tool they've made available to sync tables, there's almost no code involved in this step. All I needed to do was go to "File-->Publish" in the form target spreadsheet, then copy the .csv URL into the "New Table" creation dialog in CartoDB. Then a bit of easy styling created a custom-mapped results page:


The page itself is pretty simple, and you can grab it as a gist here. It'd be pretty easy to fork it and replace the form parameters with one of your own. As always, please share if you use it; I'd love to see where others take this.

The Caveats and Alternatives:

  • Geolocation by IP is pretty awful, unless you happen to be Google (but in that case you're currently being sued for the method by which you collected your registry). The service I used put me in the correct city on my laptop, the correct state on my tablet, but according to my phone I'm in Buffalo. There are better ways to get location, but they require user permission, which is a UX hurdle I didn't want to jump for this.
  • I subscribe to the gospel of responsive design because I'm lazy. It would probably be smoother to build this as a native Android/iOS app, but no way would it be this easy.
  • Since location is being passed to the table from the initial map, there's no geocoding and you could theoretically just tap the Google Drive API to put points on a results map. But again: EASY. CartoDB is that :)

#Facepalm

In the first run I forgot the mobile-specific meta tag. I was reminded :) Thanks, Vladimir!




Read more ...

Seamless Textures for Tilemill

Thursday, October 24, 2013
As evidenced by AJ Ashton's announcement this week, Tilemill keeps getting cooler. With vector tiles and Tilemill 2 on the horizon it seems like anything will be cartographically possible, but I've been having fun while we wait.

Tilemill allows you to fill a polygon or feature with an image instead of a plain color, which makes for some cool pattern and feel options:


However, you need to be careful not to run afoul of the tiling system that repeats any such image in gridded 512px intervals. You can get some funky edges if you're not paying attention. The solution to this is to use a seamless image - i.e. an image where every edge matches up with its opposite. There are quite a few options out there for seamless backgrounds, but not very many that are both square and 512px on a side. So I followed the lead of the talented artists at Stamen and reluctantly opened photoshop to make some seamless texture images of my own for Tilemill.

There are 12 of them, and you can grab them here, licensed CC-by-SA. 

I'm personally inclined to overuse things like this once I've made them available (I used them last weekend at StoryHack, actually), so it's going to take some time before I find the right subtle level. I'll let you know when I get there.

Have fun! And I'd love to see any examples of these in the wild once you put them to use . . .






Read more ...

5 Reference Recipes

Thursday, October 17, 2013


At the risk of beating a dead horse, I'm coming back to reference overlays. I've noticed that there a still a fair number of interactive maps online that permit a bullying thematic layer to stomp all over the poor, defenseless map labels and context information. This is not cool:



So I've hashed out the code for getting past this using five common basemap services:

Feel free to reuse them as you see fit. All but the Google example use Leaflet, but the z-index and canvas manipulation tactics are applicable to other APIs as well. If anyone wants to add adaptations for OpenLayers or whatnot I welcome pull requests. Also worth a note is that - in the leaflet examples - the thematic layer is rendered as vector from topojson, which is arguably the fastest vector format available to us today. Give it a shot as a thematic mapping tool.

Go forth and bring context back to your interactive maps!


Note: Thanks to Bobby Sudekum, Josh Livni and Jason Sanford for the starting blocks.

Read more ...

Tiled Basemaps Survey 2013 - Results!

Tuesday, August 6, 2013
From Stamen, OSM Contributors and National Geographic

The results are in!

123 helpful participants submitted their thoughts on their favorite basemap services, and without further ado I present some of the results:

Which tiled basemap is most cartographically appealing to you?



Google, National Geographic (served via ESRI), and Mapbox are the winners here, with support for Stamen's options as well. "Other" in this case included OSU's Crinkled Watercolor and OpenCycleMap. Between the Stamen, OSM-Mapnik and Mapbox selections here, tiles based on OpenStreetmap data were ahead of the others. But this is a popularity contest based on looks. Moving on to substance . . .

Which tiled basemap do you use most frequently in your projects?

Google wins on ubiquity. Almost half of the respondent pool uses Google's tiles first; though given the widespread adoption of the API this is not a great surprise. More interesting is that about half of these Google users find a different basemap to be more attractive. Which brings us to . . .

What factors drive the choice of basemap for your projects?


Respondents could choose more than one here, but it's clear that not many balked at the technical challenge or vendor lock-in concerns. Mostly it seems like choices were driven by a balance between aesthetics and quality/detail, with cost and licensing hovering in the back of respondents' minds.

What do you like best about your "cartographic favorite"?

Unpacking these results is best done with a series of [gasp!] word clouds for the top three choices. You might call it a word storm . . .

National Geographic: 

Mapbox: 

Google: 

Interpret these as you will - subjective assessments have a way of bleeding from one choice to another. For instance, it looks like everyone likes the "colors".

Whoa there . . .

This is not a scientific survey. And despite my best efforts to reach out to all the communities involved, it's probably not even terribly representative. But it's interesting to inspect the choices made by this particular group of mappers, and fantastic that so many were willing to share their thoughts. Check out the unvarnished results here - mostly interesting to see the full explanations of what folks like about their preferred aesthetic.

As always, drop me a line if you have any questions or concerns about these results. Thanks to all who participated!


Read more ...

Open Cadastral Data: a Github Test Case

Wednesday, July 24, 2013

A bolt of GIS-Manna from Heaven

A dedicated and talented GIS developer at the VT Agency of Natural Resources just finished compiling the most-comprehensive cadastral dataset to which our small state has ever had access. Coincidentally he sent out the announcement while I was annoying a few githubbers about their new geodata capabilities. What luck: a test case.

The dataset - as a shapefile - clocks in at 250MB, representing 289,000 parcel polygons in 191 towns. [The 46 VT towns not included tend to have populations under 200 and a blithe commitment to dusty paper records.] It turns out that this is over the 100MB limit imposed on individual files, so I pulled the dataset, split it by town and converted the outputs to geojson, then pushed them to a repository I just set up. The largest of these is juuuuust over the maximum file size to be renderable in Github's snazzy new Mapbox-based interactive maps, but most of the town parcel maps pop up just fine.

The Implications

Okay, great - another slippy map somewhere on the web. Who cares? These folks, actually:
  • A landowner who wants to look up the parcel numbers of her abutting properties without visiting the steam pipe distribution venue in the basement of her local town hall.
  • A town GIS manager who has a hard drive overflowing with parcel shapefiles, converted CAD drawings and map requests from lawyers.
  • Probably a lot of lawyers. Don't get me started.    
  • The overworked data gurus at the Vermont Center for Geographic Information (VCGI), who - while they're probably the only agency that can handle it - are really reluctant to take on management of a statewide, rolling, versioned database.

What Github provides in this test case, for free:
  • Fast hosting of modestly-large geospatial data
  • Relatively-simple version control of said data
  • Edit access to anyone with a github account and a GIS platform (FOSS or Arc'ed)
  • An embeddable client view for the public for files up to 10MB
  • A robust API for client views bigger than that (for example)
  • A muthaflippin' download button (well, "save page as")
For those of us who have been ranting about geoportals in recent months, this pretty much covers the bases. For free. Github is walking the #opendata walk.


What's Next . . .

There are a whole bunch of caveats here. Biggest is the file size issue (though I've seen Bill Dollins and Sophia Parafina starting to work around that in the past few days), since 10MB is a fine limit for municipalities in the second-smallest state, but Manhattan is a teensy bit bigger. Another hitch is data quality; this dataset is top-of-the-line, but like any other it's missing SPAN numbers, dates and acreages here and there. The vendors will tell you that quality can't be beefed up in a free collaborative environment, without data value-add. I don't know if they're right or wrong.

But I'd love to see a town GIS manager throw a pull request and get this ball rolling. Who's up for it?

The data hub is waiting right here


Update: 7/25/13
I've heard a bunch of awesome questions about the practicalities of using Github this way, so while I am by no means a power user I recorded a quick demo video.
Read more ...

Maps in a Responsive Page

Sunday, July 21, 2013



Heaven help me - I've been building a responsive infographic.

Specifically, I used Bootstrap to flesh out a not-really-infinite-scroll site that summarizes the findings of a survey - done by some #opengov types on social media use in the Vermont legislature. Getting the infographic confession aside, my point in mentioning it is that I wanted to include a map of responses by county, and that posed a dilemma.

I went with a responsive layout for the page because I know that 50% or more of the traffic will be from mobile clicks - the kind of viewing experience that makes a tricked-out slippy map either frustrating or useless. So I'd need something more flexible than my usual portfolio. Fortunately, a few months ago responsive-design-zealot Brad Frost had talked about using a combination of static and dynamic maps for the same purpose depending on available screen real estate. I thought it was cool but at the time had no use for it. Enter this recipe:

Ingredients:
1.) An API that provides both static and dynamic endpoints: Check. Mapbox FTW.
2.) A Responsive layout: Check. Bootstrap is out-of-the-box.
3.) A snippet of javascript to load the map according to screen size: Check. Repurposed here.

I knocked together a dynamic map using Tilemill and mapbox.js - this let me do some cool linking of SVG hover states and popup information windows. As a bonus I could disable pan and zoom to make it more like an interactive image and fit better into the overall page context. [Yes, haters, you're right - I should have done this with D3. Gimme a break; I'll get to it.] I then grabbed the map's PNG equivalent from the Mapbox Static API and plugged it into the code as the "static" option.

In Mr. Frost's original his "embed" option for larger screens is the standard google maps embed string. I wanted more features than were available with a standard mapbox embed, so I built the site separately with mapbox.js and "iframed" it in. This is just as much of an ugly hack as it seems, but it works.

The result is a map that renders big and fully-interactive on tablets and larger:



But shows a teaser image and a link to see more on a small screen:


This is obviously not a perfect solution, but it fits the purpose here. And it's also extendable - consider if you had to generate dozens of pages like this - you could do it just by programmatically changing the lat/lon or index value of both the static and the embed maps. Also worth noting is that this strategy works with both Mapbox and Google APIs.

As we move into a web that's even more solidly mobile-centric, we'll need to keep adapting maps with the resources we have available. It's actually a lot of fun trying to keep up.

Click here to see it in action

Read more ...

Tiled Basemaps: the Survey

Monday, July 8, 2013


The time has come once again for a taking of the geopulse! In years past I've asked broad questions about platform-level technology use, but this year I'm getting more specific:


Yes, I realize that the very concept of tiles is about to go the way of the Avenue script, but the present moment is still flush with amazing cartography served up in square chunks. For this survey I've chosen a selection of "Streetmap" styles from the major players, since imagery is another can of worms entirely.

I've built a comparison app here (with thanks to Dair Grant for the idea). Check it out and vote for your favorite.

Note: It's not possible to get a perfect comparison, since some maps tiles are locked to a particular API (what up, Apple), and others were too hard to parse into a single web map (Was a quadkey scheme really necessary, Bing?). Feel free to add these in if they serve as your go-to basemaps anyway.

Thanks for participating, and check back here for results in a few weeks!
Read more ...

5 Examples of an Inflection Point for Maps on the Web

Monday, June 3, 2013
Some recent thoughts by Brandon Rosage reminded me that webcartographics (I should trademark that one) are the midst of a change. Since the dawn of Mapserver, we've been building maps into our websites in a segregated fashion, often treating the map as little more than an <iframe> keyed to a CMS. This works well enough in a short blog or location description, but there are more engaging stories to tell with cartography, and better ways to deal with UX complexity than just adding more markers.

Fortunately some great developers are steadily building tighter integration between geography and web design. My favorite recent examples of this follow:

1. Chasing the Boston Bombers

A maps-in-the-background piece on the Boston bombings from the New York Times Graphics Department uses a Google static map as a location-based table of contents to a scrollable storyline highlighted with more maps and photos.

2. The Royal Navy in WWI

From Vizzuality, a moving record of Allied ship traffic in WWI was derived from their work crowdsourcing the transcription of old ships logs. This is more of a show than an interactive piece, which works perfectly to illustrate the flurry of activity around major geopolitical events.

3. The Adventure of the Bruce-Partington Plans

Young Hahn's Mapbox-based adaptation of a Sherlock Holmes storyline changes the click-and-drag paradigm of the web map by tying map navigation exlusively to a scrolling text storyline. Unlike the NYTimes piece above, the map does move - but only as the narrative progresses. Bonus points that it's responsive, making this an equally-rich read on mobile.

4. Crowdmap's New Layout

Crowdmap has some great potential here, though it's still being pulled in several directions by varying stakeholders. The previous version was a codeless window into Ushahidi's functionality, and that was great for easy access to a map-within-a-CMS. But with the current iteration they've tried to bring the focus to a map driven by its content, rather than content accessed via a map.

5. Hurricane Sandy Coastal Impacts

Another entry in the the journalism category, this tour-de-force from Derek Watkins treats the coastal axis (North-to-South) as a narrative, attaching aerial photos seamlessly to a thematic map built with the D3 javascript library. Perhaps the coolest part of this one is what it implies - D3 is one of the best emerging tools for breaking maps out of "Plugin" status and letting them interact seamlessly with the architecture of web design. This is looking more and more like a window into the future of maps on the web.


Read more ...

Taking Tilemill 2 for a Spin

Sunday, May 19, 2013

When I took my first GIS class, I was told that 5% of my time would be spent making maps, while 90% of it would be spent corralling data into usable format and 5% would be spent beating an unresponsive plotter with a cardboard tube. This balance has definitely changed as interoperability has gone from prayer to practice, but I've continued to be frustrated by the time I feel gets lost to a missing shapefile extension or a falsely-defined projection.

For all the fun afforded us by Mapbox's original Tilemill map design platform, it's always been sort of a GIS-y hassle to wrangle data into it. Mind you this is slight, glancing criticism - it's NOTHING compared to getting your geodata to work in Illustrator or any actual GIS platform. But I always found myself wishing I could spend less time racking my brain for where I put that awesome building footprint layer, or trying in vain to find the right ORDER BY syntax for a PostGIS datasource.

Perhaps you've heard, but Mapbox sort of solved my whiny problems. Tilemill 2 is in unsupported alpha, but it's already fulfilling my dream of data-agnostic cartography. The basic idea is that the world - as derived from OpenStreetmap - is some pretty centralizable basedata, so why not just tap the source and style it however you like? Instead of pulling extracts or copies or subsets, Tilemill 2 just gives you the whole damn world, all 330GB-and-counting of it, via super-fast vector tiles (great explanation here). You get to style those tiles with the CartoCSS language, and at that point they're ready for your audience.

It is A LOT of fun to have the world at your fingertips:




For the moment there's no export or serving option, and it'll be interesting to see what Mapbox does with this tool in its already-robust custom mapping lineup. It's also worth noting that planet.osm is not ALL TEH DATAZ - we'll always need a way to use local or personal geodata. But this is a great leap for an already-impressive platform.

It's liberating to be able to focus on cartography with the building blocks already in place.


Props to Gretchen Peterson for some great color ideas as I fiddle with this.



Read more ...

Toward an Ideal Geoportal

Thursday, April 18, 2013

A Geoportal Identity Crisis

A city wants to open its geodata for public use. An NGO wants to spark a transparency initiative. A regional planning commission wants to stop emailing zipped shapefiles when pestered. They want to deliver two contrasting products - raw data and parsed themes - to as many as a dozen different audiences: policymakers, technical service providers, the press, professional curmudgeons, etc.

In the past decade, the solution to this nearly-impossible balancing act has been to build a Geoportal, and we've seen some pretty memorable misfires. Most of the trouble can be chalked up to the good intentions of the technicians who produce the data and the tools to distribute it; accustomed to a desktop GIS environment for exploration and analysis, they have built and re-built "GIS on the Web" (Chris Herwig documented his travels around a broad selection of them). They've done this under the assumption that users will be filtering, buffering, selecting by location and overlaying, and as it happens that's not really true. This approach fails all sectors of the public.

For many months now, Brian Timoney has been championing sanity in the wasteland of geoportals. He expertly trolls the mapjunk and the faustian UX, but he's also offered an analytics-based selection of "Best practices" for getting geodata to the public. Beyond how users actually interact with online maps, he's drawn attention to the subtle distinction between "open data" and "useable open data". But he is still something of a voice in the wilderness, as we all nod vigorously in agreement and go back to downloading file geodatabases from the USGS. We work with the system we have, because it's hard to envision the details of the alternative.

A Template

I've been trying to envision such an alternative, spurred on by some adventurous clients. Specifically, I wanted to see if it was possible to crack open a public dataset in a way that was compelling to technicians as well as to the lay public - something that would adhere to emerging best practices as well as to my own bias toward open architecture for open data. Timoney himself has already taken a stab at this, but we have different styles. So I cobbled together a template geoportal, wrapped around a standard multipurpose bit of public data: building records.

Give it a spin here.

I assumed two audiences: 1.) citizens who want their own zoning information and permit history fast, and 2.) analysts who want to grab bulk chunks of building footprint geodata for urban planning, disaster response or just noodling cartography. The former group can search for their address, see their building in context, and get the basic info before heading off to print their fact sheet. One minute or less.


The GIS specialists of the world can hit the download link and vacuum in all the features within the current view extent, in the interoperable format flavor that suits them (+TopoJSON for the bleeding-edge types).


This is just a page for a single dataset, but I think it meets the needs of both audiences and doesn't suck to look at or navigate. A few of the other features I wanted to include:
  • Lightweight and Javascript-based - less than 1MB before the tiles show up.
  • Shareable URLS - the root URL is subject specific, and the location hash allows users to pass around a focused view of a neighborhood or house.
  • Traditional search (in the hanging dialog box) is prominent without obscuring the map, since I'm of the opinion that a map is better as a page canvas than as a tiny sidebar window.
  • Very few visible bells and whistles - This is built to some narrow workflows with no mission creep and no toolbars. A bunch of info is socked away in a modal popup for the intrepid.
  • Cartography - I'm done with auto-pixelated graphics and bad default symbology; this is 2013 and we should all be using Mapnik for the web.
  • Open-source undercarriage - this is a combo of Bootstrap, Leaflet, CartoDB and Mapbox.
  • A note on CartoDB and Mapbox - their server code is legit open-source (meaning I could just run it my own damn self), but I've used their hosted services here since it's just - ack - easier to let them handle the services and flexibility thereof. And still cheaper than the competition. 
It's true that much of my requirements were about what to omit, but it is truly a difficult task to limit the scope of an application that is being driven to omni-functionality by stakeholders. Resistance is part of the process.

Here's the code on github. The index.html is commented liberally, so you should be able to tell where to swap things in and out. Scaling this approach to an entire suite of geodata could be as easy as forking the repo for every dataset you want to present to the public. Single-theme maps will see the most use in the long run, so keep it simple.

The To-Do List

This app needs typeahead in the search box to really deliver options. Andrew Hill at Vizzuality shows how easy this is when pulling from a CartoDB address column, but I'm still trying to bolt it onto the partially-abstracted leaflet geosearch module with my meager javascript skills. I will gleefully accept pull requests. Additionally, you may note that each building's "Fact Sheet" points to the same place. The city of Burlington is doing great things with its data, but we don't have building URLs keyed to parcel IDs or addresses yet :)

The bigger question is one of data discovery; if we're going to limit geoportals to one theme at a time, how do users get where they want to go? I hate being directed to a silverlight-slinging "Map Gallery" when I'm looking for info, but I'm not sure what the alternative is for top-level geodata search. Is it Chicago's spare text-based search? Is it the 300-button web GIS that requires training to use? Is it good SEO?

We know something about how users behave once they find the map they want, but how do we get them there in the first place?

Listocracy from Chicago, GIS-in-a-browser from VT Nat. Resources
Many thanks to Jay Appleton from the city of Burlington for being the single-handed support structure of open data here, including emailing the occasional zipped shapefile :)


Read more ...

Context For Cheap - The Map Reference Overlay

Wednesday, January 23, 2013
I used to hate building reference layers in my maps. Labeling placenames was painful, but nothing compared to the chest-hair-waxing misery of scaling transportation symbology by road class. No out-of-the-box defaults were ever cartographically pleasing, and hours were incinerated in the fires of annotation placement. [Sorry, QGIS and Illustrator - you're just as much to blame here as the Redlands upstart.]

But that's basically all in the past. Not long after I slogged my way into web interactive maps, the crew at Mapbox released the finest preconfigured basemap I've ever seen, "Mapbox Streets". They quickly followed with a dozen attractive starter styles and an impressive customization pallette.

Mapbox Streets

The idea of the pre-baked map was nothing new - Google, ESRI, GeoIQ and others had entered the web map age with variations on the Reference-Canvas idea: "You provide the overlay data and the story to tell, we'll provide the geographic context." And this works really well for monodimensional Point of Interest (POI) data, as demonstrated by the Google pushpins that these days rain from the sky to skewer every interesting set of coordinates on the planet. Pins and icons don't get in the way of labels and reference features.

The siren call of the martini glass . . .

But what about polygons and the world of the choropleth? Not that it stopped anyone from trying, but polygons on top of a reference canvas either obscure the features beneath or require too much transparency to make a thematic point. I bombed out on early attempts to work with this essential truth:

NYC Metro, covered in bubble bath

The solution to this problem is under our noses. I first noticed this technique when John Keefe and Steven Melendez at the WNYC Data Desk posted a Mapbox-based interactive looking at NYC's proposed wards; the streets were curiously visible above the color-coded ward polygons.

They had introduced me to the reference overlay.

Leveraging the customization options of Mapbox streets, they had
  • Winnowed out the layers representing land and water from their basemap, leaving just roads, land use and text,
  • Set these to a modestly transparent level (maybe 30-40%),
  • Using the compositing of the Mapbox API, laid this semi-transparent layer on top of the thematic polygon layer, inverting the standard reference canvas model
After I recovered the pieces of my brain that had exploded out my ears (maybe I'm easily impressed), I set to applying this tactic to my own maps. I also realized that this could be expanded to allow for a sort of map sandwich, with land and water below, thematic data next and reference data on top:


And it's not just a Mapbox thing:
Hell, if you can do better than AJ Ashton and company, build your own mostly-transparent reference overlay and cache it in a tile server for future projects.

Subtle Context on the Census Dotmap

While I realize this is all old hat in the GIS world (yes, of course you place your labels above your polygons dude), it's usefulness in web mapping can't be overstated. The reference overlay saves us serious time, solves the "mashup" problem, and lets us focus on our data and what it has to say.

Update, Wednesday Night:
After conversations with some of the Stamen and former-GeoIQ folks, I figured it'd be worth comparing what happens when three different teams build reference overlays from the same (OpenStreetmap) data. Check it out here.







Read more ...

The Buffalo in the Room Part 2: Fade Out

Sunday, September 16, 2012
In the last post we looked at the many difficult paths that can take us to the summit of cartographic nirvana known as the "Buffalo Tint", as rocked by National Geographic Maps and others

As I noted, this effect has traditionally been impossible to pull off in a GIS platform like ArcMap or QGIS. Tilemill initially got us a little bit closer by giving us full control over styling possibilities with CartoCSS code.

But now, as of Tilemill 0.10.0, compositing functions make this kind of effect a snap. Let's look at making a full Buffalo Fade, still using South Sudan as an example. Specifically, we're going to make a fade mask in Tilemill that can be laid over some Mapbox base layers in a web map.


Step 1: Preprocessing a Mask

This step - preprocessing in a GIS platform - is optional, it just depends on where you want the fade to begin. The purpose of preprocessing is to create a fixed feature mask;

  • QGIS: Run a buffer on your focal feature, larger than the convex hull of the feature for good measure. Then run a difference process between the buffer and the focal feature. 



Either way you're aiming for a feature mask that looks like this:


Step 2: Into Tilemill


Then import that feature mask into Tilemill and style it with what might be the most efficient piece of code I've ever cobbled together, compositing the feature mask to fade inward from its border:



[Alternately, in this case you can do it by just compositing every country that isn't South Sudan and eliminate the buffer processing above. Here's the CartoCSS to do that]

That's it. Export to MBTiles format and drop it on top of a base map of your choice. You're off to the races:



There are still a few bugs when using this for dynamic tiles, notably some tile-edge artifacts that break up the smoothness. But overall I'm looking forward to messing around with these new compositing capabilities


Read more ...

Data-Driven Symbology in Carto

Tuesday, August 7, 2012

The term "disruption" makes me want to stick a hot poker in my eyes, and then post a tilt-shifted instagram pic of that poker laying on a table next to a lychee mojito.

However, "disruption" is an accurate description of the state of cartography over the past two years or so. The traditional realm of the GIS analyst has been invaded and turned upside-down by an army of new cartographers who come from a software development background. These visigoths have brought odd notions like efficiency, economy and - heaven help us - design to the mapping field, but wow do they have a different process for making maps.

This dichotomy between GIS-er and carto-developer is visible in the CartoCSS language. Stemming from a project called Cascadenik, CartoCSS is a collection of stylesheet code that allows a mapper to assign symbology to geodata and render it with Mapnik - by far the best image rendering engine available for cartography. This code distinctly resembles the assemblage of rule sets, blocks and properties that any web developer would use to style a web page (hint: the target user base):

 #wind {    marker-width:5;    marker-fill:#DADDE2;    marker-line-color:#fff;    marker-fill-opacity: 0.6;   }  

The highest-profile deployment of CartoCSS is in the excellent Tilemill mapping platform from the folks at Development Seed. The code drives every style choice you make in Tilemill, then it compiles everything to Mapnik and delivers you exceptional graphics, both static and web-ready. This is great news for developers, who are already comfortable with code in their workflows. Not so much for the GIS analysts (like me) who have been Arc-trained for years on how to push buttons and move sliders in the quest for a good map. Also, CartoCSS has had limited functionality for thematic mapping - a common complaint is that you can't do multivariate symbology (graduated colors and symbols, say) without awkwardly-nested code cascades.

Until now, that is. A key development has been taking place behind the scenes, one style parameter at a time -- data-driven symbology has arrived in CartoCSS as of Tilemill version 0.10.0. Let's take a look at what this means:

Figure 1 - Marker Size
Previously, you needed to specify a hard number as a value for almost every style parameter. Translated, that means the point data (representing weather stations) in figure 1A was coded like this:

 #wind { marker-width:10; }  

To make graduated symbol sizes, you previously had to put in a plague of conditional statements over many lines, each pointing to a different value for marker width. Now you can just pull those numbers from attribute data; in figure 1B, the points are graduated based on average wind speed, with the values coming from the attribute field labeled "mph":

 #wind { marker-width:[mph]; }  

You can even manipulate the attribute-derived values directly in CartoCSS with basic math. In figure 1B, I've multiplied everything by 2 for an exaggerated effect. Things get interesting when you try styling on multiple variables - one on symbol width (wind speed) and one on symbol height (station elevation) in figure 2B:

Figure 2 - Multivariate Symbology

Label placement has long been the lone data-driven feature in CartoCSS. This code renders the weather station labels in figure 3A:

 #wind { text-name:"[STATION NAME]"; }  

Now you can orient labels directly from a column, for instance to make them point in the average wind direction in figure 3B below (fun but cartographically criminal). Makers of typographic maps should pop a celebratory beer over this one; a little bit of geoprocessing and your work is automated now. [Update: I'm told the real purpose of this is to orient interesting text characters like Arrows (-->). It's all fair game.]

 #wind { text-orientation:[direction]; }  

Figure 3 - Label Rotation

Building heights for "Pseudo-3D" effects can also now be pulled from attribute data. Figure 4A shows building outlines, 4B shows all buildings rendered with a height of 8pts, and 4C shows them rendered at actual height in meters with a vertical exaggeration of 2. Woot.
Figure 4 - Dynamic Building Heights

The possibilities are not limitless here. It takes a fair amount of chopping away at the Mapnik source code to enable each of these data-driven parameters, so the list of available ones is still pretty short. Nor does this make CartoCSS the absolutely-ideal entry platform for a GIS analyst looking to get into "Map Development". However, this represents a forward leap in ease of use for mappers of all stripes, and I look forward to seeing what the community can do with this functionality.


Read more ...