A Primer for Getting Started With Open-Source Web Maps
Now that you know why I care about telling compelling stories with widely-distributed maps, let's look at a few of the many tools that are out there to help the process. I confess to narrow experience here; I use MapBox and CartoDB for the majority of my projects, and there are plenty of alternatives to those. But as a starting point I think that these open-source web map design platforms are perfect - they minimize the amount of code required, they use the best graphic rendering engine in the field, and they are extremely cheap (or free) to use, even in an enterprise or high-traffic environment. I'm avoiding ESRI's "ArcWhatever Online for Server" options because of a.) the high price tag, and b.) the actual user-facing sites are only as robust as the javascript or flash developer who builds them. My preferred options give you a lot more to work with out of the box, for free.
I am not going to walk you step-by-step through the process below - I'll point you to resources that will - but rather I mean to sketch the structure that can get your map applications up and running. If you're stumbling and need some help, drop me a line through the GeoSprocket contact page or on Twitter
Step 1: Data
Data in this context can mean a lot of different things. The tools we're using aren't picky, so this includes:- Shapefiles (but this will need to be in a compressed folder)
- Spreadsheets (CSV, XLS, DBF, you name it)
- KML or KMZ
- GPX or in many cases raw text from various GPS units
Desktop GIS isn't dead. Those who say so aren't paying attention. You still need some form of spatial data manipulator to analyze and prepare your source information, and if you can do that with pure GDAL hacking at the command line of a virtual machine, you have no need for my advice. Yes, "Desktop Platform" includes ArcMap, but if you want to make a clean break I recommend Quantum GIS for robust, open-source geoprocessing potential.
Step 2: Choose a Platform and Get a Hosting Account
Your data should live on the web somewhere - where, exactly?
- Option 1: MapBox - Most advantageous for its speed and complete cartographic design potential. However, once you've rendered your map into tiles (in step 3), there's not much you can do to change them on the fly. Use this option if you have a specific vision of symbology in mind to present to your users, if you have a lot of information to present interactively (i.e. text/charts/images that pop up when a feature is clicked), and also if the underlying data will be accurate for more than a few weeks. Sign up here for the free starter account.
- Option 2: CartoDB - This is a user-friendly, cloud-based adaptation of the popular PostGIS database architecture. It can run a bit slower than MapBox (since it's rendering on the go), and the cartography options are more limited. However, the provided SQL API means that you can sort, filter and process your data basically in the browser. Go with this option if your application will have a lot of user-generated queries (i.e. "how many points are within 50 miles of this one?"), or if you regularly update the data being mapped. Sign up here for the free starter account (notice the pattern of freeness).
Step 3: Mapping Your Data
- Option 1 is a two-step process; first you style your map using the open-source design platform TileMill (download and install it here), then you upload the resulting set of tiles to a hosted account (you signed up in step 2, but note that the hosting program is also open-source if you prefer to manage the hosting on your own machine). Much digital ink has been spilled on the capabilities of TileMill, and the tutorial documentation is excellent. The process can be characterized as:
- Option 2 is relatively simpler to implement. The documentation is also solid, but the mapping portion can be broken down to essentially 2 steps:
At this point using either option, you should now have maps that are ready to be launched.
Step 4: Serving Your Map to the World
Step 4a: Embedding - By far the easiest way to get a URL that you can distribute to your audience. Both Mapbox and Cartodb have fantastically-easy embed interfaces for plugging maps into your blog or website content management system, and in each case they essentially host a full-page map website for you as well. Examples:
Racial Breakdown of Census Blocks in Burlington, VT Using Mapbox
Participatory Farm Mapping in Vermont with CartoDB
Step 4b: The World of Pages, Javascript and Beyond - This creeps a bit further beyond the scope of what I hope to cover here, but this is the ultimate destination for most of my map applications. There is simply greater flexibility in a website or mobile application that you can manage yourself, though it requires some knowledge of HTML, CSS and Javascript.
I say "some" knowledge - I didn't know the first thing about code when I got into this world, but it was amazing how easy it was to adapt a little bit. The resources available in the open-source software community are spectacular; the help and guidance offered to me by experienced developers out of sheer goodwill has been uniformly superior to the high-priced support of proprietary vendors (Trimble and ESRI, I'm looking at you). I am no developer, but with community help I've been able to explore and utilize some of the most exciting tools available to cartographers in our time. And things are only getting better from here . . .
Here are a smattering of places to start for customizing a page of your own:
- Mapbox Templates - these require a bit of github knowledge, but otherwise they are as close to plug-and-play map sites as you'll find.
- CartoDB Examples - Plenty of code available for re-purposing and adapting. In many cases this just involves changing a line or two to point to your own data.
- Codecademy - Might as well learn how to do this stuff for real . . .
Addendum: Playing the Field
These are not the only platforms out there by any stretch. I previously mentioned alternatives, and they are impressive as well. Here are three of the best, along with my caveats:
Update 7/12: The third option on this list just got snapped up by ESRI. Bully for them buying a great resource, but I haven't heard a thing about integration with ArcGIS online. Which makes me fear for a very talented team and some very cool applications . . .
- Google Maps - the innovator in this field, Google still provides the best imagery base layers, an intuitive javascript API, lightning fast speed and more features than you will ever use. The downside is the $10,000-a-year price tag for applications that cross some very low use thresholds (e.g. a modest number of hits or a password-protected deployment).
- OpenGeo - this is an open-source (unlike Google's) database-to-client stack of tools that have been around almost as long as maps have been on the web. You can bend these components into just about any imaginable mapping purpose, and this has provided the backbone to some high-profile projects. Unfortunately, operating the stack yourself is code-intensive on both the DB and the client side, and the company's services are fairly expensive.
- GeoCommons - Possibly easiest to use of any of the options, this web platform has some basic geoprocessing capabilities and a dedication to social engagement. I use it only infrequently because the really useful "GeoIQ" features are socked behind a paywall, and the free version suffers from a few bugs.
P.S. this post has not been an exhaustive look at cartographic storytelling in 2012. Had it been that, I would have spent 90% of it talking about the importance of outreach and distribution. People don't visit your map unbidden, and as such you should spend the majority of your time getting the word out. It's just fortunate that web maps are easy to distribute in these connected times.
whats wrong with the solar site?
ReplyDeleteNothing's wrong with it; it's an amazing example of what can be accomplished with a huge budget and really advanced flash development.
ReplyDeleteoh, from the context I thought it was a bad example of the esri stack.
DeleteSorry for the confusion there. My point was more that ESRI trots out some amazing applications but fails to point out that it's not ESRI tools that make them so. Rather it's the client mojo of good web developers; Solar Boston could have been built with exactly the same front end but with CartoDB swapped in for the ArcServer parts. It'd be far cheaper and speedier
DeleteBill, wxGISCatalog: http://code.google.com/p/wxgis/ appears to be a promising OS ArcCatalog alternative. Screenshots: http://code.google.com/p/wxgis/wiki/ScreenShots
DeleteGreat post. I think the future is some combination of open source and proprietary. A good example of this is the mix of packages most of us use to work with LiDAR data. While I am on the topic of LiDAR, those you say desktop is dead haven't worked much with LiDAR lately! Data of that size just doesn't scale well just yet. I still think the best thing Esri ever developed was ArcCatalog. Any pointers to an open source geospatial file manager?
ReplyDeleteLiDAR strikes me as kind of a "big data" problem, and there are open-source tools that are very helpful with that at a certain level of abstraction (Awaiting "SwiftRiver for LiDAR"). At the moment you're right, though.
DeleteAnd ArcCatalog is great for sorting through shapefiles; I don't know of an alternative in OSland. But I also see a shift away from the format and toward single-file geodata (kml, jp2) and RDBMS structures like PostGIS, which inherently structure your data in a neat way. In both cases it's become easier to use a GMail approach to cataloging geodata: with a combination of folders, tags and searches. More for the future.
The LiDAR problem reminds me of a talk given by Young Hahn at FOSS4G-NA this year - about how to maximize processing to render a planet's worth of tiles: http://bit.ly/JU2zKG
DeleteIt's not ArcCatalog but QGIS has a nice File Browser plugin you can install.
DeleteBill, thanks again for making our wheels spin (sprockets do that). Excellent grist for the mapmill. Why do you think so many "GIS Professionals" continue to use the high-priced stuff vs the free stuff? Is there more reluctance to use stuff in OSland on the Desktop GIS side vs web ($1000 GIS desktop app is a "manageable" cost vs $10000 server software)? Cost of learning new software? Demand in the marketplace (eg: those with ArcGIS Desktop skills vs Quantum GIS)?
ReplyDeleteAs you've noted the biggest opportunities for cost savings (and arguably better solutions) are in the web sphere. That is where the real growth and demand is...and therefore innovation.
Thanks for the reply, Steve. I think ESRI left a huuuuge opening in the web space, and even now they're reacting too slowly to keep up (That webinar yesterday showed a 2-year lag in technology). In addition to that, the majority of the online driving has been done by software developers rather than GIS pros, and they're starting from a point of comfort with open-source tools (between apache and javascript, the web is 90% open-source). This is very slowly trickling back to the desktop as QGIS and JOSM etc. enter maturity. But ESRI has a lock on desktop still, thanks to a long history of outreach and education. Until the term "GIS" is no longer a synonym for "ArcGIS" for the vast majority of users, there won't be demand for much else in the marketplace. I think the actual drive for change will be the public, who increasingly don't recognize "GIS Maps" [ugh] as the type of maps with which they like to interact.
DeleteThere is a vmdk virtual machine and a EC2 Amazon AMI available with CartoDB pre-installed:
ReplyDeletehttp://blog.lolo.asia/cartography/2012/06/11/cartodb-ec2-ami-vmdk-vm-available/
so CartoDB can be tested locally or on the cloud.
Nice! Thanks for sharing! Got one for TileStream by any chance?
DeleteNo, I don't have any TileStream VM :( .
Delete