The Buffalo in the Room Part 2: Fade Out

In the last post we looked at the many difficult paths that can take us to the summit of cartographic nirvana known as the "Buffalo Tint", as rocked by National Geographic Maps and others

As I noted, this effect has traditionally been impossible to pull off in a GIS platform like ArcMap or QGIS. Tilemill initially got us a little bit closer by giving us full control over styling possibilities with CartoCSS code.

But now, as of Tilemill 0.10.0, compositing functions make this kind of effect a snap. Let's look at making a full Buffalo Fade, still using South Sudan as an example. Specifically, we're going to make a fade mask in Tilemill that can be laid over some Mapbox base layers in a web map.

---

Step 1: Preprocessing a Mask

This step - preprocessing in a GIS platform - is optional, it just depends on where you want the fade to begin. The purpose of preprocessing is to create a fixed feature mask;

• QGIS: Run a buffer on your focal feature, larger than the convex hull of the feature for good measure. Then run a difference process between the buffer and the focal feature. 

• ArcGIS: Run a buffer on the focal feature, with "Side Type" set to OUTSIDE-ONLY, again to a good distance out beyond your feature's bounding box.  

Either way you're aiming for a feature mask that looks like this:

---

Step 2: Into Tilemill

Then import that feature mask into Tilemill and style it with what might be the most efficient piece of code I've ever cobbled together, compositing the feature mask to fade inward from its border:

[Alternately, in this case you can do it by just compositing every country that isn't South Sudan and eliminate the buffer processing above. Here's the CartoCSS to do that]

That's it. Export to MBTiles format and drop it on top of a base map of your choice. You're off to the races:

There are still a few bugs when using this for dynamic tiles, notably some tile-edge artifacts that break up the smoothness. But overall I'm looking forward to messing around with these new compositing capabilities

Read more ...

The Buffalo in the Room Part 1: Fade In

Since the ancient days - well, since late 1993 or so - production cartographers have been been stuck in an awkward technical limbo between GIS and art. Two platforms were required to get a map from vector geoprocessing to publication-quality graphics: Mostly ESRI's ArcGIS for the former and mostly Adobe's Creative Suite for the latter. Sure the two tried to overlap each other as time went on ("Seven hours to export a 900dpi TIFF and I can still see the pixels? Thanks ArcMap!"), but the basic math was tough to overcome: with finite memory on a workstation, ArcGIS focuses its resources on geoprocessing at the expense of the graphic outputs, and vice-versa.

This was the way of it when I started mapping. And the classic example of "You can't do that in GIS" is the buffalo tint popularized and used to wicked effect by National Geographic Maps. Basically it's a targeted feature fade, meant to draw attention to a focal point or to one side of a divide. And pretty hot too.

And it's not really possible in ArcMap. Here, let's try doing the inverse of a fade, which is easier to envision. This is more of a halo, and it's theoretically possible to do this by adding line layer after line layer, each offset and transparent-ed a bit more than the last:

Buffalo Halo a la ArcMap

Not too shabby, I suppose. A fade out from a clear focal feature. Maybe a bit heavy-handed, but it gets the message across. Too bad it took 30 minutes of clicking into five successive sub-menus on each of ten layers to get it done. And since ArcMap isn't a graphic engine, there's no anti-aliasing, and pixels are visible in every feature. This is not a production-quality graphic.

Let's try that again with Tilemill. I know I know, it's not a GIS engine, but it's a lot closer to one than Adobe Illustrator is, try as they might. Tilemill has full support for operations like selecting and styling by attributes as well as basic geoprocessing if the data is tied to a source like PostGIS, Google's data API or CartoDB's SQL API. Also it's free and open-source (I love that such news is ancillary to my point here. Woot!). As I've mentioned before, Tilemill brings the efficiency of CSS code to the map styling process, and it pushes everything through the sophisticated Mapnik graphic driver to look damn pretty for web or print.

Code will save us, right? Here, check it out:

Buffalo Halo a la Tilemill

This is a more subtle effect, with no striping artifacts, and all the linework is anti-aliased for smoothness. Bonus points for also providing an interactive output where the halo scales dynamically.

So what kind of Carto CSS went into that? Oh, just more than three hundred lines of recursively offset style code. Oy. It's true that it's portable (feel free to plug the code into your own project), but it's not ideal. Definitely not for fast projects under a deadline.

This is where compositing comes in. Last month, the indefatigable Mapnik team added support for the graphical magic that underpins programs like illustrator. This is part of a long-running effort by cartographic designers at Stamen and Development Seed to get out from under the Iron Adobe boot. (or the supple GIMP moccasin, I suppose). With compositing, all sorts of things become a lot easier to do in Tilemill, for instance what we've been trying above is now about 30 lines of CartoCSS, and much richer:

Buffalo Halo with Mapnik Compositing Mojo in Tilemill

The possibilities are sort of mind-boggling, and I invite all the actual graphic designers of the world to figure them out (The composite parameter alone in CartoCSS has 35 options). In the meantime I'll continue to look for ways to enhance my mapping toolkit; the next post will focus on reversing the direction of this effect, like in the NatGeo example linked above.

Free-Range Buffalo Halo, Thriving in its Natural Environment.

Thanks to Dane Springmeyer for pointing out the time-saving parameters on this one.

Read more ...

The Official Takeover

High Seas by AJ Ashton. Pirate by Lego, clearly.

ESRI has made some interesting moves in the web mapping space in the past year. I don't blame them for being desperate to get a grappling hook up on a ship dominated by a combination of Google Maps and an open-source polyglot, but at this point the attitude is starting to border on dickish:

• Step 1: ESRI rumbles toward a unified way of getting licenseholders' data online, while pulling a group of early innovators - GeoIQ - onboard to help. Admirable.
• Step 2: ESRI adopts the term "Web Map" along with "Story Map" and a few other items that they clearly stole. No worries; a bit of rebranding and we're all one big-happy-web-mappy family, ESRI, Google, FOSS4G and your grandmother with her Bing API side project.
• Step 3: ESRI kicks everyone else off the boat they just boarded. Now only ESRI makes "official" web maps, and clients should be wary of cut-rate imitators. Specifically a "Web Service" is only an ESRI REST service, and a "Web Map" is "[Like an] .mxd file, but for the web." 

Okay, this is an oversimplification - particularly that step 3 doesn't come from ESRI directly, but from a well-intentioned partner. Well-intentioned because the author clearly wants the "Average GIS professional" to have access to the brave new world of the cartointerwebs and that path is not currently an easy one. But this attitude doesn't arrive in a vacuum; ESRI has cultivated it in the hope that no one will notice they didn't innovate the web mapping space. Web developers did and still do. Folks from tiny open-source shops all the way up to search giants.

Fair play to the big guy with the marketing budget, you say. But here's why I want the developers that built this ship to retain control of it: THEY'RE BETTER AT IT. The user experience is uniformly superior in non-ESRI web maps, the implementation costs are lower and the data is faster. Though this will change and ESRI will catch up if new hires like Sean Gorman have anything to say about it.

But for now I would love to see a bit of humility and willingness to listen on the part of the GIS giant, instead of taking the ship by storm and kicking off everyone who knows how to steer it.

Read more ...

Results of the Geo Toolkit Poll 2012

Many thanks to everyone who participated in this survey of the geospatial industry. I tried to get a platform-agnostic look at the tools that are most-frequently used in our community, and after 250 responses I think we have a useful glimpse of the scene.

My reason for running this survey: I'm tired of speculating. As a multi-platform user I don't know where I stand with my clients and competitors, let alone in this sprawling industry, and I've heard lots of similar curiosity from colleagues. I call this an independent survey in that no vendor funded it (nobody funded it, actually) and it isn't for marketing use. This is a community poll, nothing more. Onward . . .

---

Methods

This is worth a cursory glance, since the results are inevitably colored by the collection routine. I relied almost entirely on social media to get the word out, specifically:

• Numerous twitter solicitations to my ~500 followers, retweeted to a combined audience of over 11,000 utilizing hashtags for both the ESRI User Conference and OSCon - probably annoying the crap out of everyone in the process
• Google+ and Facebook posts
• Listserv posts to ESRI, OSGeo, OSM and Google user groups
• Posts on the three largest geospatial groups on LinkedIn

Given the warren-like distribution network, I do not know how many people saw this poll. Thus the sample size is 250 out of an unknown population, and no big-picture conclusions should be drawn. Also let it be known that the balance - even if measured properly - changes from month-to-month.

---

Results

Question 1: Which of the following geospatial technologies have you used on at least one project in the past year? [Note - I Included pre-purchase GeoCommons on its own out of morbid curiosity; I otherwise would have included it with FOSS4G Web Tools]

Some of the technologies that went into the "Other" column include FME, MicroStation, ENVI/IDL, GIS Cloud, AutoDesk, Maptitude, Idrisi, ERDAS, MapProxy, R-Spatial, Garmin Basecamp, Oracle XE, Ushahidi and Geocortex. Sorry to have ignored those, but it's a big ecosystem out there.

---

Question 2: Which of the above technologies did you use most frequently in the past year?

---

Platform Gregariousness: Do you cross over from your primary platform? e.g. ESRI is your main platform but you've also used Google Maps/Earth at least once in the past year. [A venn diagram would be cooler but the chart API was inscrutable]

---

Use by Business Sector:

---

Use by Country: (Click here for fullscreen glory - we're all cartographers here)

---

There's a lot to see in these distributions - an ESRI lean among U.S. respondents, a FOSS4G lean among Europeans. Also interesting to see how the sectors use these tools. See anything of note? Anything obviously-spurious? Do tell - I think there's a good discussion to be had here.

I'll say it again: this was not a scientific, controlled survey. It's a snapshot or an anectodal collection; take your pick. But it is nonetheless interesting to see what this group of mappers uses to get the job done. Thanks again to you all for pitching in, and maybe we'll try an expanded version next year.

If you're interested in the raw, messy results (stripped of unique identifiers of course), hit this link for an XLS download, and happy parsing!

Read more ...

GeoTools 2012 Poll - Round 2

Expanding the search around the Geo Community:

Read more ...

Results of the Geospatial Technology Users' Poll 2012

Update 7/23/12:  The poll is now reopened and live results are appearing at a new post. The figures and discussion below should be considered preliminary

Thanks to all those who hit yesterday's poll of technologies at work in the geospatial field. I've got some interesting results below.

First a note on experimental design: This crap is not scientific. First I tweeted, facebook-posted and Google+'d, so I got in contact with the core community of geogeeks with whom I regularly interact. Then I sent it out via the Vermont GIS listserv, the ESRI user conference hashtag and the O'Reilly open-source conference hashtag, hoping for balance. There is surely a geographic skew toward the U.S. Northeast, but I'm pleased with the general distribution of respondents. n = 117, which seems pretty good to me. Hit me on Twitter or on the GeoSprocket contact page if you'd like a copy of the raw survey results.

Here's a look at the participants using the generalized locations of reported companies/institutions (lots were left blank, so who knows):


The results of question 1:

Note: Some of the technologies that went into the "Other" column include FME, MicroStation, ENVI/IDL, GIS Cloud, AutoDesk, Maptitude, Idrisi, Mapserver and Geocortex. Sorry to have ignored those, but it's a big ecosystem out there.

And the results of question 2:
Ayup, ESRI Desktop is the big winner in this circle. But a surprising number of Google Maps folks there too. Also intriguing is the even split among the open-source toolset types, contrasting with the topheavy ESRI lean toward desktop.

Here is primary toolset use by overarching category:


Things get interesting when we parse out some conditional results:

• 40% of users whose primary tool is an ESRI product have also used an open-source geo platform in the past year.
• But a whopping 80% of users whose primary tool is open-source (desktop, web or DB) have also used an ESRI product in the past year.
• Same with Google - 80% of respondents who primarily use Google Maps have also used an ESRI product in the past year.
• That favor is largely returned - 75% of primary-ESRI users have used Google Maps.
• OpenStreetmap and GeoCommons had plenty of casual users, but very few used them/built them as their primary tool (1% each).

      There's a venn diagram to be had in there somewhere, but I'm not up to it.

Without leaping to conclusions, I would say that it's still an ESRI world. Even the folks whose day-to-day revolves around open-source or Google tools still fire up an Arc license every now and then. The converse is not equivalent; fewer than half of ArcJockeys use any of the open-source tools, though they are partial to Google Maps.

There are a lot of potential reasons for that, but it seems safe to say that open-source geo is still developers' territory, and Google mapmaking tools are more comfortable ground for ESRI's users. I recall that specific path when I was making my own way from ArcGIS to GDAL and Javascript.

There's a lot to read here; what are your thoughts? Anything surprising?

Read more ...