Pages

Showing posts with label spatial analysis. Show all posts
Showing posts with label spatial analysis. Show all posts

Crime Doesn't {{Insert Variable}}

Tuesday, September 17, 2013

tl;dr

There's no citywide relationship between crime rate and elevation in San Francisco. It's because there are other factors at work - like property value, income, tourism, etc. - in between the two. But there are a few discrete spots around the downtown area where it is at least partially accurate to say "Crime doesn't climb" - check out the last map on this page.


Crime Doesn't What?

Dammit, I love hexbins. I love their flow complexity, I love their visual appeal, I defend them where necessary, and I use them wherever I can. However, sometimes hexbins don't tell the whole story.

Last week a couple of talented Bay Area developers put some of San Francisco's new open data to use, testing the old adage that "Crime Doesn't Climb" in the city. They compiled SFPD crime statistics in elevation-based strata and found - indeed - that the lion's share of SanFranCrime occurs closest to sea level. Recognizing a bit of simplicity in this argument, they went a step further and adjusted the numbers to account for the fact that there is simply more space - more crime-canvas, if you will - at lower elevations. The results were the same: lots of crime low, not much up high.

The web reacted with characteristic nuance and reason:



While there's irony and sarcasm at work here, I think it would probably be a shame if even a handful of people now contented themselves with the certainty that criminals are lazy, or if some misguided readers resolved only to pass through SoMa in an armored car.

Because at it's heart, this analysis has already been critiqued by Randall Munroe:


Crime occurs where people - perpetrators and victims - are already concentrated. As such, any explanation of when/where/why has to account for population.

Population

The best level of detail available on population is the U.S. Census block. In a city as big as San Francisco, there are thousands of these, each with a very credible population count. They're not as spatially consistent as hexbins, but they're accurate:



Elevation

Elevation variation also contributes to the city's distinct character:




Crime

And the frequency of crime seems to reflect a bit of both population and elevation:




Read more ...

Navigation on Planks

Wednesday, February 6, 2013
We're not there yet . . . (Skier dude by Saman Bemel Benrud)
The news yesterday was that Google has added the trails of 38 ski resorts in the US and Canada to its Maps database. Here in Vermont, my stashes are safe from further encroachment, since the only two Green Mountain resorts on the list are perennial tourist-bait favorites Okemo and Stowe. That said, this is a cool move by Google in its desire to permeate our mobile world. I'm used to leaving my phone behind while I ski, but this sort of information would be useful to have at my fingertips if I were in unfamiliar terrain at a new resort. It could also be useful in tandem with the ski-run-tracking apps that are proliferating these days.

However, one item is missing from the new functionality: ski navigation. You can't request the shortest (or gnarliest) route from the top of the Sensation Quad to the bottom of the Alpine Double. These new lines - while color-coded according to difficulty - are just background images in Google's otherwise-rich network of information. The navigation engine suggests that the shortest route down the Liftline at Stowe is to pop off my cables and trudge down the Toll Road or the Long Trail.

The longest way around
I can see why Google would be nervous about offering directions in this context - There are two problems that are unique to alpine skiing as a navigation paradigm:

Obey the Rope.

Trail conditions (especially in the East) change more rapidly than road conditions. However, they don't change more rapidly than traffic conditions, and Google's got that covered in near-realtime in major metro areas. 
Solution: GTFS for ski resorts - While it's not immediately realistic to ask every resort to come up with a trails API, it's possible to work out a common format for trail reports (Y'know, those things you look at in the morning to see where it's not bulletproof) much like Google does for public transit. It would be an afternoon task for a data ninja at Mountain View to figure out a way of scraping and parsing that information on a daily basis.

Gravity's a . . . Precondition.


Leaving nordic and backcountry skiing aside, ski navigation would require consideration of elevation change. No uphill turns allowed. 
Solution: Spatial Analysis -  This is one of those bread-and-butter geoprocessing problems that GIS undergrads are given: calculate a flow accumulation surface, add turn attributes to the trail data accordingly. This is actually kind of a fun one.

Lots of folks are never going to look at a mobile device while skiing, maybe myself included. But for the ones who do, it'd be pretty danged cool to have navigation assistance built into the vertical territory that Google is now adding to its already-thorough map system. Just a few hitches to overcome, and I think they're up to it :)

"If only there were some way I could technify this experience . . ."

Read more ...

Secession & Racism: a Spatial Analysis

Thursday, November 15, 2012

Can a state's tendency toward racism determine it's willingness to secede from Barack Obama's America?

Maybe.

There's the short answer. The longer one follows, and please note the many qualifiers before you fire up the Troll-o-Matic 9000.

Post-Election Racism

It turns out that there was a lot of racist language being bandied about on twitter after the election. And a small group of brilliant individuals elected to post their hate speech with geolocation attached. The good folks at Floating Sheep were there to catch it all and they boiled it down to a normalized index of the prevalence of racist tweets about the election by state:


A very small number of people in Alabama and Mississippi don't come out of this analysis looking very good. But the point here is that among social media users, Alabamans and Mississippians are slightly more likely than others to vent about the election using hate speech.

Post-Election Support for Secession

At the same time, citizens of some (initially Southern) states initiated online petitions calling for peaceful separation from the United States, AKA secession. These petitions have quickly spread to every state in the union, but the number of signatures varies wildly. As of this writing, Texas has over 100,000 signatures on its petition, while Vermont has 869 [and some colorful arguments have been made about keeping these specific two in separate countries].

Obviously you can't compare these two numbers on the same plane. To get a picture of the real support for secession in each state, I normalized the number of signatures by the state population in 2011 per census estimates:


Balanced by 25 million people, Texas is actually not quite as hot for a second republic as the petition might suggest. The northern rockies on the the other hand are desperate to escape, with a much as ONE AND A HALF PERCENT of the states' populations showing up on a secession petition (yes, this is a tiny percentage). The only states with no stomach for independence are pretty close to the map that John Kerry won in 2004 (with the odd inclusion of Virginia). This is fascinating on its own, but let's dig deeper . . .


Why do you really want to secede? I mean, really why?

All well and good, I thought, but what sort of motivations drive someone to sign onto an idea that half a million Americans died over not so long ago? There are as many explanations as pundits to make them up: culture, climate, economics, and one that comes up every now and then: racism.

So much coded language was thrown around during the election cycle that "Welfare" translated as "Greedy Black People" and "States' Rights" translated to "White Power" to some who were inclined to hear it. In such an environment it's not inconceivable that some citizens would rather leave the U.S.A. than dwell another four years under a president whose legitimacy was so exhaustively questioned.

Relationships like that are extremely difficult to tease out. There are potentially hundreds of factors driving an individual's decision to sign a petition like this, and the president's ethnicity may be only the smallest one - if it's present in the calculation at all. Colinearity is the term the statisticians like to throw around: you think you've got a significant relationship between two variables, but it's really a parallel factor that you forgot to measure.

Therefore, a non-exhaustive list of caveats before I get to any results:
  • Fewer than 400 geolocated tweets included both racist language and a reference to the election. This is a tiny number compared to the overall election-related traffic, and Floating Sheep has made this clear in their analysis. The author of that post judiciously used the term "Thin nail to hang on" in reference to my use of his dataset. That said, there was also a significant amount of non-geolocated twitter traffic using hate speech in the days after the election, so these are representative of a larger tone if not of a geographic location.
  • Signatories to these secession petitions are not all residents of the state in question. About 3/4 of the Texas petition participants appear to be Texans, while only about 1/4 of Oklahoma signatories are locals. Without a good way to sort out locals-only, I just included the total number on each petition and in doing so introduced more error. 
  • The analysis method I use below doesn't eliminate colinearity. It just suggests where it might be strong and where it might be weak. It's possible that racists in Georgia really love pandas, and it's actually the broader panda-loving population there that wants to secede so that the federal government can't seize any Georgian pandas.
  • Similarly, this analysis helps with - but does not eliminate - the modifiable areal unit problem.
  • Alaska and Hawaii have secession petitions too, but their non-contiguity with the lower 50 skews the results.
  • This is only scratching the surface. If there's a grad student out there with funding and no project, I invite them to burn time improving the input data.
  • More as they occur to me . . .
With these qualifiers in mind, let's look at the straight-up linear relationship between racist tweeting and secession-petition-signing. The big question I'm asking here is "Can a state's tendency toward racism determine it's willingness to secede from Barack Obama's America?" And I'm not answering that question; I'm answering a highly-qualified version of it based on the data I have available. Plus it's kind of a nutty question.

And at the national level, the answer to the question is no:
This "relationship" at the national level is not significant at any useful confidence interval, and it sports only the tiniest of trends (P = 0.36, R-squared = 0.017).

Before I lose the rest of you, let me say this: The answer to the above question could still be "maybe".

There's a statistical tactic available called Geographically-weighted regression (GWR); it's best described as a diagnostic tool to figure out how relationships like this can vary over distance. Instead of measuring the relationship between racism and secession at the national scale, we can look at it regionally to tease out places where it might be valid:

This map shows where it's possible that the prevalence of racist post-election tweets can predict the local support for secession. 
Cutting through the fog of colinearity, we're left with four states that exhibit the signs of correlation between racism and secession: Kentucky, Tennessee, Georgia and Florida. Anecdotally this isn't nuts; the very real secessionist movements in Northern New England express an odd motivating mix of social liberalism and Rand-ian libertarianism. On the other hand the Southern states actually did secede once, and whether you give it a Marxist reading or not, the resulting war was about the intertwining of race and governance. The banner-bearers of southern secession before this petition fracas included not a few white supremacist groups.

Many on the political left are trigger-happy with the use of racism as an explanation for any opposition to the president. The above analysis shows that this may be the case in some places. However, it must be pointed out that it also shows there is no strong link between racism and secession (using proxy terms here) in many "red states", including Texas and Mississippi. Other factors not analyzed here are most likely driving the petitions in those states. This is not to say there's no racist twitter activity in Texas, just that it's not really correlated with support for secession.

Here's the qualified conclusion born of all this flying data:

In a few southern states it is possible that the drive to secede from the United States is being informed by racism on some level. 

This is surely news to no one, but I thought it would be useful to look at the statistical underpinnings of it. Make of this what you will.



    Read more ...