Uncategorized / Comments Off on Making maps with ACS data
Uncategorized / Comments Off on A user asks…: acs.R and the 2013 census data
An acs.R user asks:
Are there any plans for 2013 data to be incorporated into the acs package?
Great question. Here is a great answer:
At present, the package is actually able to fetch the 2013 5-year ACS data, with two important caveats:
- you must specify the table number or variable number directly – you can’t use keywords, since the current version of the package lacks the correct lookup tables for 2013; and
- the acs.fetch function will give you some warnings about how “As of the date of this version of the acs package Census API did not provides data for selected endyear” – but you can safely ignore that.
See below for a basic example. (This said, I do want to release an updated version soon that will include the lookup tables and avoid the warnings.)
> acs.fetch(geography=geo.make(state=25, county="*"), table.number="B01003", endyear=2013) ACS DATA: 2009 -- 2013 ; Estimates w/90% confidence intervals; for different intervals, see confint() B01003_001 Barnstable County, Massachusetts 215449 +/- 0 Berkshire County, Massachusetts 130545 +/- 0 Bristol County, Massachusetts 549870 +/- 0 Dukes County, Massachusetts 16739 +/- 0 Essex County, Massachusetts 750808 +/- 0 Franklin County, Massachusetts 71408 +/- 0 Hampden County, Massachusetts 465144 +/- 0 Hampshire County, Massachusetts 159267 +/- 0 Middlesex County, Massachusetts 1522533 +/- 0 Nantucket County, Massachusetts 10224 +/- 0 Norfolk County, Massachusetts 677296 +/- 0 Plymouth County, Massachusetts 497386 +/- 0 Suffolk County, Massachusetts 735701 +/- 0 Worcester County, Massachusetts 802688 +/- 0 Warning messages: 1: In acs.fetch(geography = geo.make(state = 25, county = "*"), table.number = "B01003", : As of the date of this version of the acs package Census API did not provides data for selected endyear 2: In acs.fetch(endyear = endyear, span = span, geography = geography[], : As of the date of this version of the acs package Census API did not provides data for selected endyear
Census, Code / Comments Off on Presenting acs.R at the ACS Data User Conference
On May 12, 2015, I’ll be presenting the acs.R package in a session of the American Community Survey Data User Group Conference in Hyattsville, MD. The paper, titled “Estimates with errors and errors with estimates: Using the R ‘acs’ package for analysis of American Community Survey data,” is available through the SSRN or my faculty publications webpage.
Better yet, the session will also include a presentation by Michael Laviolette, Dennis Holt, and Kristin K. Snow of the State of New Hampshire Department of Health and Human Services on “Using the R Language and ‘acs’ Package to Compile and Update a Social Vulnerability Index for New Hampshire.” It’s great to see how planners are using and extending this package in all sorts of exciting new settings and applications.
Uncategorized / Comments Off on acs.R Question: using FIPS codes as rownames
Q: An acs.R user asks:
Is it possible for an acs object to use FIPS codes for rownames?
A: Absolutely. Here’s how:
Start with some data:
> some.geo=geo.make(state=25, county=001, tract="*") > some.data=acs.fetch(geography=some.geo, table.number="B01003")
Check out the geography functions:
> head(geography(some.data)) >
The output of the final command should display the start of a dataframe with descriptive titles in the first column, but then fips codes for the state, county, and tract. When displaying acs objects, the first column of the object’s geography() dataframe is automatically used to name the rows. But this can be changed – see ?geography.
To use FIPS codes instead, we can extract the relevant columns from the object’s geography() and paste them together to recreate fully qualified FIPS codes. (The relevant columns are everything expect the first one, so “geography(some.data)[-1]” will do the trick.)
> my.fips.codes=apply(X=geography(some.data)[-1], MARGIN=1, FUN=paste, collapse="") >
Then we can re-assign the object’s geography() to include these codes as the first column:
> geography(some.data)=cbind(my.fips.codes, geography(some.data)) >
> head(some.data) ACS DATA: 2007 -- 2011 ; Estimates w/90% confidence intervals; for different intervals, see confint() B01003_001 251010100 2994 +/- 13 251010206 2858 +/- 256 251010208 1903 +/- 260 251010304 2395 +/- 269 251010306 2616 +/- 270 251010400 3056 +/- 296 >
and you should see FIPS codes as the rownames.
Important note: Given that I actually don’t work with FIPS codes all that often, there is a chance I’ve deviated slightly from the proper formatting here – you may need to paste in extra leading zeroes or something to make sure the pieces line up in that apply/paste command – but hopefully you get the idea. (For example, I think tract IDs are supposed to be six digits long, not three.)
Census, Code, Free Software, Open-Source / Comments Off on acs.R useRs: Share your success stories
Have you been using the acs.R package to download and analyze Census data in your work? Do you have a story you’d be willing to share, to help us promote the package and show off all the cool ways people are using open source tools to make sense of data and help inform communities, policy-makers, and researchers? If so, please let us know: email your news or project descriptions to firstname.lastname@example.org and we’ll post them here to inform and inspire our readers. Be sure to include good images, news coverage, quotations, or other materials to help tell the story — and feel free to include links, scripts, or examples as well.
PS: Don’t forget to subscribe to the acs.R mailing list to remain in touch with the growing acs.R user community.
As some of you have noticed, the new five-year Census ACS data has just come out, and is now available via the Census API. To make sure you are able to fetch the freshest possible data to play with in R, I’ve updated the acs.R package to version 1.2, which now includes full support for the 2008–2012 ACS data.
The latest version is now available on the CRAN repository. If you’ve already installed the package in the past, you can easily update with the update.packages() command; if you’ve never installed it, you can just as easily install it for the first time, by simply typing install.packages(“acs”). In either case, be sure to load the library after installing by typing library(acs), and install (or re-install) an API key with api.key.install() — see the documentation and the latest version of the acs user guide for more info.
To get the latest data, just continue to use the acs.fetch() function as usual, but specify endyear=2012. (By default, endyear is set to 2011 if no year is explicitly passed to acs.fetch, and I didn’t want to change this for fear of breaking existing user scripts. In the future, we might to rethink this, so that it selects the most recent endyear by default. Thoughts?)
(Note: If you’re not sure which version you are using, you can always type packageVersion(“acs”) to find out.)
A while back I posted a recipe (based on some great examples on the Revolution Analytics blog) showing how to use the acs package in R to create choropleth maps. Now, through the magic of open-source software development — and the hard work of developer Ari Lamstein and the generosity of his employers — this process has gotten even easier: I call your attention to Ari’s new chorolethr package for R.
Ari is a Senior Software Engineer at Trulia, where he works on data science and visualization, primarily related to real estate and housing markets. As part of the company’s “Innovation Week” he developed the choropleth package, moving well beyond the sample scripts to create a powerful suite of mapping functions. With a single command, a user can now generate maps at the state, county, or zip code level, from any of the data available via the ACS.
The package is not yet up on CRAN, but Ari promises that’s in the works; for now, you can learn more about it — including installation instructions using install_github() — on the Trulia Tech + Design blog. (I’m of course proud to note that the acs.R package lies at the foundation of these tools, doing the heavy-lifting of fetching and processing up-to-date data from the American Community Survey — but Ari’s work is already moving beyond these humble roots, allowing users to create choropleth maps of any data they can get their hands on….)
To learn more about the types of projects undertaken by Trulia staffers during Innovation week, see this short video. Congratulations — and thanks — to both Ari and Trulia for helping to drive innovation forward in R and other open source projects.
Just passing this along — hope to see you all in March.
LibrePlanet is an annual conference for free software enthusiasts. LibrePlanet brings together software developers, policy experts, activists and computer users to learn skills, share accomplishments and face challenges to software freedom. Newcomers are always welcome, and LibrePlanet 2014 will feature programming for all ages and experience levels.
This year, the theme of the conference is “Free Software, Free Society.” How can free software protect journalists, whistleblowers, activists, and regular computer users from government and corporate surveillance? How can free software, or free software values like copyleft, community development, and transparency, be used by people fighting to create free societies around the world? What challenges are standing between us and our goal of free software ubiquity?
The event will feature a keynote addresses by Eben Moglen, founder of the Software Freedom Law Center, and Karen Sandler, executive director of the GNOME Foundation. The conference will also feature workshops in using free programs, talks and discussion panels with free software luminaries, plentiful networking opportunities, and a pre-conference social gathering. And as always, FSF founder and president Richard Stallman will be on hand to present the annual Free Software Awards.
In the words of the organizers, “If you’re interested in the role of technology in struggles for justice, community, and freedom, then you will find a lot to be excited about at LibrePlanet.”
Others have written—and I’m sure will continue to write—with more enthusiasm and hyperbole about the ways that new web portals and mobile apps are changing the landscape of public participation and responsive city planning: it seems that we are constantly being showered (or perhaps barraged?) with fun new social media tools to engage citizens and activate urban sharing networks – for everything from reporting graffiti to mapping public murals (yes, the irony is noteworthy), from finding a parking space to avoiding being mugged, and so on. Whether or not these apps will ever wind up being the “game changers” we are often promised remains to be seen, but the level of excitement and activity they are generating is undeniable, especially after so many years of resignation and inattention to urban problems.
That said, despite the energy that has been thrown behind developing—and promoting—these new weapons in our urban information arsenal, one aspect of these tools has been noticeably overlooked: the potential they provide to facilitate regional collaboration between municipalities, an as-of-yet-unfulfilled dream of urban planners in the past century.
For starters, consider the unremarkable case of permits for “open burning” in Massachusetts. Regardless of whether one supports the concept of open-air burning of brush, clippings, forest debris, and agricultural waste (and there are many reasons to oppose the idea), the law in most states still allows this practice, with all sorts of regulations and permitting requirements. In Massachusetts—where, famously, “all politics is local”—it is not surprising to learn that while the State Department of Environmental Protection has established a broad policy framework for the issue, actual permits must be obtained from one on the state’s 351 different municipalities, typically from the local fire chief. (And, of course, 351 different municipalities means 351 different addresses, 351 different forms, 351 different hours of operation, and so on.)
If you’re an old-timer (and chances are, most open-air burners are), this probably doesn’t strike you as all that unusual — just head down to the fire station, grab a cup of coffee, chat with some of the other old-timers, and maybe pick up a permit while your there: it sounds rather civilized, in fact, and quite communal. That said, it is nonetheless a pretty inefficient system: why can’t we just do this from home, via some on-line interface?
Well, if you’re lucky enough to live in Berkshire County, you probably can. Residents in 12 of the county’s towns can visit the Berkshire County Online Application for Open Burning Permits to read the regs and apply in real-time for a permit. The site is simple—crude, even—without any bells or whistles, and the process is still a bit arcane (permits are only available between 8:30 AM and 1:00 PM; if you live in the Town of Dalton “you must first visit the fire station between 8 am and 2 pm and pay a $5 fee for the season,” and so on), but it gets the job done. And more importantly, this single little unassuming website represents a major step in regionalization, breaking down 12 little principalities of permitting power to deliver simpler, more consistent, and more efficient municipal services across the county.
Indeed, one of the real strength of apps and online portals is their potential for scalability: once one agency creates a tool to solve a common problem (such as issuing burn permits), there is little cost to sharing it with others. If it’s done well and widely-adopted, it can even help set the standard for entire urban information networks, which is what we are beginning to see on the other end of the state, with a tool called Commonwealth Connect. This mobile app (originally known as Citizens Connect) was developed by the coders at the City of Boston’s Office of New Urban Mechanics to help empower residents to “be the eyes and ears of the City,” reporting potholes, vandalism, missing street signs, graffiti, and the like. Recognizing that the virtues of this “participatory urbanism” do not stop at the city border, the state’s Community Innovation Challenge Grant Program provided $400,000 in funding to expand the program, which now seamlessly serves over 40 cities and towns in the region.
Stories such as these bring new hope to the vision of metropolitan regionalism. As always, the devil is likely to be in the details of implementation, but by starting small, scaling up, and working incrementally through the challenges of cooperation to improve the delivery of some of these basic services—and in the process, recognizing some cost savings and economies of scale—we are starting to see the inklings of a quiet revolution. And, in time, I expect that this sort of “regionalism from the ground-up” is likely to result in more lasting change than the top-down approaches of the past.
I’ve been asked to provide a very quick example of using the acs.R package to conduct a t-test of significance when comparing ACS data from two different geographical areas—so here goes: a quick example.
Let’s look at the number of school-age children in different towns on Martha’s Vineyard. There are seven towns on this island, and luckily (like all New England towns) they are each represented as a different “county subdivision;” in this case, they together make up Dukes County. So we can get some quick age demographics for all seven of them in two quick commands:
> towns=geo.make(state="MA", county="Dukes", county.subdivision="*") > towns.pop.sex=acs.fetch(geography=towns, table.number="B01001", col.name="pretty") > # one more step just to shorten geography -- just town names > geography(towns.pop.sex)[,1]=str_replace( geography(towns.pop.sex)[,1]," town.*","")
If you look at the column names using acs.colnames(towns.pop.sex), you will see that we are most interested in columns 4-6 (male children age 5-17) and 28-30 (female children, same ages). We also might need column 1 (the total population for the town), for the purpose of calculating percentages later on.