Google Refine: A power tool for working with messy data

A story on Mashable tells us that "if you live for data, slave over spreadsheets and constantly find yourself sifting through endless rows and columns of facts and figures, Google’s got a lovely new product just for you — and it’s free and open-source, too." The product in question is Google Refine: a tool for cleaning up data, and more. I watched the introductory video on this new piece of kit; it seems to be developed to a very high standard.

However, I would not recommend Refine to graduate students conducting empirical research. As we have mentioned before on the blog, in academic research it is crucial to keep a record of any changes made to a data-set using the syntax-editors available in most software packages. Nonetheless, Google Refine could be extremely useful for undergraduate projects or for data-analysts who only use Excel. One appealing feature for security-conscious users is that the program must be downloaded, which means that the data is never on the web.

For more sophisticated data-handling, the first thing that comes to mind is Scott Long's recent book on "The Workflow of Data Analysis". This is an excellent starting-point for anyone looking to go about best practice in their research. Also, Daniel Hamermesh has a paper on replication in economics, that is arguably a must-read for graduate students beginning a program in empirical economics. An IZA WP version of the Hamermesh article is available here: Replication in Economics. The most recent discussion of data-issues on this blog (including workflow, publication bias, replication, retractions and empirical controversies) is available here.

The Data Deluge

Martin has discussed the increasing amount of publically available data several times (e.g. here). This week's Economist features a special report on the "Data Deluge".

Dropbox

Thanks to John Regan for pointing me towards the DropBox software. It comes with 2GB free and syncs your files online and across your computers. The website says it does the following:

- Syncs files of any size or type
- Shares large files and photos easily
- Automatically produces online backups
- Tracks and undoes changes to files