Scientists Playing with FOIA, Bound to Get Burned: Stop Worrying, Embrace Transparency

Today's scientists are effectively public servants, and should start behaving accordingly.  The public has a right to know what scientists do and how they do it.  To prevent scandals like Climategate, scientific correspondence, critiques, and even data manipulation must be done in public.  The University of East Anglia and associated climate scientists didn't understand that. I'm not sure they do, even now that their emails were hacked and used to sow doubt about climate science.  Jon Stewart sums up the problem better than I can:

The Daily Show With Jon Stewart	Mon - Thurs 11p / 10c
Scientists Hide Global Warming Data
www.thedailyshow.com
Daily Show Full Episodes Political Humor Health Care Crisis

This isn't an excuse for climate change denialists.  (Please don't call those folks "skeptics".)  Climate change  is real, it's caused by man, and there are reasonable steps we can take to halt and reverse the damage.

The hackers who extracted emails from UEA committed a crime.  In so doing, they exposed that several scientists called people dismissive names, manipulated access to blogs, adjusted data in non-obvious ways, and derided their obligations under the law such as the U.S. and U.K. Freedom of Information Acts (FOIAs).


What bothered me most was the disregard for the law.  The most damning example from the emails I've found includes this passage, from Phil Jones to Michael Mann (emphasis mine):

Mike,

I presume congratulations are in order - so congrats etc !

Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don't leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I'll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? - our does ! The UK works on precedents, so the first request will test it.

We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it - thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who'll say we must adhere to it !

The passage goes on to other topics, but you get the picture.

The reaction to this crime should not be bigger and thicker firewalls, or at least not solely that.  Scientists should realize by now that privacy is not absolute, and in most cases, it's not even a good idea.  They should embrace transparency, by moving correspondence to more open formats like blogs, wikis, and Google Wave.  This move is critical to maintaining the long-term respect and public support for science.  Obviously these platforms need to be adjusted to accommodate the complex manipulations of large data sets; the scientific community is in the best position to demand for and contribute to the development of appropriate applications.  They will find great allies in the library science and open government communities, folks tackling the same essential problem.  One great example is the Alliance for Taxpayer Access.  In any case, scientists need to get out in front regarding transparency, or risk letting deniers define the route, themselves.

Some scientists may argue at this point, claiming that such presumptions of transparency will stifle free and open discussion and debate, that scientists working in the public sphere will censor themselves, and that the best ideas will not come forward.  They might worry that science would become even more politicized than it already is.  These might be fair points; after all, much of scientific inquiry is based on proposing and discarding ideas, ideas that seem crazy at first, but may just be right.

The answer is to embrace the embargo.  Embargoes are already used throughout the scientific enterprise.  Usually, they're used by scientific journals to delay access to non-paying readers, but they can be used in other ways, as well.  Scientist A collects data and wants to analyze it and publish.  While working on it, she shares her data with Scientist B so as not to delay further analysis.  Oftentimes, as a courtesy or by agreement, Scientist B holds his own results under an embargo until Scientist A has had a chance to publish her results.  This avoids confusion over which person should get credit for the initial data and analysis.

Wary scientists should know that everything they do, every email they write, every correction of data, every keystroke, could eventually wind up in the public domain.  The platforms that will serve those scientists best will incorporate a time-bound embargo, with definite and obvious, rolling expiration dates.

As Judith Curry of Georgia Tech puts it (emphasis mine):

[G]iven the growing policy relevance of climate data, increasingly higher standards must be applied to the transparency and availability of climate data and metadata. These standards should be clarified, applied and enforced by the relevant national funding agencies and professional societies that publish scientific journals... The need for public credibility and transparency has dramatically increased in recent years as the policy relevance of climate research has increased. The climate research enterprise has not yet adapted to this need, and our institutions need to strategize to respond to this need.

This actually isn't a loss of privacy–which doesn't really exist, anyway–but rather a move to make science even more legitimate and accessible to the public.  We should all recognize the great value the world has derived from access to earlier scientific correspondence.  Occasionally, the public needs to be reminded that scientific inquiry is a human enterprise.  Transparent scientists are ones they can believe in.

[P.S.: Judith Curry has done a fantastic job of corresponding with (initially hostile) commenters at the Climate Audit blog.  Some of her top comments are here, here, here, here, here, here, here, here, here, here, here, here, here and here.]


[Edit: Added emphasis.]