There is a petition to “Delay DA-RT Implementation.” I have signed it, as have (as of right now) hundreds of political scientists, many of whom are colleagues and friends.
A problem with petitions in general, though, is that people sign them for many reasons. I always struggle with petitions that have multiple demands or complaints, and this is no exception. I do think that the appropriate guidelines for data access and transparency deserve wider disciplinary discussion, though. And while I think the DA-RT statement is 100% clear that sensitive and/or protected research can be protected…
If cited data are restricted (e.g., classified, require confidentiality protections, were obtained under a non-disclosure agreement, or have inherent logistical constraints), authors must notify the editor at the time of submission. The editor shall have full discretion to follow their journal’s policy on restricted data, including declining to review the manuscript or granting an exemption with or without conditions. The editor shall inform the author of that decision prior to review.
…I do worry about the incentives that such a policy may have. Such as, for example, thoughtless position-taking on the methods wars, leading people to argue bizarre things like
unlike quantitative methods, qualitative methods put the analytical process right up front, in the written piece. Qualitative scholarship already has analytical transparency and, insofar as it is possible, reproducibility.
The idea that clarity or transparency is an inherent property of a methodology is completely unfounded, which is exactly why current debates about things like active citation exist in the first place.
I sign the position to encourage us all to get DA-RT right, to ensure that we have a hearing of the balance of costs and benefits. There is a view out there that the requirements of transparency would produce a neopositivist straightjacket, which I think is only true if you insist on it being so. I find myself, on balance, agreeing with the part of Jeffrey Isaac that welcomes deliberation and debate and is skeptical that there is a single model out there that is appropriate for all research endeavors.
I am not at all convinced, though, with the bigger claim in that piece by Isaacs, that being concerned about replication and transparency amounts to an aspiration to “put some more science into political science so we can science the science.” I see concern as more mundane: let’s make sure that when we publish things that rest on factual claims, we can check them, if at all possible. As for why Isaac and so many others hold this to be inconsistent with the
value of publicity and of vigorous intellectual engagement – within our scholarly community, among and between diverse scholarly communities, and between the academic world and the broader public world,
well, I am not so sure. But to be abundantly clear, I welcome that discussion, and I think we need it.
tompepinsky.com 
Jim Berry November 5, 2015
Dunno, AEA has had a similar policy for all of its journals for a number of years now, and I don’t think too many people are complaining. I certainly don’t agree that “the decision about whether or not to make research materials publicly available should belong to scholars and not journal editors or reviewers.” This implies that a refusal to be transparent on the part of scholars shouldn’t be considered in the review process. Yikes.
tompepinsky November 5, 2015
I think, though, that the AEA policy isn’t as impactful as you’d think. Being able to download a stata file is probably necessary but not sufficient. Let’s talk offline, I have an example in mind.
Nate Jensen November 5, 2015
I’ve always wondered why all journals don’t require a NSF style “data dissemination plan”. Author have to at least outline what data they will make available and justify what is shared and what isn’t shared.
Jarrod Hayes (@JarrodNHayes) November 6, 2015
Hi Tom, thanks for linking to the Duck piece–though some angst on my part at being labeled bizarre. At any rate, the point I was trying to make is that the analytical move being made with the data in scholarship relaying on qualitative methods is, as a general rule, more prominent in the scholarship than in quantitative scholarship, where it is either in the dataset or in the regression, neither of which are usually published in the article. On Twitter Jay Ulfelder made this point, that qualitative analysis is performing the same move that is done in creating the quantitative data set. If Ulfelder is right, and it seems reasonable, then my point about analytical transparency I think has some veracity even if I phrased it poorly (an accusation I accept and which I tried to hedge against at the end of the post).
tompepinsky November 6, 2015
Thanks for reading, Jarrod. I suppose that where we disagree is pretty fundamental: the “analytical move” is not “more prominent” in qualitative scholarship. Perhaps you can help me to understand what exactly an analytical move is.
If it is the data itself, well then no, that is sometimes included in quantitative scholarship, sometimes not, and the same is true as a general rule in qualitative scholarship (think comparative historical work, any serious ethnography…).
If it is a description of the logic of inquiry, well then no. That’s what the “data and methods” section is in your standard quantitative article.
Perhaps there is something else that you have in mind, and I welcome that conversation. But my fear is that you are working with caricatures of both quantitative and qualitative analysis, and I want to highlight exactly why I think that “move” is unproductive for those of us who insist that there is a wide variety of meaningful work out there, none of it perfect, and all of it contestable.
Jarrod Hayes (@JarrodNHayes) November 9, 2015
This is what I mean. If I want to make a point that something in an interview or an archive or whatever has some analytical significance, I put the ‘data point’ in the paper and explain why it has the significance I attribute to it in the context of the argument I am making. This is, for example, characteristic of my work but also of much of the qualitative work I read as a reviewer. That is the analytical move to which I refer. I may have caricatured qualitative and quantitative methods *a bit* in the piece, in part because it is a blog post but in part because it seems to me DA-RT is a lot like other initiatives to get qualitative methods to emulate quantitative methods and wanted to make the point that quantitative methods haven’t captured some fundamental scientific essence. I *completely* agree with you that there is a wide variety of meaningful work out there and that none of it is perfect. Indeed, that is my problem with DA-RT, that it has the potential to constrain or otherwise marginalize a great deal of valuable work. I was certainly not making the case that qualitative methods as a class are somehow above reproach. Indeed, I am a little bit wary of discussing ‘qualitative’ and ‘quantitative’ methods as categories because I think the variation within them may make such categorizations more obscuring than illuminating. And, finally, I have extensively reviewed scholarship drawing on quantitative methods (pretty fairly I like to think) and am participating in a project to build a great power identity database for quantitative analysis, so I am not some kind of methods warrior out to flame ‘the other side’.
tompepinsky November 9, 2015
Thank you, Jarrod, for explaining this more fully. This is clarifying, because it helps me to understand what the source of our disagreement is. It’s really a theoretical point, and one that I’ve thought a lot about.
When you write “I put the ‘data point’ in the paper” I disagree with this as a description of what you do. What I think you do is, you create a written summary of events, actions, and sequences that exist out there in the world but which are interpreted by you and worked into a model of that world, one which highlights certain features of those events, actions, and sequences and ignores others. That act of interpretation and model creation is fundamentally and inescapably theory-laden, and it could never be otherwise. As with any empirical exercise, “the facts” do not speak for themselves. That data point is not literally placed into the paper, even as a first approximation, any more than it is when you see a cross-tab.
I have no reason to doubt that you are not a methods warrior, and I’m glad to hear that you do not think that qualitative methods are above reproach. I think, however, that your post makes a clear statement about qualitative methods as a class having features that do not apply to them as a class. If you are “little bit wary of discussing ‘qualitative’ and ‘quantitative’ methods as categories because I think the variation within them may make such categorizations more obscuring than illuminating,” then great. I think that that post conveys exactly the opposite point.
Pingback: Put a DA-RT in it | Duck of Minerva
Stuart Buck November 7, 2015
Can you say more about why you would have signed the petition? It seems that you disagree with most, if not all, of what the petition’s defenders have said.
tompepinsky November 7, 2015
Thanks for reading. I signed the petition for a couple of reasons.
First, it raises what I consider to be important questions whose answers I do not know and deserve further debate.
Second, the comment about all of the communities potentially affected by DA-RT might have had a chance to debate them.
Third, the fact that I do not read this as a petition against the objective of transparency in political inquiry, but rather as a request to delay implementation of one particular version.
As my post tried to make clear, I do not endorse everything in the petition, nor do I agree 100% with everyone else who signed.
Pingback: Political Scientists Trying to Delay Research Transparency | Political Science Replication
George A. Gonzalez November 7, 2015
Qualitative research is inherently transparent, as independent verification of evidence is possible in virtually every qual paper — save for those that rely on anonymous sources (and on the whole they should never do so).
Pingback: Perspectives | Dialogue on DA-RT