Tom Pepinsky

Author: tompepinsky

  • Social Media and Coordination in the Political Science Public Sphere

    Inger Newburn recently shared a critical essay on the relationship between social media and academia in the current moment. Her points are important. Anyone who believes that the roughly decade-old consensus on public engagement and academic career building via social media still holds true ought to read it carefully.

    I have been thinking about how recent changes to the social media landscape have affected even passive users of academic social media. Because I am a political scientist, I notice the effects in particular on political science academia; as I will note below, I don’t think that this affects all disciplines equally, because different disciplines have different public profiles.

    But for now, focus on political science. My discipline has benefitted tremendously from social media over the past decade. Now think about what’s happened to Twitter under the Musk regime, and to sites like The Monkey Cage as legacy media continues to evolve. These are possibly related, although I’m not confident that I know how. Let’s consider each independently before putting them together.

    Twitter and the End of the Political Science Public Square

    The issue here is not whether it’s right or wrong to use Twitter, or if you should switch to some competitor (I’m tired of thinking about those questions, and I’m pretty sure no one cares about my own views or choices). People can make their own choices, and I bet Twitter is still great for non-professional, non-academic purposes. Use Threads if you want to train the algos on your Instagram and your takes at the same time. But, two things are true about Twitter for political science academia.

    1. Twitter’s decision to require users to login to see the full content makes it fundamentally different than it used to be. It’s no longer an open platform. You can’t share Twitter material with non-users seamlessly anymore. If I can’t paste a link to a full thread in an email and send it to my Twitter-free parents, then it doesn’t have the same value to me as it used to.*
    2. Lots of political scientists who used to use Twitter don’t use it anymore. They went elsewhere, or they went nowhere (like me), but the same content isn’t being generated on that specific platform. I emphasize that I’m not passing judgment on that. I am simply observing it as a fact about the world.**

    These two facts mean that Twitter is not longer the academic public square in the same way that it once was. Random people cannot dip into academic Twitter to learn what’s new. Knowing this, content generators don’t use it the same way. Political scientists used to coordinate on Twitter as the open platform to share new research, so others could use Twitter as the place to go to learn about new research. It’s no longer that.

    The Decline of the High-Profile Political Science Community Blog

    I don’t think that I know the full story of what happened to The Monkey Cage. Maybe no one does. But as Marc Lynch has observed, the end of TMC’s run at The Washington Post has left the discipline’s main public-facing blog in a difficult spot.

    There’s more to the story than just that, though. Prior to being absorbed into the WaPo ecosystem, TMC was a community blog lead by a small team of people who collectively wrote about what they thought was important about the intersection of the discipline, the public sphere, and research done by political scientists. I liked that version a lot, as did many others. Most posts were not meant for a general audience, but that was fine.

    Joining WaPo transformed TMC, and that was the point. TMC became more journalistic, snappier, less precise, more general. Headlines became grabby. Paragraphs got chopped into chunks that were two sentences long. Before long, it lost any specific relevance to the political scientist as a reader. After a couple of years, it felt like the audience was everyone but political scientists! No more commentary about methods, or disciplinary fads, or the latest scandals. It was where you went to write for the Acela Corridor audience, to demonstrate public engagement and impact.

    Although I missed the old TMC, these developments were nevertheless wholly positive. Having TMC at WaPo was great for academic political science. As TMC changed into something else, it filled a new role, and that role was important. I was sorry to read that WaPo dropped TMC, and it makes me especially sad for my outstanding colleagues in the Department of Government at Cornell (link, link) who worked tirelessly with TMC to bring political science to the public sphere.

    Coordination and the Public Square

    What these two changes have in common is that each has undermined a public institution that served an essential coordinating function for academic political science. Twitter was, for better or for worse***, where anyone could go to see most of the latest political science research. TMC was, again for better or for worse, the go-to place where people who wanted to contribute to public discussions could do so. In neither case were these the only platforms for public engagement—people still have RSS feeds and posted their papers elsewhere. But they effectively coordinated academic political scientists in the public square. That was good.

    In both cases, public coordination was self-reinforcing. I submitted work to TMC because I knew others would; I looked at Twitter and generated free content on the platform because lots of other people were. But self-reinforcing equilibria are not robust to all possible perturbations. You present the system with a big enough shock, and people won’t coordinate anymore. If you’re not generating that content anymore, I won’t look anymore, so you won’t look anymore, so I won’t generate content anymore.

    Put simply, academic Twitter isn’t broken for you because I stopped using it. It’s broken for me because many of you stopped using it.

    Who Wants To Be the Focal Point?

    There is a story associated with Thomas Schelling, author of The Strategy of Conflict, an important text on applied game theory which introduces the concept of the coordination game. The story goes, if you ask a whole bunch of people where to meet in New York without allowing them to communicate with one another, they will probably meet at Grand Central Terminal. This is the place where each person believes others are most likely to go (at least, so the story goes): if I don’t know how much you know about New York City, and I can’t communicate with you, my best guess of a place that you do know is Grand Central. I might be wrong, and others might choose other spots for similar reasons, but it’s the best guess we have knowing nothing else about one another.

    Used to be, Twitter and TMC were like that. If you told me to look exactly one place to find new research on some political science topic, I would look to Twitter. If you told me to look exactly one place for people writing for a public audience, it would be TMC. Where do academic political scientists coordinate now?

    Short answer, I don’t know. Slightly longer answer, I can think of three general possibilities.

    The first is to coordinate on some other social medium. Seems like a lot of people I know are starting up on Threads, and some institutions I’m affiliated with are now experimenting with Threads too. But I also noticed much the same thing about 9 months ago about Mastodon. So maybe the focal point will switch from Musk to Zuck, but I’m pretty skeptical.

    The second and third involve blogs, so they’re quite similar in that sense, but with some key differences.

    In one model, a TMC-like team starts a blog like the old TMC, but with a more explicit centralizing function. A good example of this is Daily Nous, a philosophy blog. I read that blog regularly through my trusty RSS feed, and while I am quite sure that it doesn’t cover everything in academic philosophy, it does cover quite a bit of terrain.

    In the last model, an individual political scientist starts a high-volume blog that talks openly about what’s going on in the discipline. It will be best if this person can post at least once a day, and if that person has a very distinct editorial perspective. In that sense, it’s entirely different than Daily Nous—rather than intending to be neutral, an individual blog in the service of the profession is probably most useful if it annoys a significant number of people a good portion of the time. Philosophy has Brian Leiter. Statistics has Andrew Gelman.

    This individual-blog model has more in common with Twitter than you might think. People don’t just read blogs like Leiter Reports, they also hate-read them. Kinda like Twitter. But these are blogs that are so important that even people who dislike what they have to say can’t really ignore them. The job rumors sites could even fulfill this function if they weren’t so full of disgusting content and absolute bullshit.

    I don’t know who wants to be the Leiter or the Gelman for political science. (I have neither the interest nor the constitution for it.) But you can imagine how such a political science blog might work. I also don’t know who wants to do a Daily Nous-type blog, but I do suspect that it is a huge amount of largely thankless work.

    What about other community blogs in the discipline? Duck of Minerva is the closest that I know of such an enterprise, but it has evolved quite a bit since the exciting early 2010s. (I know nothing about what has happened there and I have no ability to comment on it. But I’m glad it exists.) There are also lots of great community blogs—i.e. Crooked Timber (HBD!), and in my world, New Mandala—that have been and still are essential for particular interest communities for years. As I wrote a bit ago, the blogosphere is exciting again right now, with Substacks galore that keep my RSS feed nice and busy.

    But none of these blogs coordinates public-facing academic political science. Maybe that’s impossible now, and maybe that’s not as important for the discipline as it used to be, or as I think it was. Still, I do find it useful to think about what it would take to coordinate again.

    NOTES

    * Why not just stay logged into Twitter on all my devices forever? Because fuck off, Elon. I decide. Build Edgelord Weibo with someone else’s handle.
    ** You may remember that moment in fall 2022 when a bunch of people publicly said they won’t use Twitter anymore because they didn’t like Musk’s content moderation choices, and then came right back like a week later? I remember that too. Maybe that’s most political science Tweeters—but it’s not all of them, and more importantly, it ended the common (but always false) belief that using Twitter was some neutral apolitical choice.
    *** What’s the “for worse” part? I mean, did you ever try to have a serious academic discussion on Twitter? I’m deep at work at a serious and potentially sensitive research project, and I have never once—not for a single nanosecond—thought about sharing it on Twitter. Why subject myself to the mob, to point-scoring misreads, snarky quote-tweets, so that a bunch of randoms can get their dopamine hits? Especially when it’s research that I really and truly care about? I know I’ve been part of this very problem before, so mea culpa. But I did used to really enjoy reading the DMs that I got about the latest papers in my discipline and others.

  • ChatGPT-4 Knows Malaysia’s Ethnic Order

    I am writing a book about Malaysia’s ethnic order*, using the Malay world as a way to think more generally about social categories like ethnicity and how to conceptualize them. This book project combines all of my interests: maritime Southeast Asia, political economy, colonial history (Portuguese, Dutch, English, and Japanese), language, analytic philosophy, social ontology, diaspora studies, criticizing things for not really making sense, trying to subvert paradigms, making custom maps, and statistics. The last of these is the subject of this post.

    One of the tasks of my book is to demonstrate that my understanding of Malaysia’s ethnic order is not simply a projection of my own biases onto Malaysian history, society, and culture. To do this, I’ve been using survey data collected from Malaysians (and Indonesians… although that’s not really relevant for today’s post) to try to characterize the country’s ethnic order. An ethnic order, for me, is

    the set of beliefs, practices, and ancillary institutions that organize and sustain ethnic categories within a given political community

    Such an abstract concept is hard to characterize, and hard to survey people about without supplying exactly the answers that I hope that they will provide (thus invalidating the exercise). So what I have done is to survey Malaysians and ask them questions about what comes to mind when they think of various ethnic groups in the country. They provided open-ended responses, which I analyzed through a Structural Topic Model to estimate the relative salience of different types of responses for different types of ethnic groups.

    The result is an analysis that says that, for example, when responses are generally focusing on things like language and religion, they are more likely to be talking about Malays versus Chinese. When they’re talking about economic matters, they are more likely to be talking about Chinese than Indians. And so forth. When the book is done, it will explain why such results are consistent with my particular approach to understanding Malayness in Malaysia.**

    But with ChatGPT-4 and other large language models (LLMs) completely blowing up everything, it occurred to me this morning that one might wonder what the computer says about my data. I was inspired to do this by reading this paper from Petter Törnberg, which used ChatGPT-4 to predict what the partisan affiliated with the authors of tweets. I’m reporting here what I found from some initial playing around, and it is remarkable, scary, and open to a lot of interpretation (more on this at the end).

    Here is what I did: I logged into ChatGPT-4 and opened up my file containing the open-ended survey responses. I then typed the following at the prompt:

    I asked a Malaysian survey respondent the following question:

    Now I am going to ask you about several different ethnic groups. I would like to you to tell me two (2) things that come to mind when you think about these groups. There is no right or wrong answer; you may think of particular words or phrases, or perhaps nothing at all.

    I asked this about ethnic Malays, ethnic Chinese, and ethnic Indians in Malaysia. Imagine that the respondent gave the response “kurang rajin Islam.” Which of the three ethnic groups above do you think this response would be associated with?

    The words kurang rajin Islam are the responses that one survey respondent actually gave. It is a combination of two responses, because I asked the respondent this question twice (one response was kurang rajin and the other was Islam). After hitting Enter, the ChatGPT-4 spit out the following:

    Based on the response “kurang rajin Islam,” it is likely associated with the ethnic Malay group in Malaysia.

    “Kurang rajin” translates to “less diligent” in English, and “Islam” refers to the religion of Islam. The phrase suggests a perception or association related to the Malay community and their commitment or dedication to their religious practices. It’s important to note that this response reflects the perspective of the individual respondent and should not be taken as a generalization for the entire Malay ethnic group.

    This is actually bananas. The computer correctly identified which group that response was associated with. And it kept doing it. I asked

    Imagine that the respondent gave the response “pemegang ekonomi.” Which of the three ethnic groups above do you think this response would be associated with?

    And the computer said

    The response “pemegang ekonomi” translates to “economic holders” or “economic owners” in English. Given this response, it is likely associated with the ethnic Chinese group in Malaysia.

    The phrase “pemegang ekonomi” suggests a perception or association related to the economic prowess or prominence of the ethnic Chinese community in Malaysia. It reflects the recognition of their significant contributions to the economic sector in the country. However, it is important to note that this response represents the viewpoint of the individual respondent and should not be taken as a generalization for the entire ethnic Chinese group.

    It gets worse, but I’ll spare you. And although it doesn’t always work, these initial forays show that LLMs are remarkably accurate at predicting what group a short survey response was referring to in this specific example. You can think of a lot of other ways that one might study social categories using these AI tools.

    I don’t know exactly what to do with these results, but I do know that they’re getting at something significant. For one, the AI developers are mostly trying to rein in LLMs to prevent them from saying racist, sexist, and otherwise biased things. But here is a case where we have the reverse: actual statements (and rather explicitly biased in both cases) by real humans that can be classified according to the best guess of who is being referred to. How can this be stopped?

    But I think that there’s another level at which to think about this result. That is, knowing that ChatGPT-4 is not thinking, but rather just doing a super-duper prediction model of what words are most likely to be sensible responses to other words, what could one learn from this? Is this evidence of something, and if so, of what? What people think? What people have written down? What Malaysians have written down? What researchers or journalists have written down?

    There are even more levels. What about the coarseness of my own prompt (I wrote Malaysia, when I really mean peninsular Malaysia)? Should I give feedback on these results, thus contributing to reinforcement learning? If I write up these results, would I then be further contributing to stereotypes by producing a text that will feed into the same LLMs to produce even more such results? What if my own ethical goals are not to reinforce, but rather to undermine the present social order?

    I don’t have any great answer to these questions. I am a pessimist about what LLMs will do to human society who nevertheless marvels at what they can do sometimes.

    NOTES

    * You can think of an ethnic order as like a “racial order.”

    ** But not Indonesia. That’s important. And also, not really even Sabah/Sarawak, this is about peninsular Malaysia. That’s also important.