What We Still Don’t Know About What Persuades Judges – And Some Ways We Might Find Out

Over 25 years ago, in his foreword to the first volume of Legal Writing, Chris Rideout nailed it: legal writing as actually practiced by lawyers and judges needs to improve, “[b]ut more fundamental inquiry into legal writing . . . is needed as well.”^[1] The intervening decades have seen many laudable efforts on the latter front, as our collective scholarly discipline, then in its infancy, has matured. But one particular question that Rideout identified remains largely unaddressed by our discipline, although recent developments suggest a welcome increase in attention to the topic. Specifically, Rideout explained that our field did not know as much as we would like about how legal documents are “actually read.” His diagnosis was concise: “Much of the existing literature about legal writing . . . offer[s] fairly prescriptive advice about organization and style. Very little of this advice, however, is based on research into the ways in which legal documents are actually written or read. Rather, it largely depends upon time-honored, general maxims for writing, translated into the language of legal writing . . . .”^[2] As all LRW professors know, legal writing in practice is by its nature often unavoidably complex both substantively and stylistically, making it imperative for the discipline to try to unpack those complexities to suss out what makes legal prose effective. To Rideout, it was “distressing” that we do not know – we in fact “need to know” – such matters as “what a judge responds to stylistically in a brief, or a client in reading an opinion letter, a will, or a contract.”^[3]

Do we know more now than we did then? Yes. Do we know as much as we could, or should? No. This holds for many aspects of legal writing, but my jumping-off point in this short essay is the specific topic of what rhetorical techniques actually persuade judges. A gap persists between what we think we know and what we actually do know about whether and, to a slightly lesser extent, how and why rhetoric influences judicial decisions. Below, I set out some quick and necessarily incomplete thoughts on why this actually is a problem worth addressing. I follow by identifying some holes in our knowledge that several scholars have started to try to fill, and some obstacles we might face along the way.

The Problem(s)

As Rideout mentioned, much of what we teach our students about legal writing boils down to “time-honored, general maxims.”^[4] That observation, as applied to what students learn about persuasion, reflects the undeniable fact that many persuasive techniques stem in some way or another from the teaching of classical rhetoricians. Aristotle’s influential division of effective rhetoric into logos, ethos, and pathos is overtly acknowledged in many legal writing scholars’ works about advocacy,^[5] and implicitly underlies much if not most of the rest. Put simply, like the many generations before ours who have looked to the classical rhetoricians for guidance in how to construct powerful arguments, our field collectively believes that these approaches work.

I’m no exception. Classical and current formulations of how to appeal to an audience’s sometimes-conflicting senses of justice, fair play, and naked self-interest correspond generally to my observations and personal and professional experience about how people make decisions in legal and non-legal situations. The same holds for guidance framed in terms of reason and emotion, of syllogism and story and an advocate’s credibility. I teach my students accordingly.

I’ve become less and less certain, however, sometimes as a result of insightful student questions, about how solid the foundation is for at least some of the techniques I encourage students to use. It’s not that I think a particular technique or rhetorical approach is ineffective or, worse, counter-productive; if I did, I wouldn’t encourage students to use it. But I occasionally hesitate to justify my recommendations due to what seem to me to be ultimately unsatisfying rationales. For example, I might explain the reason(s) I or others believe a technique to be effective. Is this only a sophisticated “just so” story?

Alternatively, I might explain that lots of commentators and experienced attorneys say that judges prefer particular techniques. Sometimes I can invoke judges themselves who say that about something or another. But this isn’t really a convincing reason. I of course believe judges who say they prefer a particular technique. Preference is a poor proxy, however, for whether using or not using a technique actually plays any role in shaping a judge’s decision. Judges, like anyone else, are often unaware of subtle influences on why they decide as they do; speaking broadly, legal realism suggests that their supposedly rational basis for making a decision may not be the actual basis for doing so, and in fact may be flatly at odds with that “real” reason.^[6]

To the extent my reactions are representative of other legal writing professors, two questions come to mind that I believe are likely shared by others. First, do these persuasive techniques actually have the claimed positive effect? Second, if so, why do they have that effect?

Do Rhetorical Techniques Perform as Advertised?

Properly answering the first question is largely a matter of empirical study, and in that sense both tracks and goes beyond one of the underpinnings of classical rhetoric. Aristotle’s and other classical rhetorical texts, like similar texts today, combine aspects of (among other things) observation and theory. As to the former, Aristotle’s Rhetoric is on one level a lengthy description of the techniques that advocates used in classical-era law courts and political fora, accompanied by observations about whether and when these techniques seemed to prove effective.

Modern scholars naturally can’t hope to replicate classical-era rhetoricians’ contemporaneous observations about what seemed to be persuasive millennia ago. But we can and should devote more attention to trying to determine empirically what seems to work now. Sophisticated statistical methods give us tools Aristotle didn’t have. These methods allow us to overcome the deficiencies of so-called “anecdata” by rigorously assessing whether and to what extent particular techniques actually do influence judges’ decisions.

Several recent works by legal writing scholars exemplify the sort of analysis needed to start bringing some certainty to often-untested assumptions. The first, published in the current volume of this journal, examines whether a brief’s overall readability influences whether a party is likely to prevail on summary judgment.^[7] In their article, Shaun Spencer and Adam Feldman study a collection of briefs filed in state and federal courts, and conclude that “a statistically significant relationship [exists] between brief readability and the outcome of summary judgment motions.”^[8] They suggest potential explanations for their results,^[9] and identify additional research questions that their study poses.^[10]

In the second article, another survey of actual briefs, John Campbell set himself the task of measuring “whether there is a measurable relationship between writing style and winning.”^[11] Accordingly, he examined briefs from three appellate courts. His results, although not statistically significant, were consistent with the hypothesis that stylistic choices affect the chances of winning on appeal.^[12]

Finally, Ken Chestek designed an experiment to examine whether negativity bias – “the brain’s natural inclination to attend to and process negative stimuli” – affected judges’ perceptions of a hypothetical case.^[13] Chestek created eight separate preliminary statements for a summary judgment brief, expressing either positive or negative themes, plus a neutral control.^[14] After reading one of the preliminary statements and some other materials, judges were asked which party they were inclined to favor.^[15] Results were mixed: “In some situations, negative themes seem to be important in priming a reader to disfavor the opposing party; in other situations negative themes backfire.”^[16] Chestek draws out various implications of his results, while recognizing that the artificial nature of his experiment limits how solid those implications are.^[17]

These articles start a discussion that more of us in the discipline of legal writing should join.^[18] They illustrate different ways of approaching the empirical uncertainty (measuring the results in actual cases versus carefully designed experiments) and attempt to assess both broad-based and narrow rhetorical matters (readability, which draws in many different considerations, versus a specific emphasis on negative priming). They are properly cautious in the conclusions they draw, recognizing that their results might be explained for reasons other than those the authors propose, or believed to be likely before the study began. They leave open many further questions to be explored – not the least of which is replicating their results, an important though unglamorous part of any scientific inquiry. Future researchers may wish to design studies or experiments that attempt to drill down still further, measuring the impact of specific stylistic techniques.^[19] And, it bears emphasizing, none of these studies explain the causal connections that might be at work, or establish that such a relationship even exists. Finding a statistically significant correlation between a particular rhetorical measure and a positive (or negative) result doesn’t imply causation. That something appears to work doesn’t explain why.

Why Do Rhetorical Techniques Work?

Instead, causation raises a different issue, the second question posed above: why these persuasive techniques appear to work. Returning to Aristotle’s Rhetoric, some of his theoretical explanation along these lines rested on positions about what it means to be human – man as a “rational animal,” for example – and what those positions suggested about why and how particular persuasive techniques would be expected to work. To put it gently, those assumptions are not widely accepted today in the light of some 2000-years’ worth of scientific advances. Alternate explanations are needed.

LRW scholarship has made progress here, advancing the ball by beginning to explore the scientific underpinnings that help explain why various rhetorical techniques might have the effect that they do. A few examples: Kathryn Stanchi has explored a number of these topics in depth, such as her influential article that explores social science research on persuasion as applied to how legal advocates should present a court with negative information about their client or position.^[20] She and Linda Berger have recently published a textbook combining their interests in science and persuasion, setting themselves the ambitious goal of “unit[ing] persuasion science with rhetorical theory and the real-life practice of persuasion.”^[21] In doing so, they combine insights from both classical and contemporary rhetoricians with lessons from contemporary persuasion science, emphasizing cognitive and social psychology.^[22] Lucy Jewell has looked at classical rhetorical categories through the cognitive science lenses of categorization theory and information processing. From this investigation, she concludes that although classical categories don’t mirror how humans actually think, our belief that they do “closely approximates how we best respond when complex information is presented to us.”^[23]

These works are models for how legal writing scholars might approach trying to provide a scientific grounding for legal writing. As Lance Long as observed, however, there’s not as much of this sort of scholarship as our discipline might wish. Indeed, he and Catherine Cameron have written a helpful textbook compiling scientific support for various aspects of legal writing, ranging from substantive reasoning to organization to style and even citation.^[24] But as Long later explained, the book didn’t contain as many purely legal writing studies as the authors had hoped, for the simple reason that there weren’t many such studies to include.^[25]

A Related Gap: The Connection Between Persuasion and Prediction

Another reason for scholars to explore the sorts of topics addressed in this essay is that doing so might help us get a better handle on a deeper problem inherent in making predictions about anything related to legal decision-making. Work by Mark Osbeck delves into why accurately predicting the outcomes of legal disputes is so difficult.^[26] Our discipline emphasizes the importance of making accurate predictions – the first semester of many an LRW course is devoted to how lawyers communicate such predictions to supervisors and clients – but, as Osbeck explains, the impediments to doing so are surprisingly undertheorized in legal writing scholarship.^[27]

For example, studies conducted by academics in other fields reveal the persistent and systematic errors that professionals make when rendering predictions – errors that tend to put a thumb on the scales in favor of a client’s interests. Accountants overestimate the likelihood that financial measures will or will not pass muster, depending on whether they represent a hypothetical regulatory agency or reporting company.^[28] Doctors overestimate the time that terminal patients will likely survive.^[29] Similar studies about lawyers – by non-legal writing scholars – reach similar results: experienced attorneys overpredict the chances of a successful result in ways that mirror the position of their clients.^[30] Even law students get into the act: in moot court exercises, students randomly assigned to one side or the other tend to overpredict their hypothetical client’s odds of victory.^[31]

LRW professors recognize the biases that all lawyers (and all people) confront, of course; we acknowledge them in class discussions and instruct students how they might be able to reduce (although not wholly eliminate) the impact of such biases. Pedagogically, this serves our students well. From a scholarly perspective, though, one area that’s ripe for further exploration is trying to assess how the uncertainty in how judges make decisions intersects with the inevitable biases lawyers face in predicting those decisions, and what that intersection suggests for how we teach students to make predictions.

Conclusion

Many LRW professors, myself included, might be interested in empirical work, but lack formal training in conducting such research. As Spencer says, empirical research isn’t easy.^[32] Naturally enough, this might appear to be an insurmountable obstacle to engaging in empirical work, especially when considering the already extensive demands on a professor’s time due to the heavy grading and other student-facing obligations of our courses. This position is completely understandable. In response, I’ll second Spencer’s encouragement that “[g]iven the talent and energy among legal writing faculty, we are well positioned to study what lawyers write, and the lawyers who read and write it.”^[33] And, much can be gained from reaching out to academics in other specialties who do have expertise in empirical work, potentially leading to interdisciplinary work that could reach a broader audience.^[34] Long sums up the point well: “If we want legal writing as a discipline to be taken seriously, we must be able to show, through rigorous studies, that we engage in serious legal writing scholarship.”^[35] Doing so, whether in response to Chris Rideout’s unanswered questions of 25 years ago or Long’s more recent challenge, can only benefit our collective scholarly community.