Not Seeing Our Brains: The Future of Legal Research

Come writers and critics
Who prophesize with your pen
And keep your eyes wide
The chance won’t come again
And don’t speak too soon
For the wheel’s still in spin
And there’s no tellin’ who that it’s namin’
For the loser now will be later to win
For the times they are a-changin’^[1]

Dylan was more right than he knew. He doubtless didn’t have the fall of the book and the rise of digital information in mind when he wrote his anthem of social change in 1963, but of all the changes that have happened since then, the decline of the Gutenberg age might be the most significant.

Dylan’s caution is appropriate for someone foolishly trying to summarize thoughts about the future of legal research in 900 words. But since I’m apparently in the mood to make unsupported and provocative assertions, here’s another one: most of our students don’t understand how to use an index. A simple-sounding proposition, perhaps, but one that might signal the most profound change in legal research habits since West introduced its Key Number System.

To state the obvious, Key Numbers are nothing more or less than indexing tools. Using them, researchers could search vast amounts of case law quickly and effectively rather than annotating case reports and trying to maintain their own index of the law. Key Numbers appeared at exactly the right time: a more efficient indexing system was crucial to the development of the law as a profession and the complex engine of social justice it was on its way to becoming. Without it, lawyers could not have responded to the new and complex demands placed on them by their clients and by the increase in volume of court decisions near the turn of the nineteenth century.

Self-indexing isn’t impossible now, though. Technology allows us to define our index terms in the form of a search; hit “enter,” and a search engine searches the vast body of text we’ve selected for our research and returns the results instantly. Natural language searching makes things even simpler. Now we need not even understand the arcane language of Boolean searching because an expertly constructed algorithm will disambiguate a sentence or string of words and generate a search that, often enough, seems to return exactly the results we’re looking for.

Of course, it’s that “seems” that’s the problem. When we execute a natural language search on any legal database—when we self-index the data—we have the inquiry we typed and we have the results it generated, but we don’t know how we got from one thing to the other and, crucially, we don’t know what information we’re missing. We’re even in a worse position than our self-indexing predecessors because they, at least, had read the decisions as they were indexing them. If we don’t have an alternative research strategy to check the results, we’re living in the world of existential doubt so disturbingly captured by the Rumsfeldian concept of unknown unknowns.

The alternative, for us old folk anyway, is the indexed search. Employing both strategies might seem like a belts-and-suspenders approach, but it can give the researcher some comfort to know that the results of creative self-indexing can be verified by old-fashioned pre-indexing.

But if we are moving to a world in which indexes themselves are incomprehensible, and even Boolean searching is viewed as an anachronism, lawyers might soon be completely dependent on algorithms developed by others, without their specific research needs in mind, to find the law that supports their arguments and forms their case theories. Either that, or research will be conducted by specialists who are, at best, tenuously connected to the construction and prosecution of legal theories.

Is this so bad? Is this perhaps not just another turn in the law’s long and winding road, and nothing we have to worry overly much about? Maybe, but I’m not so sure. The less nuanced our research skills become, the less subtle and supported our arguments will be, and the more specialized legal research becomes, the more bespoke research will be restricted to those who can afford it. And once those skills are gone, they’re gone. There’s an argument to be made that the Key Number System was one of the most powerfully democratizing forces this country has experienced because it allowed lawyers equal access to the law without regard to how much their clients could pay. In losing our understanding of indexes and what they do, we’re in danger of losing something profoundly important and—literally—irretrievable.

I started with Bob Dylan. Let me end with J. K. Rowling, a sage for a different generation. After an information-retrieval system has taken over her daughter Ginny’s mind and caused her to wreak havoc and mayhem, Molly Weasely, one of Rowling’s most beloved characters, admonishes her daughter that she should “[n]ever trust anything that can think for itself if you can’t see where it keeps its brain.”^[2] If there’s a better warning against the future and the dangers of allowing an algorithm to dictate the results of legal research, I don’t know it.