How Legal Tech is Leveling the Legal Playing Field, with Casetext Co-Founder Pablo Arredondo

Pablo Arredondo  0:03  
Yes, technology will probably make it so that attorneys without a drop in quality can reduce some of their supporting stuff.

Announcer  0:11 
Welcome to the California appellate podcast, a discussion of timely trial tips and the latest cases and news coming from the California Court of Appeal, and the California Supreme Court. And now your hosts, Tim Kowal and Jeff Lewis.

Jeff Lewis  0:25 
Welcome, everyone. I am Jeff Lewis.

Tim Kowal  0:27 
And I'm Tim Kowal California Department of podcasting license pending references check. The California appellate law podcast is a resource for trial and appellate attorneys. Well, Jeff, and I do some stuff in trial court and we do some stuff in appellate court and we try to bring some actionable insights to our listeners for at least one or or both of those venues. All right,

Jeff Lewis  0:47 
Welcome to episode 37 of the podcast. Today, I want to thank CaseText for being a podcast sponsor. CaseText is a legal research tool that harnesses AI and a lightning fast interface to help lawyers find case authority fast. I've been a subscriber since 2019. And I highly endorse case text as a legal research tool. And listeners of the California appellate law podcast can receive a 20 for 25% lifetime discount available to Matt to them if they sign up at casetext.com/calp. That's casetext.com/calp.

Tim Kowal  1:23 
So Jeff, when we had Ross Guberman on the show a couple episodes episodes ago, he told us that he was seeing a kind of Renaissance and legal tech right now. And case tax and its search tool parallel search are great examples of that parallel search, does what I had always expected the natural search and I put natural search in quotation marks the natural search function of the big search engines. But those natural searches never really delivered on their promise, at least from my vantage. It's about a parallel search and case stakes are about as fast as Google. And the results are always for me right on point. It's been it's been a very pleasant surprise using case texts and parallel search. And the way it accomplishes that is through the use of AI and Jeff, you and I had the opportunity recently to talk with the Co-Founder of case text Pablo era Dondo about how it works, and what other legal tech is right around the corner that our audience should be aware of. So we're pleased today to welcome to the show Pablo era Dondo Pablo is Co-Founder and Chief Innovation Officer at case text. Pablo is innovations including case Tech's pioneering brief analysis tool, Kara aI had been recognized by the World Economic Forum, the associate American Association of law libraries and the ABA. Pablo was also a code ex Fellow at the Stanford legal Stanford Center for legal informatics, focusing on civil litigation, and how litigators access and assemble the law. Pablo is a former patent litigator at Kirkland and Ellis and Quinn Emanuel and took his law degree from Stanford. So, Pablo, welcome to the podcast.

Pablo Arredondo  2:56 
Thank you. Thank you for having me.

Tim Kowal  2:58 
So as I mentioned, you are a practicing attorney before you went off and founded case text, what were the legal tech needs that you saw when you were a practicing attorney, that were not being addressed by by the legal tech industry at that time?

Pablo Arredondo  3:13  
You know, the short answer might be all of them. I was practicing, you know, at Kirkland, and we were representing fantastic technology companies like Apple Computer. And I remember I was a PC user up until that point, but here we are, you know, I had this Daydream that Steve Jobs would come check on the associates, which he never did, if anyone's wondering. And I thought, well, I don't want to offend the man. So I better buy a Mac. And this is right, when Steve Jobs had just come back from the wilderness. You know, he has big Renaissance. And so I was comparing sort of Steve Jobs at its prime technologies can traverse the Santa square, it's true to the tools that we were using to represent his company. And this, you know, these tools, whether in legal research in eDiscovery, and knowledge management, basically, if you threw a rock, you would hit something that didn't feel like it was put together by people would, you know, that loved what they did?

Tim Kowal  4:06 
Well, and and obviously, you went off and started case text and parallel search. And these compete pretty much directly with the big guys, Westlaw and LexisNexis. Why did you choose that, that corner of the market in particular, to innovate?

Pablo Arredondo  4:21 
Absolutely. So I mean, going all the way back to the days of Cara, the idea of using the litigation record, coupling the research engine directly to it, to have like, sort of much better results that are tailored to what you're actually working on was sort of one of the ideas I thought really, you know, that excited me and so I wanted to sort of anchor things there. And the truth, though, very soon, after pretty much shoulder to it, you start to realize just how unfortunate it is to have a calcified duopoly sitting on top of something so fundamental of the lawyers ability to navigate the law and find the precedent that they need and what it means to have these exorbitant services that then once are both exorbitant but also not feeling enormous pressure to innovate and to keep the ball moving. And I would hear horror stories from, you know, Paulo Mio, who was at the time, the director of the Stanford Law Library, and somebody, you know, very much considered like a mentor would tell me about attorneys who like would get 1015 minutes at the law library before their turn was up, you know, almost like, you know, toddlers at the water fountain. And so, you know, it was not just an issue of wanting to raise the bar. But quickly, it's like almost an issue of distributive justice, right? If one side has access to these tools, and the other one can afford it. It's not a fair fight. And so, you know, we love to think of justice with her scales and the blindfold on but you know, the tools you use to get what you put on each side of the scales really matters.

Tim Kowal  5:41 
Yeah, yeah. And before we dive further into case text, and its comparison with with Western Lexus, what are some other ways common ways that attorneys are under utilizing legal tech?

Pablo Arredondo  5:55 
Okay, well, so I think I mean, that's kind of a broad question, in some ways. I know, you know, states now, there's been a trend towards having a duty of technological competence, right, that that's now for the first time being sort of added. And I thought, okay, great. So surely that means transformer based neural nets, right. I mean, come on. And what I learned a little bit is that sometimes even simple things like, you know, PDFs and DocuSign, and things like that, things that are a little bit more pedestrian, still haven't really, you know, gotten everywhere that they should be. But so, you know, I hesitate to go into all the different aspects of it. But I think that it behooves attorneys, especially now that we're seeing a lot more gains, take a look around at their systems. And there's a great song by Jim Morrison, where he says, I've been down so long that it looks like up to me. And I think for some attorneys, the state of their technology has been all they've known. And so to them, it seems like no, this is what it's supposed to do. And it's really not. Yeah, well, Pablo,

Jeff Lewis  6:51 
when I went to law school, we shepardize cases with books, the shepherds, and you know, we didn't Westlaw was just coming around, but it was mostly books. And, you know, I like to think that open minded in terms of technology, tell me if I've got a 20 volume appellate record that I have in PDF form, why couldn't I just hit the F search button and put in a word search for something I'm looking for in the appellate record, as opposed to using one of the case text products?

Pablo Arredondo  7:18 
All right, there we go. So now you're at the heart of it in a way, what happened right after the books was the digitization of case law of legal materials. And that was a profound step forward. And by the way, for all the smack I can be going to talk about Alexis got hats off to the that was a very good thing to do in the 70s. And but what you left then was you were leaving the prison, if you will, of Westlaw is key topic system where Westlaw editor's got to decide how everything was organized. But you entered a different prison, you entered the prison of the keyword. And what that meant is, you're entirely at the mercy of the word that you happen to guess you know, to enter into Ctrl F, or to enter into a case law search bar. And what that meant was two important things. One, you were going to waste a lot of time because you were going to pull in things that you didn't actually care about. But that happened to contain the word that you chose. And that's frustrating. And that makes you a less efficient lawyer. But even more insidious, you were going to miss things that you did want, that didn't happen to use that exact word. And that makes you a less effective lawyer. And those twin things in the data science term precision and recall, are, you know, the fundamental problem with literal keyword search is that that's what happens.

Tim Kowal  8:32 
Okay, well, Pablo, back up a minute because, you know, keyword searching, what other way is there to search for, for something that you're looking for other than to hit one of those search terms? Isn't that how Google works? And I want to find something that's in a case when I have to search for the words that are in that case?

Pablo Arredondo  8:50 
Not anymore. And if there's one headline for your listeners, it's this is that that has been true, you were right for the last four or five decades, because what was actually being searched with something called the keyword index, which was like a literal spreadsheet where each word in the English vocabulary got a row. And each case or document got a column and you know, X if it's there, blank if it's not, and the profound breakthroughs that have been happening in an area of AI the subfield of AI called natural language processing, and specifically the applications of these things, called neural nets have allowed us to encode and capture language in a much richer way, in a way where actually you're no longer at the mercy of your keyword. And to be clear, 1990s Westlaw natural language search, which you mentioned, that's casual Friday in the keyword prison. That's all that was, right. So the answer is no, you no longer have to be at the mercy of your keyword. And I sure all lawyers will join me in rejoicing, because much of our life will be improved because of this.

Tim Kowal  9:47 
Okay, so there's a new way to search do attorneys need to be trained on on this new way to search everyone? All attorneys know now, whether it's the old way of searching where I want to see subject matter within Five words, if I search terms of jurisdiction to make sure that I get all the returns for subject matter jurisdiction, or whatever it may be, and maybe natural language strips out the the term that connectors, but you still have to have the terms in there. What is the new way of searching? And do attorneys need to be trained on the new way to search within this natural language processing system?

Pablo Arredondo  10:21 
Well, as it happens, the neural nets are now another important thing along with being able to now match language independent of articulation, you can now input directly the complete sentences that you want to include in your brief your memo your letter to opposing counsel, the technology is able to have a full sentence as an input, you can even leave in PARTY NAMES exactly as you have it in your brief. And it's able to look at that sentence as a whole, not just as individual words, but as a whole, and bring you relevant language. So the truth is, what lawyers know how to do well is think about what sentence do I want to write to the judge, and that's the training, you need to use these tools, you can just put that exact sentence in our system.

Tim Kowal  10:59 
And those those results are able to be as as relevant as the results I would get from the traditional search term based searches far, far better. How, why why why should it be better? If I'm hitting fewer of the search terms? How could it possibly be better?

Pablo Arredondo  11:17 
Well, because you're putting in a complete sentence, which really encapsulates exactly what you want. And what it's going to do is bring back things that match that sentence as a whole, and not things that might happen to have one of the words in them, but are completely unrelated to the sentence that you put. That's part one. And then the other reason it's better is that there will be things that use completely different words to get at the same concept. And those will now be surfaced, right. So there's essentially in every way that one might measure search, this is progress.

Tim Kowal  11:45 
Well, so does Does Google use this natural language processing? Yes. They are there currently.

Pablo Arredondo  11:53  
Yes, Google uses it for web search, I believe something like now 98% or something like that? Don't Don't quote me on that. Exactly. But have they always are they switched over it sometime? No, no, they switched over. And in their words, the sort of underlying technology Now to be clear case, tech stuff has been specially tailored for law. And there's a lot of things we've done. But this sort of underlying breakthrough, this ability to now bring these neural nets to bear on language, believe I'm quoting Google's blog correctly, one of the great leaps forward in the history of search

Tim Kowal  12:21 
is, what about Westlaw and Lexus? Have they moved over to natural language processing? Well, if

Pablo Arredondo  12:27 
they have, they've hidden it from everybody, because I haven't seen it in their their products at all. Now, please understand, you know, Westlaw, you know, boasts of their 25 years of using AI, and that's not inaccurate in the sense that they were using these earlier techniques. But it's, it's telling, when you look at their description of it, they talked about like, alright, you know, the most important thing is that content, you know, that key topic system that I referenced, you know, that's, that's what really gets the AI going. And the truth is, that's the whole point of the neural nets now is actually no, there's actually much more, there's better ways, in fact, to capture language that don't require an army of nameless editors that are, in fact, built on different approaches. And so it's only in the last couple of years that we've seen this, like, really tremendous jump forward.

Tim Kowal  13:10 
Why What was the motivation for that jump forward? I understand the natural language processing stands, it's kind of a different paradigm from search term based searches. Was this? What was the motivation for the big switch? Why do we wait so long?

Pablo Arredondo  13:26 
Okay, great question. So I mean, the wanting to tackle language, and then the wanting to bring computers to bear in language is once you know, as old as computers themselves, and was there from the very earliest days of artificial intelligence, which I think the term was coined at a conference at Dartmouth in the 50s, you know, and they were going to spend the summer and just figure everything out, right? Well, it didn't quite work out that way. So language has always been something where you wanted to apply it. But language is a really tricky thing, right? Language has a lot of weird aspects to it. And that were just not as amenable as something like chess, right? Where there's these like clear rules, clear winner and loser, right? Things like sarcasm, kicking, right? Things like metaphor, all of these different things that language does. And so that made it more difficult, especially using these earlier, expert approaches. Right? And maybe Can I take a step back? And just mention what I mean by that? Yeah, from the earliest days of AI. And by the way, there are many people much more qualified to speak on this than I and I refer you guys, you know, guys like Geoff Hinton and stuff like that. But let me let me do my best. Since the earliest days of AI, there were these two schools of thought. One of them said, the way you make something smart computer smart, is you get a bunch of experts together, people that really know the thing. And they think, how would I go about thinking about this, you know, what the rules are, you know, mnemonic, whatever I would do, and you kind of chisel that into the code, right? You basically and so for chess, it would be go bring all the grandmasters of the world together and ask them like how do you think about chess and you know, queens are worth more than rooks and better not to Castle etc, etc. And that Experts system was sort of one major school of artificial intelligence. And a lot of things were created under that sort of approach. But there was another approach that said, Wait a minute, what's actually intelligent on this planet? It's this weird thing that we have in our heads, these brains. And these brains don't have a flowchart. You know, in them, they don't have, you know, somebody who wrote a bunch of rules, they've got this rather amazing constellation of neurons that are all connected to each other, and that have, over time sort of been manicured by experience to be oriented in certain ways. And if that's what's really smart, and we want to build something smart, why don't we do what that does. And so this was the neural net approach. You don't want to overstate how much it borrowed from neurobiology. But loosely, it was saying, What if we could create sort of mathematical computer neurons and feed a lot of data?

Tim Kowal  15:55 
So you're creating a brain? Basically,

Pablo Arredondo  15:59 
you're you're you're adopting certain aspects of how a brain works, right?

Tim Kowal  16:05 
And why was this? Why was this approach resisted for so long? Well, I

Pablo Arredondo  16:08 
don't think it worked very well in the beginning. Because the amount of computational power you need to really test it is massive. And again, these theories, the underlying sort of concepts are very old. And so go back to like, you know, when the computers were the size of a dorm room and had the power of like a Casio watch, you know, and so I think that the neural net folks had a tough go of it. You know, I think there was a lot of times they were kind of, you know, why are you wasting your minds pursuing this path?

Tim Kowal  16:33 
You know, you're trying to you're like trying to build the space shuttle when it's still the Bronze Age.

Pablo Arredondo  16:37  
Right, right. But what changed is, the computers did get a lot better. And there was now a lot more also access to digital data to train with. And, you know, there were certain I think, breakthroughs in terms of some of the applications and things that would figure it out. And so it culminates in things like aI actually, I shouldn't say culminate, because it's very much just getting started. But some of the like moments that you'd see now, with this new wave of resurgence of these neural nets, I mentioned the chess systems, right, that the experts would come and teach. Well, there was a neural net that taught itself to play chess in four hours just by playing itself. And they pitted it against the system that represented, you know, for centuries of mankind learning to play chess, and decades of chess experts working with computer scientists to try to do it. And the I just played by sell for four hours when you know, held its own and won often in these games. That's sort of the I think that's a very colorful illustration of, you know, sort of the power of these neural nets, versus the expert system.

Tim Kowal  17:35 
Well, that that helps to conceptualize the difference in how you would approach running a search on a search term based system like West or Lexus, where you you, you assume that the that Westlaw and Lexus do not understand language, so you're telling them search for these terms so far apart from each other. Whereas when you run a search on parallel search, this is a system that understands language, so you can give it a sentence with terms and it will pick up the concepts and it will return relevant results, even if they don't match your search terms, because it picks up your meaning. And it's not strictly based on the terms. Do I have that right?

Pablo Arredondo  18:10 
I would say, yes, I want to be careful when you say it understands language, right? On a podcast, especially you can't see the air quotes. You know, computers are stupid. They don't understand anything in the way that like a human does, right. But what's happened is the way that love language is now captured and stored. The search engine essentially acts like it understands language, which is for an attorney working to make his deadline very, very close to having one that actually does understand it. So but I want to be careful not to overhype things, and I think later we might talk about like the, you know, robot lawyer kind of hype and all that stuff. And it's important to draw the distinction. But yes, basically, once you start storing language with this approach, your ability to find what you're looking for, feels much more human rights, because it's not tripped up on the exact word literal word that you chose. Yeah.

Jeff Lewis  18:58 
Hey, can I ask Pablo does case text, use the inputs from lawyers and the results that lawyers let's say, click on more often to refine and change the way case Tex behaves?

Pablo Arredondo  19:13 
So not in the direct way that I think you're meaning, right, where it's like a direct feedback loop? Obviously, we're monitoring, you know, success rates of certain activities, right copy with, you know, with, you know, an anonymous aggregated high level, you know, constantly evaluating things. But the power of this technology is actually something where it's actually trained on the back end, using a massive corpus. We actually trained it by putting the entire common law through one of these neural nets. And, you know, you make it play these little weird Mathematical Games. So it's actually not required the, you know, the attorneys are not the one that are fueling its ability to do what it does. And I think that's important in law, because so much of what a lawyer clicks on is context specific, right? It matters who your client is, what side you're on, right? What are the unique facts that may or may not be reflected in that query. So I think you can all He goes so far trying to like directly use the click feedback in terms of relevance.

Tim Kowal  20:06 
Well, we talked about a lot of a lot of what's under the hood, case texts and parallel search. But for our, for our listeners who are just interested possibly in what the product can do for them, Jeff and I have shared a little bit of our experiences that I have had, I've had such an easy time running searches, as compared with, with using the big guys in the past that, that when I'm on, when I'm on an oral argument with the, with the court, and an issue comes up, I feel comfortable just typing propositions into parallel search, and it will spit things out at me. And sometimes I wonder if it's if it's making it up, because it's so on point right away. And I haven't had the confidence to be able to do something like that with with the other search engines because they're just so much slower. And I usually don't find the relevant result until at least three or four cases in, and I've just had so much more success. Is that a with with case text? Is that a common testimonial that you're getting from your users?

Pablo Arredondo  20:59 
Yes, and I should point out, because I know we lose our skeptical bunch, we hear that a lot from people that we are not sponsoring as well, including members of the judiciary, you know, law firms, were so pleased at how well and widely this has been used case, Texas being used at some of the largest firms in the world, but also a lot of folks that just hang their hunger shingle, you know, boutique firms. It's designed to be, you know, really powerful technology, but also to be priced in a way that it's not precluded from, you know, everybody who wants to represent their clients to the best of their ability.

Tim Kowal  21:32 
All right, well, what are some other applications that the the engine under the hood of case text can be used for? And I know you've you've alluded to, to the proposition that the appellate attorneys, for example, might be able to run searches across their the appellate record to find propositions that support statements that they want to put in their briefs. impossibly the same thing for trial attorneys preparing for trial, and they want to find what evidence supports, you know, I've got, I've got tomes and tomes of deposition transcripts and documents produced in discovery, I need to find the evidence that supports the key proposition for my case, can can case text and the Kara AI engine, I think underneath it all, can that be leveraged for those purposes?

Pablo Arredondo  22:18  
Right. So you know, a few weeks after we put parallel search out for searching case law, we started hearing requests for, Hey, can I have parallel search over more things? Right, you know, case law is a very important thing to search through, but it's just one ocean of contents that a litigator finds themselves swimming through on any given day. What about expert reports, right, what about transcripts? What about ediscovery? And because we had to solve some pretty significant technical hurdles, in order to scale this, these techniques up to work for, you know, rather massive common law that grows every day, we were well positioned to if people sort of take it out for a spin. And so we tested it, we said, what happens if you take you know, a quarter million emails and put it in here and simulate sort of ediscovery. And what we found, perhaps not surprisingly, is that you know, emails are language to and a search engine that can act, you know, not based on literal keyword, but sort of I get what you're saying and bring back stuff was quite useful there. And we, you know, took a select number of our firms and did a beta project for them. And we were so pleased to see it start immediately playing a role, you know, on things like anti trustee discovery, right. And things, you know, when you hear lawyers say, you helped me find critical evidence much earlier in the litigation, right. Anyone who's practiced law knows, like, that's the muse. That's kind of what you do this for, right. And when you hear, you know, we found things we might have overlooked. But for the fact that the search ended do this, you know, this really validated for us that what we needed was to point these, this neural net technology and everything that a litigator needs to deal with. And so as of last night, we just released something called all search, which now all of your listeners can use. And essentially, what it says is tell, you know, give, upload the docs that you need to search through, you can just drag and drop a folder. And within minutes, you'll then have the power of neural net search directly on those documents, right? It's basically

Tim Kowal  24:12 
so you'll be the user would basically be able to run, run the same kinds of searches that they normally do using parallel search. But now it'll be against the database, have their own record their own trial record.

Pablo Arredondo  24:24 
Exactly. Their documents will now be stored in a much richer fabric that's been enabled by these breakthroughs. Yeah, let's

Jeff Lewis  24:31 
talk specifics. Pablo. If I if I had a murder trial of doing an appeal on and I upload to case text 10 volumes of trial transcripts, and the issue of death is discussed a lot and homicide and murder. If I put in the word die or death, what could I expect to come up in terms of search results?

Pablo Arredondo  24:48 
So that's just it to just the word dire death, right? When this you know, lawyers aren't poets, right? When are we just concerned with death as a concept, which you might be saying is what did anyone testify about? out, you know how they reacted to the death? Right? And what you'd find here and you know, I'm obviously pre selling this a little bit is stuff about, you know, subsequent to his being deceased, you know, What actions did you take or after the patient expired right? And you know, one of the real ways to feel the value of this is to put in a search like that, and just marvel at how many different ways in English you can say anything, right. But that's a great the exact kind of use case we're talking about, because you would hate to miss testimony, because it happened to say deceased or expired right passed away.

Tim Kowal  25:32 
So case tags and parallel search search will pick up that if you if you have a search term that says, you know, when, when the victim dies, liability obtains such and such, it will know that die also means kicks, the bucket gives up the ghost passes away, expires, whatever other words that we've come up with in our language for expressing that that ultimate concept.

Pablo Arredondo  25:56 
Yes. And by looking at the context of that sentence, it won't bring you any reference to like The die is cast right to singular die even though that word is there, right? The way we demo it with Enron is we search I feel uneasy about this something you know, maybe more people at Enron might have said, do anything about it. And you know, it's everything from this feels weird, I have to tell you, I'm conflicted. This is unsettling, you know, I personally to, you know, every variation of it, and we take it for granted as English speakers how easily we can compress all of these different articulations. But before these neural nets, it was absolutely impossible for a computer to do it.

Jeff Lewis  26:31 
Hello, without breaching confidentiality in terms of the firm's that tested this out for you? Do you have any great success stories about how attorneys have used this tech to either get a win or be more efficient and how they prepare cases?

Pablo Arredondo  26:44 
Yes, absolutely. So um, there's been a few different use cases that have come to mind. So one of the firm's has used it now on three major litigations in the eDiscovery context. And I've described exactly the kind of detail you'd love to hear how we found critical documents faster, and we found critical documents we might have overlooked to firms have actually started taking discovery requests, you know, like interrogatory ease, and saying no, wait a minute, that's a full sentence? I mean, it's a question, right? What if I just put that entire interrogatory as is directly in as a query, and that's surfacing what they need, right? So you can imagine the difference, right, the interrogatory itself can be answered, as it is, right, you don't have to go try to devise the query that best gets it. And that can also matter, because, you know, the interrogatory will say, you know, what, you know, list all your manufacturing plants, you don't want to miss the thing, talking about factories, right, or production facility, right, things like that. So not only can it take the interrogatory as a whole, but it returns all the different articulations that are responsive, right? We've seen folks using it, some firms actually created client facing databases, right to, you know, you know, put a bunch of collective bargaining agreements and stuff so their clients can more easily search what they need, again, not needing to, you know, vacation and time off all the different things that you can come up with, with contracts. And then, you know, folks have been using it to prep for summary judgment, I think that's a great use case, right? You put everything that you might need the whole litigation record into neural net. And then whenever you're trying to find something, you can do it. I should note that you don't forfeit literal, Boolean keyword to use this stuff. You know, there's a great New Yorker cartoon where the guy's in front of his microwave, and he says, No, I don't want to play chess, I just want you to reheat the lasagna. And the truth is, there are some times where like, you want to brute force keyword approach, you just need every reference to Frank Reynolds or whatever it is. Right. Right. And that's certainly available to you. Yeah. So those are just some of the use cases that we've seen. And to be honest, every week, we go and talk to our firms and there's a new one they want to try pellet record is another one.

Tim Kowal  28:46  
Well, there was a I want to I want to ask you a couple more questions about about the possible applications for parallel search in the legal profession. But there was you had reference to me earlier off offline a book by Cade Metz called genius makers, and it's about artificial intelligence and these neural networks and there was something in there, I just read the Cliff Notes version of it, but I want to ask you something because because when you refer to it as released, I got the impression that it's something like you're you're you're coding a brain you're you're designing something that that by which the computer can after fashion, think for, for itself or understand language, and I'm using the the air quotes for that. But here's, here's what Cade Metz said, at least in the cliffnotes version. So I don't know if this is an accurate quote, but it's the summation neural networks can learn tasks that human engineers cannot program by themselves. But the engineers have to be careful when choosing the data that the network will use to learn. Otherwise the network can learn unintended things, and quote, and I wonder is, is that true?

Pablo Arredondo  29:51 
Absolutely. And fundamentally, and what are now? unavoidably Yeah,

Tim Kowal  29:57 
what does that mean unintended things right.

Pablo Arredondo  29:59 
So let's talk about some of the weak sides of this stuff. Because I think I've extolled the positives right? First. And this is just to get to your question in a more to show you just how right you are right? neural nets are a black box. And I don't mean by that, like the way the recipe for Coke is a black box, meaning somebody has it in their safe and just would prefer not to tell you, right? With traditional search engines, the kinds of case tags Westlaw Bloomberg built, etc, I could take any query you gave, and tell you exactly why these results are where they are, right? We give this much weight to the citation count of the case. And the date is, you know, factored by this much weight with neural nets, I can do absolutely no such thing. I can tell you how we created it, you can tell me that it works really well. But why a given result is where it is is not something that we can discuss in any intelligent way. So that's part one.

Tim Kowal  30:53 
So in that way it is thinking, I mean, because because you're not telling it how to get from point A to point B, but it is getting to point B somehow.

Pablo Arredondo  31:02 
Right? Well, I mean, yeah, it becomes sort of it's, it's just that it's mathematical properties that can't be tied back to human things. Like I could say, well, this is clearly dimension for nineteen's vector pointing, it doesn't mean anything. And, you know, there, you might think, Oh, I can't use this thing. Right. That's if I don't know how it works? Well, I think, you know, a lot of the legal profession is using tools that they don't necessarily understand the full algorithm underneath it. And moreover, I would say like for most of human history, right, if you had like a rash, you know, the shaman would come and rub the leaf on your rash, and it went away. And you didn't know about the physiology or anything like that. It just worked. And, you know, here, obviously, attorneys review these cases, it's not like you're having the AI right. And so anyway, so that's part one about the blackbox aspect. But then to get to your direct question, absolutely, the way these neural nets are created is you basically push a huge amount of data through them. So for us parallel search, we put the entire Commonwealth through them. And there are biases encoded in the common law, right. And those biases will be reflected in how the neural net is sort of oriented, to the point where searching Pablo into the car might get different results than if you search Michael went to the car. Right. And I'll leave the listener to think about the different ways that that? And so this is a key area. And it's frankly, why I think having ai do things like you know, should we give parole, right, I think is a profound violation of due process, right? I think when it comes to legal research, where they're surfacing results for an attorney to view, right, I don't think it's quite, you know, in the same way. But But absolutely, one of the things with these neural nets is that they reflect the biases and what they're trained on.

Tim Kowal  32:45 
And that's very interesting it because as we know, technology never stands still, we're always pressing the envelope. So we'll see where things go. So I want to ask with with that, further bit of background about how it works, can can AI be used to predict how judges will rule maybe how certain judges will rule or certain courts will rule on certain kinds of issues?

Pablo Arredondo  33:09 
There's a lot of there's a whole field of legal analytics, that attempts to bring quantitative thinking in ways to predict things like that. My personal view is that a lot of it is basically phrenology right? Remember, they used to measure skulls to tell you if this guy was a criminal or not. And what I mean by that is, you know, you wrap yourself in the banner of quantitative measurements, like we are measuring things, surely. But when you actually dig a little bit deeper, those measurements are not in any way related to what ends up happening.

Tim Kowal  33:39 
I would assume that, that judges that people in general are predictable, and do things consistently. And it's like, remind you at dusk, Dostoevsky said that one thing you cannot say about people is that they are reasonable. The very word sticks in your throat, you know, you can shower him in bubbles of bliss, and then he'll do something just just just to show that he's a he's a person and not a piano key.

Pablo Arredondo  34:04 
Right? And yeah, exactly. I don't want to overstate it like I think Lex machina company that was acquired by Lexus, I think has shown some useful sort of statistical differences in how certain jurisdictions, you know, how long does it take to get to summary judgment, right, I would just advise them to be used forgive the pun judiciously. Right. And just understand that just because someone purports to measure, does it mean actually, that they're there, they're predicting with any with any seriousness. Again, there are other folks besides myself, who could probably come and go deep dive on that.

Tim Kowal  34:34 
What about on a related question, but maybe not going so far, when, when an attorney is preparing for oral argument, is there is there a future in which AI can predict what kinds of questions the judges will ask out oral argument?

Pablo Arredondo  34:50 
That's actually kind of brilliant. I haven't thought about that. So if we took the briefing, you know, we go gather 100,000 brief pairs, and then took the actual trend. scripts of the oral arguments right and fed that into a neural net. Can you then take two new random breed breed pairs and put it in and say, Oh, I bet the judges gonna ask that? I don't know the answer to that. Well, here's what I will say. It will make a guess. I don't know how accurate it is. Right. Right. All right, we're gonna need

Jeff Lewis  35:17 
to call that the Cole. oral argument, sir search if you if you end up proceeding that way.

Tim Kowal  35:24 
Okay, and then, you know, we we lawyers like to think of ourselves as irreparably, irreplaceable, you know, a machine can never do my job. But I don't know, is that am I right or wrong? It sounds like I'm right, for now, at least for for large portions of my job. But you know, how much? How much might be how much of our profession could be replaced? Because one of the things that you're describing sounds like it could, you know, could level the playing field. And I think, I think Jeff might have mentioned, you know, promote some access to justice, because where we're large cases would require armies of attorneys and law clerks and paralegals to, to rummage through all the files to find the answer to, you know, a simple question about, you know, what's the answer of this evidentiary question, parallel search and AI could could do that in an instant? And that technology is now available to everybody? So, so in what way? Do you see the shaping the legal profession?

Pablo Arredondo  36:21 
So you know, my first response was, of course, it's absolutely ridiculous to think that AI is going to replace a lawyer, you know, fully the idea. There's so many aspects to the practice of law that are so beyond I mean, it's not even close. It's essentially laughable. But what's interesting to me is what you the way you couch that was, do you need 10 lawyers, as long as you have one lawyer, to handle all the lawyer, you know, the strategy and that sort of aspects of it. I do think, yes, technology will probably make it so that attorneys without a drop in quality can reduce some of their supporting staff. That's just the reality. And I think that's not unique to law. Whether that's a ministerial paralegal or kind of junior associate, I think it kind of depends on how you allocate tasks among their right. But, you know, we do need lawyers who are able to do the parts that really can't be automated. And frankly, the way those get created is they their junior lawyers next to attorneys that know how to do it, and they learn how to do it. So I think it would be unfortunate, right, if we suddenly got rid of all the junior Associates, and again, I think also in talking about this, we're talking a little bit long term now where it's heading. The truth is the breakthroughs that happen that are leading to what I can show you now with real what you can go use right now. They broke through walls that are now leading to like increased acceleration, right. And so it's harder to sort of rule out this or that, you know, aspect of it. But you know, to me, my test is, show me an AI that can tell when Scalia is being sarcastic when Justice Scalia is being sarcastic, and then we'll talk about replacing attorney. Yeah, yeah.

Tim Kowal  37:52 
Case, tech says it has another another product called compose, which is very interesting. It's, it's been it's been marketed recently. And it allows attorneys to, to use to use the case Tex software to basically piece together a brief, can you tell us a little bit about how compose works?

Pablo Arredondo  38:12 
Absolutely. And you know, in case takes wood, our aim is to build the best thing for attorneys not to necessarily build AI into everything, right. So we really try to start with the user, you know, the attorneys experience and work our way there. So what composer does is, you know, we take huge amount of human effort, we map out, you know, if you're moving to dismiss under this service claim, right, or you're moving for sanctions, because of this, or that, we spend a huge amount of time to organize that, and sort of what are all of the major arguments, main arguments, common arguments you might make. And then we use technology to make it so that once you've selected what you want to make it wherever you are, as a court, it'll go, it'll bring you the support for that. And that's technology assisted but human at the end of the day is very human curated. And the reason for that is, of course, it needs to be right, because the way composer works is the idea that you can essentially plug and play right and put it in that way. And so it's sort of an attempt to some you know, I've heard mixed reviews on this sort of weaponized treatise right, you know, that sort of takes a treatise, but turns it into, like, you know, I realized there's a litigation going on. So, you know, I'm going to ask you which side you are on movement or non movements, and I'm going to behave accordingly. But another analogy that comes to my mind is if you think of legal research as being dropped in a jungle with a machete, composes sort of like a French patisserie, right, where it's just all laid out, and you go, Oh, I would like one of those little criminalized ones and things like so ya know. So it's a very powerful product. And, but because it's human, it's so human intensive. You know, California Attorneys listening to this will have a wide library of motions there. But, you know, for instance, if you're in Arizona, we don't have many motions, you know, that are set up for you directly. So compose is extremely powerful for the library covers. And we're expanding that library every month, but it's unlike parallel search. Are all search which is kind of universal out of the box composes expanding on a more targeted basis? Yeah, we put, let me put a little more detail on that answer in that I played around with those a bit in the context of anti slap motions. I've made or oppose probably 40 or 50, anti slap motions over the years, and I have my own personal library of arguments, what composed us makes that library obsolete? Really, because you're presented initially with a question, are you making or opposing the anti slap motion? And then it tailors a list that you simply do checkboxes as to what kind of prong one arguments do you want to make? Well, what

Jeff Lewis  40:36 
kind of prong two arguments are you gonna make? And then it spits it out into a word processing format, an excellent first draft of a brief, not anywhere close to a final draft, but boy, does it save you the headache of assembling that first draft,

Tim Kowal  40:50 
you don't have to stare at the blank page or you know, get a grant to go and get the legal, the legal standard, at least you get that started. It's very useful.

Pablo Arredondo  40:59 
And that's where we're going. I mean, if you think think of a lawyer in 1800s, right, he's got his pen, his paper, his bookshelf, and his file cabinet, you know, a couple other things, calendars, you know, stamps for envelopes, right? Our firm conviction of case Texas that the ideal system is going to unify these different key functionalities, legal research, you know, keeping your case files, drafting your brief read and put it into one system where they can talk to each other where they can inform each other. And, frankly, where things can be just a little better designed, because that's what technology lets you do. Yeah.

Jeff Lewis  41:29 
Yeah, that's a great point. Because everything we've talked about have been so far today, really separate symptoms and systems that don't talk to each other. Something we haven't talked about yet is, when I get a brief and appellate brief, especially one of the first things I do for my opponent, is I upload it to case text to see what kind of report that case Tex can generate regarding the cases cited in the quotations. And I'm always intrigued about the results. Why don't you talk a little bit with our listeners about that product

Pablo Arredondo  41:55 
and uploading a brief? Right, yeah, so our first sort of big commercial product case text was something called Kara. And what Kara did was let you drag your brief whether a Microsoft Word, Doc, your draft before you file, or opposing counsels brief, and it would recommend highly relevant cases that weren't in the brief. And it did that, honestly, this was before some of the breakthroughs I've talked about in AI. And so we did that, in some ways by having a much more robust, you know, it's good shepherd decision. Let's face it, that's the word. You know, I know Lexus owns it. That's the word, right? Shepherd, isation, and Keysight are very, you only look at direct sites, these are the cases that directly cite to this case. And the problem with that, imagine back when there were blockbuster videos, if you went to the blockbuster clerk and said, You know, I really liked the godfather. Can you recommend any movies? And the clerk said, Godfather Part Two, Godfather Part Three? That's all I got? Sorry. You know, great, you would find that one thing, right? You would say that's not exactly. You know, what about all these other great mafia movies? Right? Well, in the same way, what we did with Cara was we said, we're not just going to look at what cases cite directly to the cases in your brief. But what cases get cited alongside them, right? What are in the same string sites? What do we know when Judge is talking about a? What other cases do they talk about? Even if they don't cite to each other? It's how Pandora recommends signs to tell Amazon, you know, this, this sort of soft citation relationship. So that coupled with actually just paying attention to the 30 pages of text in the brief, was a pretty powerful query. Right. And so that put us on on the map that was sort of our first, you know, our first, you know, kind of prizes fanfare million dollar round, you know, the various things. And Westlaw and Lexus and Bloomberg have copied it to varying degrees of success. But what they missed, and I'm expanding a little bit on your question, but what their systems at least last I checked, they only take documents that have brief sites, and that have case sites in them. They're only looking at, it's like a brief analyzer. And they're missing something very important there, which is often attorneys are drafting their own motion to begin with, right there. They're doing research towards creating that first sort of draft. And so what our system and our system alone does is let you drag and drop a complaint, or, you know, a patent in suit, or a cease and desist letter. Not so that you can see what cases are missing because you don't have cases and a complaint, but so that the entire research experience can be tailored to what you're working on. If you search copyright and Westlaw or Lexus or traditional case, text, just search the word copyright, you literally get the phonebook, you get a case Feist about copywriting the phone book, right? With our system, you can drag and drop your complaint. And if it's about a musician being sued in the Southern District of New York for infringement based on some phrase, you can then search copyright the same simple word. But your results are going to be completely different. There are going to be of course, musicians being sued in the Southern District of New York for copyright infringement. So you know that that ability to have a research engine pay attention to the litigation, record complaints and briefs. We're very proud to have pioneered that. And that's something that you can you can go do right now and kiss text and knock his text alone.

Tim Kowal  45:02 
All right, well, Pablo, we've covered case text and parallel search. We've covered how AI is different from from search term based searching and why it's superior in your every day searching that that lawyers do just to get an answer to a simple legal question. We talked about how it's going to revolutionize possibly the way attorneys prepare for for large scale trials and appeals. It may obviate the need for large armies of attorneys to find the needle in the haystack when AI might be able to just do it with with one or or at least fewer than an army of attorneys. And we've talked about case Tech's compose product that can help attorneys put together the first draft of almost any kind of brief so that they avoid the panic of staring at the blank page. And is there anything else that our listeners should be aware of that may be up and coming? And how AI might be able to to improve their their practice and their business?

Pablo Arredondo  46:00  
Well, yeah, certainly we have some stuff in the pipeline that we can talk about that gets into, you know, the fancy sembly question answering that spans many things. Right. So you're looking at deposition testimony, and you know, RFA is request for admission and kind of synthesizing them. But rather than just talking about our pipeline, I didn't know for you guys, obviously kiss Tex is dear to my heart. But you mentioned at the beginning of this called brief catch right to him from Ross Gruber. Yeah, think brief catches example of a one of many tools now that are out there. The truth is, venture capital has finally started paying attention to legal tech, you know, did it about, you know, sort of getting frothy 10 years ago. And there's a number of new abilities from a lot of great companies out there that are worth checking out. And so I guess if I would leave you guys with something, you know, listeners with something it's like, just because it's what you've known for 20 years, and there's a comfort in that right. But be open to new possibilities, new functionalities, because you might find that after using them for just a few minutes, you can't believe how long you spent doing it the other way. And so I think it's a very exciting time for the technology that underpins the practice of law.

Tim Kowal  47:05 
Yeah, that was the that was the sentiment that Roscoe ruined express that we're in kind of, we're in an exciting time in legal tech right now. And, and your innovations with case techs and parallel search really seems to bear that out. So we're excited to look look forward to see what what you and others have have for US attorneys in the future. Thank you guys. Alright, Pablo, thanks for joining us, Jeff. Do we have anything else to cover?

Jeff Lewis  47:30 
No, I think that wraps up this episode. Again, we want to thank case tax for sponsoring the podcast and each each week we include links to the cases we discuss using cakes, text and listening to the podcast can find a 25% discount available to them. If they sign up at case tech.com/help That's the case tech stocks.com/ca LP.

Tim Kowal  47:49 
All right, we want to thank our listeners if you have suggestions for future episodes, please contact us at info at cow podcast.com and tune in next time for more actionable insights in the trial court and appeal court.

Announcer  48:02  
You have just listened to the California appellate podcast, a discussion of timely trial tips and the latest cases and news coming from the California Court of Appeal and the California Supreme Court. For more information about the cases discussed in today's episode, our hosts and other episodes, visit the California appellate law podcast website at ca do podcast.com That's c a l podcast.com. Thanks to Jonathan Cara for our intro music. Thank you for listening and please join us again.