Fear and loathing: AI intoxication and the hallucinations keeping lawyers employed
A new generation of lawyers, born after the year 2000, are now emerging from law school. They are the first generation, known as digital natives, to have grown up not only with the internet having always existed in their academic and professional lives, but for many, with the use of AI tools integrated into the way they approach problem solving.
While this is an exciting opportunity for innovation and efficiency in the legal workplace, Gen Z has expressed concern regarding the cognitive effects of frequent AI-usage.
The integration of artificial intelligence into the legal profession has had well-documented hiccups. Generative AI platforms have frequently hallucinated case law, creating citations and quotes that don’t exist. Some lawyers have relied on this imagined caselaw, filing briefs to the court without even checking.
So far, in the first four months of 2026, there are already several hundred identified cases where U.S. courts have sanctioned attorneys for making false statements of law generated by AI.
A recent high-profile example was seen in the letter from a prominent big-law firm in Manhattan (no names, please) to the Southern District of New York apologizing to the court for using fake citations provided by an unnamed AI program. The firm explained that their “review process did not identify the inaccurate citations generated by AI.”
While AI’s creation of false legal authority continues to make headlines in the law world, attorneys are increasingly emboldened to crank out self-assuring articles explaining why the legal profession is not one of those workforces that is replaceable by AI.
Are we over-confident because of AI’s current failures to accurately emulate our research and writing without considering what AI will become in the future? Some of us were shocked upon hearing that Westlaw’s AI research tool, CoCounsel, generated false quotations that led to the filing of misleading legal arguments before the Sixth Circuit Court of Appeals.
Many thought that such careless mistakes only happened with the ChatGPTs and Geminis of the world—the layman’s AI generative tools—not a bespoke piece of software tailored and designed for assisting lawyers with legal research.
Apparently, an attorney sanctioned by the Northern District of California for citing fake cases generated by Westlaw shared my belief. In his explanation to the federal court, he “acknowledged reading the Thompson Reuters disclaimer about the accuracy of the CoCounsel AI tool but emphasized that the sales representative stated there was ‘no way’ the program could produce hallucinations.”
Detectability considerations are now at the forefront of how we consider AI’s role in legal writing, but there is an element of pride as well: “Am I better than AI?” “Will an em-dash trigger readers to wonder if ChatGPT helped draft this article?” “What if Chat could have written this article better?”
While these questions become points of creative and professional pride, it is the ethical considerations that should still be most important for attorneys. Brazen reliance on AI-generated research, without verification, equates at best to a lack of care and at worst an outright ethical failure towards clients. As noted by Judge Anna Manasco of the U.S. District Court for the Northern District of Alabama, attorneys’ use of fake citations, “whether generated by artificial or human intelligence,” reflects an “extreme dereliction of professional responsibility.”
While the U.S. legal system may be safeguarded by an adversarial “neighborhood watch” of lawyers eager to catch one another’s mistakes, in our industry the true stakeholders are the clients. Clients deserve the strongest protection but are often the ones most affected by the consequences of attorneys’ mistakes and misconduct.
Which raises the question, what if AI legal tech gets better? If AI-generated legal arguments become more accurate and human-sounding (which is more or less the existential goal of “agentic” AI – being more human) then what are the prospects for junior lawyers whose firms might bill $1,400 for two hours of their work drafting a legal brief that AI could do just as well in 2-3 minutes?
Maybe with the use of AI, that junior lawyer could spend fifteen minutes drafting a well-crafted prompt, asking AI to generate a brief in a specific format, and then proofread and make some finishing edits in far less time than the traditional process, thus saving the client a lot of money. This would threaten law firms’ core revenue source, the billable hour, and detract from the junior lawyer’s training.
Consider a world where AI stopped making up case law and got it right 100% of the time. Then, not only could AI write like a human (or better) but it could develop sound legal arguments based on accurate and up to date cases. This is a huge portion of what lawyers do and much of what pays the bills of law offices. While protecting the client should come first, keeping the lights on doesn’t necessarily come second if you follow the put-your-own-oxygen-mask-on-first principle.
One thing that has remained consistent throughout the evolution of the legal profession is that people want attorneys they can trust. Clients, especially those in dire straits, want to rely on experienced attorneys with human minds that can process real-world issues. But junior and mid-level attorneys are the only ones who can develop into, and eventually make up, that future class of trusted legal advisors.
If junior attorneys are prevented from doing the hard work of rowing the oars by an entirely cost-efficient AI-model, they may lose the years of on-the-job training needed to become those experienced senior attorneys that can take the helm.
Clients of the future will be left with either a generation of unsharpened super-juniors whose legal instincts and reactivity have never fully matured, or sycophantic legal chatbots that lack human sensibilities and judgment.
The kind of support clients need can only be built over time, through consistent and arduous effort, human error, and the lessons learned along the way. That requires preservation of a culture of solid ethics and dogged service, without giving way to the replacement of junior lawyers by AI. New pricing models will almost certainly replace the billable hour for many legal tasks, but the profession hopefully will always need junior lawyers to learn discretion and judgment through experience.
David Weinstein is an associate at Lewis Baach Kaufmann Middlemiss, focused on cross-border litigation and arbitration. Adam Kaufmann is the executive partner at the firm, practicing in the areas of investigations, white-collar crime, sanctions, and complex commercial fraud.
Reprinted with permission from the May 18, 2026 edition of The New York Law Journal © 2026 ALM Global Properties, LLC. All rights reserved. Further duplication without permission is prohibited, contact 877-256-2472 or asset-and-logo-licensing@alm.com.