ChatGPT Invented Fake Police Testimony in Murder Appeal

A defense attorney in New Mexico has been disciplined for relying on OpenAI's ChatGPT in preparing a murder appeal – an appeal that included fictitious police testimony and wholly fabricated witnesses, the state's high court ruled.

The New Mexico Supreme Court has fined attorney Stephen Aarons $5,000 and held him in contempt of court for not verifying the accuracy of information put in his client's appeal. "The court's brief included false testimony from wholly fabricated witnesses" and the justices lambasted Aarons for showing a lack of remorselessness and concern for his client. The high court also sent the case to the state's lawyer disciplinary board for an investigation.

Aarons told Reuters he used ChatGPT to summarize trial proceedings after agreeing to take the murder appeal of Oscar Renee Sandoval, who was convicted of killing the mother of his children and sent to prison for life. He said that he did not know the full extent of the AI's capacity to "hallucinate."

Previously, Aarons had been told by the court to explain why a fabricated claim appeared in his primary brief in Sandoval's appeal, the court filing said. Among those fictional details in the court filing apparently was this: "The shooter was wearing a pair of white trousers and a black shirt." Aarons claimed he had been uploading to ChatGPT his computer-Generated transcript and other documents relating to the case, and thought the system would provide a trustworthy overview.

This is a more serious problem because "the AI-produced material was not merely inaccurate a legal citation or fact," according to the brief. It says that "it consisted of factually inaccurate and fictional testimony by witnesses which could have influenced the arguments on a criminal appeal."

Sandoval's appeal still is pending. The New Mexico Supreme Court has now reassigned it to New Mexico public defender Kim Chavez Cook. The New Mexico Supreme Court action continues the trend of U.S. attorneys disciplined over the past year for using material from AI tools without a proper review of the information, cases and testimony generated by the AI tools.

Courts throughout the U.S. have warned lawyers about the ultimate accountability for any submission in their name, whether it comes from an AI program, or not. Attorneys face fines and admonishments over charges that AI systems generate fake legal citations and testimony and quotations.