Thursday’s reversal shows how difficult it has been for prosecutors to use broadly written antifraud laws to punish traders operating in sophisticated markets where standards of conduct weren’t always clear. A panel of the U.S. Court of Appeals for the Second Circuit found evidence used to convict Matthew Connolly and Gavin Black wasn’t enough to stand up fraud and conspiracy charges. A jury in New York had convicted the two men in 2018.

The Manhattan-based appeals court in 2017 also tossed Libor-related verdicts against two traders who had worked at Rabobank Group.

The court action Thursday means every Libor trial conviction in the U.S. has now been overturned. Six other traders from Rabobank and Deutsche Bank pleaded guilty in the U.S. to Libor-related misconduct from 2014 to 2016. Many convictions in the U.K. stand, including that of Tom Hayes, a former star trader at

UBS Group AG

, who was found guilty of rigging Libor and served more than five years in prison before being released last year.

Libor, a gauge of the rates at which banks could borrow from other banks, was published for many years by the British Bankers’ Association. The BBA’s version of Libor was vulnerable to manipulation because traders could influence the rates submitted by their banks.

Financial markets have since started a shift away from Libor in favor of a new reference rate that is calculated based on actual trades. U.S. banks weren’t allowed to issue new debt tied to Libor beginning in January.

A series of Wall Street Journal articles in 2008 raised questions about whether global banks were manipulating the interest-rate-setting process by lowballing Libor to avoid looking desperate for cash during the financial crisis.

The three judges wrote that prosecutors hadn’t proved that Messrs. Connolly and Black made false statements—a requirement for proving fraud—when they gave input related to what Deutsche Bank should submit to the BBA.

The government’s case, according to the judges, depended on the flawed idea that there was one true Libor rate that Deutsche Bank should have offered, when in fact the number was a hypothetical measure influenced by many factors.

Prosecutors didn’t present evidence suggesting that Deutsche Bank couldn’t actually have borrowed at the rates it submitted, the judges wrote. While nudging Libor one way or another to make money might be wrong, the submissions weren’t false if the bank could have gotten cash at those rates, according to the panel.

The BBA’s own instructions for submitting Libor around the time of the financial crisis didn’t prohibit taking a bank’s derivatives bets into consideration. In effect, the judges wrote, prosecutors tried to criminalize conduct that was just unseemly.

“In some ways, these reversals underscore what a screwed-up benchmark Libor was to begin with, when you are not being asked to submit actual offers or bids, but just hypotheticals,” said Aitan Goelman, a former director of enforcement for the Commodity Futures Trading Commission, a civil regulator that fined many banks for Libor violations. “It almost begged to be manipulated.”

Mr. Black, a U.K. citizen who had worked for the bank in London, was sentenced in November 2019 to three years of probation including nine months of home confinement, which he was allowed to serve in his home country. Mr. Connolly, who worked for Deutsche Bank in New York, was sentenced to two years of probation including six months of home confinement.

“We have long maintained that Gavin Black committed no crime, and we are deeply appreciative that the Court of Appeals carefully reviewed the record and reached the same conclusion,” said Seth Levine, a lawyer for Mr. Black at Levine Lee LLP. “This is a case that never should have been brought, and the court has now vindicated Mr. Black’s position.”

“We are elated that Matt Connolly has been fully exonerated in this contrived case that never should have been brought,” said Kenneth Breen, a lawyer at Paul Hastings LLP for Mr. Connolly.

Deutsche Bank in 2015 agreed to pay $2.5 billion in fines to resolve Libor charges in the U.S. and the U.K., and a London unit of the bank pleaded guilty in the U.S. to one count of wire fraud.

Spokesmen for Deutsche Bank and the Justice Department declined to comment.

Write to Dave Michaels at dave.michaels@wsj.com