Among Mr. Zatko’s claims are that Twitter executives, including Chief Executive

Parag Agrawal,

deliberately undercounted the prevalence of spam on the platform. Those claims could further complicate Twitter’s battle with Mr. Musk, whom the company sued in July to enforce a $44 billion takeover deal. Mr. Musk has alleged Twitter misrepresented its business, particularly as it relates to the level of spam or bot accounts—claims Twitter denies.

A five-day nonjury trial is slated to begin in October.

The existence of the whistleblower complaint was earlier reported by the Washington Post and CNN.

A Twitter spokeswoman said Mr. Zatko was fired “for ineffective leadership and poor performance” and that the complaint “is riddled with inconsistencies and inaccuracies and lacks important context.”

A lawyer for Mr. Musk said: “We have already issued a subpoena for Mr. Zatko, and we found his exit and that of other key employees curious in light of what we have been finding.”

Twitter shares were down roughly 5% in Tuesday intraday trading.

Mr. Zatko, a former hacker who is known as “Mudge,” has been a noted computer-security researcher for decades. He was a member of a Boston cybersecurity collective that came to prominence in 1998 when it offered warnings about the state of national cybersecurity in testimony to the U.S. Senate. During one Senate hearing, the group told lawmakers they could take down the internet in 30 minutes.

He was hired by Twitter in late 2020 after a career that included other corporate roles.

Whistleblower Aid’s founder John Tye said Mr. Zatko first approached the nonprofit in early March through the encrypted messaging app Signal. Mr. Tye said Mr. Zatko has never met or spoken with Mr. Musk and that Mr. Musk’s team hasn’t been in contact with the nonprofit about Mr. Zatko’s complaint.

“He sees this whistleblowing as sort of the last resort,” Mr. Tye said of Mr. Zatko. “He obviously worked hard inside the company, used the internal channels and ultimately has ended up as a whistleblower.”

Mr. Zatko was brought into Twitter by co-founder

Jack Dorsey

after a high-profile hack by a teenager who bypassed the company’s securities systems. Mr. Dorsey “specifically recruited Mudge for his reputation of speaking truth to power,” according to the complaint.

Mr. Dorsey, however, was only a sporadic presence at the company, and the new hire—who had hundreds of staff reporting to him—was quickly overwhelmed by the task at hand, according to the complaint. At one point, Mr. Agrawal told his team, “Twitter has 10 years of unpaid security bills,” per the complaint.

The relationship between Mr. Zatko and Twitter’s leadership deteriorated over the subsequent months, according to both parties. Mr. Zatko helped oversee a critical report on Twitter’s ability to fight misinformation and spam, which other executives watered down, according to the complaint, which said Mr. Zatko was told by a Twitter lawyer that the changes were intended to hide the findings and prevent them from leaking internally or externally.

The complaint also expresses concerns about Twitter’s ties to foreign governments and says the company may have foreign spies on its payroll. It states that Mr. Zatko believed that the Indian government had forced the company to knowingly hire at least one employee who had access to “vast amounts of Twitter sensitive data.” India’s Washington embassy didn’t immediately respond to a request for comment.

Earlier this month, a former Twitter employee was found guilty by a U.S. jury of spying for Saudi Arabia by passing on private user information associated with critics of the kingdom in exchange for hundreds of thousands of dollars while he worked at the company from 2013 to 2015.

Much of the complaint, though, deals with fake or spam accounts, a topic that Mr. Musk drew attention to in his takeover bid for Twitter.

Like the

Tesla Inc.

CEO, Mr. Zatko alleges that Twitter miscounts such users by focusing only on what are known as monetizable daily users, or MDAU, rather than all total daily users. The former category counts only those accounts that are thought to view advertising.

“There are many millions of active accounts that are not considered ‘mDAU,’ either because they are spam bots, or because Twitter does not believe it can monetize them,” Mr. Zatko’s complaint says. “These millions of non-mDAU accounts are part of the median user’s experience on the platform.”

Twitter has said it has a system for measuring users and spam that entails multiple human reviews of thousands of accounts sampled at random over time.

Mr. Zatko’s complaint said he attempted to formally notify Twitter’s board of his concerns but was steered off by Mr. Agrawal.

In a memo to employees Tuesday about the whistleblower complaint, Mr. Agrawal said: “I know this is frustrating and confusing to read, given Mudge was accountable for many aspects of this work that he is now inaccurately portraying more than six months after his termination.” Mr. Agrawal defended Twitter’s work on privacy and security, while adding that the attention the complaint has brought to the company will make its work harder. “We will pursue all paths to defend our integrity as a company and set the record straight,” he said.

Twitter in 2011 reached an agreement with the Federal Trade Commission to maintain rigorous security, including limiting the number of employees with access to its key security and privacy controls. Mr. Zatko alleges that the company is in violation of that accord. The FTC didn’t respond to a request for comment.

Copies of the complaint were sent to the Senate Judiciary and Intelligence committees, aides of each panel said.

Democrats and Republicans have raised concerns about Twitter and other social-media companies in recent years over how they use and protect customer data, and have considered legislation that could require firms to adhere to certain data transparency or security standards. “If these claims are accurate, they may show dangerous data privacy and security risks for Twitter users around the world,” Sen.

Dick Durbin

(D., Ill.), chairman of the Judiciary Committee, said in a statement.

—Dave Michaels, Erin Mulvaney and Dustin Volz contributed to this article.

Corrections & Amplifications
Parag Agrawal is the CEO of Twitter. An earlier version of this article incorrectly spelled his last name as Agarwal. (Corrected on Aug. 23)

Write to Sarah E. Needleman at sarah.needleman@wsj.com