OpenAI CEO Sam Altman on Monday night announced a revised agreement with the Pentagon governing the department’s use of AI services, saying it provides stronger assurances that the military will not use OpenAI’s systems for domestic surveillance.
The new agreement states that “AI systems will not be intentionally used for domestic surveillance of U.S. persons and nationals,” according to a post on OpenAI’s website. OpenAI faced some backlash after news of an initial agreement between a major AI company and the Department of Defense surfaced on Friday. Many observers argued that the original language shared on OpenAI’s website provided enough loopholes for the government to monitor Americans.
The move comes after weeks of heated discussions between rival AI company Anthropic and the Department of Defense over how the military can use advanced AI systems. The Pentagon had asked Anthropic to agree to use its systems for “any lawful purpose,” but Anthropic argued that its systems could not be used for domestic surveillance or control of lethal autonomous weapons. Until last week, Anthropic was the only major AI company whose services were actively used on a classified network.
Without guardrails, researchers argue, AI could allow authorities to surveil individuals with unprecedented speed and accuracy, combing through mountains of digital data to track people’s movements and actions.
“It is important that we protect the civil liberties of Americans,” Altman wrote in a post on X-TV announcing the new contract language Monday night, saying he would prefer more restrictions on domestic surveillance. “The Department of Defense has also confirmed that our services will not be used by Department of the Army intelligence agencies (such as the NSA).”
Katrina Mulligan, head of national security partnerships at OpenAI, added in a separate post on X Tuesday morning that “the defense intelligence component is excluded from this contract” and said she is open to future cooperation with the NSA “provided appropriate safeguards are in place.”
OpenAI did not respond to requests for comment.
Many observers remained unperturbed Tuesday, concerned that the pieces of the Pentagon contract that OpenAI released remained intentionally vague, providing carve-outs for domestic surveillance by various intelligence agencies within the Pentagon. The full text of the contract has not been made public.
“OpenAI said it has contractually agreed that the Department of the Army will not use ChatGPT in agencies that monitor American citizens,” said Brad Carson, a former congressman and Army general counsel who now heads the policy group Americans for Responsible Innovation in Washington, D.C. “They have been willing to point out contract language if it benefits them, but they refuse to make this contract provision available to the public.”
“I reluctantly came to the conclusion that this provision doesn’t actually exist and they’re just trying to fabricate it,” Carson told NBC News. Carson recently founded an AI-focused super PAC and received $20 million from OpenAI rival Anthropic.
Several legal experts agreed that greater transparency around the entire contract and other key provisions is needed to properly evaluate the company’s claims.
“We still need to see the entire deal to say anything with a reasonable level of confidence,” said Brian McGrail, senior adviser at the Center for AI Safety, a nonprofit research and advocacy group. “This is definitely a step in the right direction, and I’d like to give OpenAI some credit.”
OpenAI’s agreement with the Department of Defense was announced shortly after Secretary of Defense Pete Hegseth said he was designating rival AI company Anthropic, which has been in contract negotiations with the Pentagon for years, as a supply chain risk to national security. Anthropic said the designation, which forces the Department of Defense and contractors to stop using Anthropic’s services for defense purposes, had never been publicly applied to a U.S. company before.
At an event in Sausalito, Calif., on Monday, retired Gen. Paul Nakasone, former head of the National Security Agency and U.S. Cyber Command, said the Pentagon should work to incorporate technology from all major U.S. AI companies into national defense.
“We need Anthropic, we need OpenAI, and we need all of our large language modeling companies to partner with the government,” Nakasone, a member of OpenAI’s board of directors, said at a conference hosted by the Aspen Institute. “I don’t think the supply chain part is good. The discussion over the weekend and the content of that discussion was painful for me to hear. As an American citizen, as someone who has served in the government, I just don’t think it’s right, okay? This is not a supply chain risk.”
Antropic has long maintained that the Pentagon cannot use its AI systems for domestic mass surveillance or direct use on autonomous weapons, but in December it added concessions for the military to use its systems for cyber and missile defense purposes. After Anthropic CEO Dario Amodei and Mr. Hegseth met last Tuesday, the Pentagon gave Anthropic an ultimatum to reach an agreement by 5pm ET on Friday.
But on Thursday, an Antropic spokesperson told NBC News that the Pentagon’s latest “compromise and framed language was combined with legal language that allowed these safeguards to be freely ignored.”
But as Anthropic’s relationship with the Department of Defense fell apart, OpenAI’s relationship deepened, and Friday’s contract announcement added new intrigue to a saga that had already captivated many in the tech and defense communities. In a post Monday night, Altman said OpenAI’s haste to finalize the deal made negotiations appear “opportunistic and sloppy,” despite OpenAI’s “genuine efforts to de-escalate the situation and avoid a worse outcome.”
Over the weekend and early this week, numerous legal experts examined OpenAI’s latest public contract language, attempting to determine whether the company’s terms actually added substantial protections beyond the Department of Defense’s “any lawful use” standard.
“I’m baffled as to why the Pentagon would accept this language when they just tried to nuke Antropic for something very similar to this,” Charlie Bullock, a senior fellow at the Institute for Law and AI think tank, wrote about X after the latest language appeared.
Many legal experts say the government will interpret contract terms as broadly as possible, arguing that each word in the contract is significant.
“The pattern that we’ve seen over and over again in these surveillance discussions is that the intelligence community and the national security community interpret the exceptions very broadly, much more broadly than any normal rational person would,” McGrail said. “And because much of its content is secret, there is limited public visibility for the public to respond.”
“So could there be a new loophole exploited here that we didn’t foresee? It’s very possible,” McGrail added.
Experts are also concerned that the government could change the boundaries of “any lawful use” with new executive orders or legal opinions, focusing on whether contracts are permanently locked into today’s notions of legality.
Recent debates over military use of AI for domestic surveillance have focused specifically on whether the government can use commercially available data for operations, as other methods of monitoring American citizens may prove more difficult to obtain legal approval.
For years, companies that serve or display ads on mobile phones and laptops have been able to collect targeted data about users, including precise location data, and sell that information to various government agencies to determine an individual’s travel and behavioral patterns.
Mulligan, OpenAI’s national security director, said in a Monday night X post that the contract’s “new language emphasizes that domestic surveillance involving commercially obtained information is prohibited under this agreement.”
Sen. Ron Wyden (D-Ore.) has repeatedly warned in recent years that the federal government is buying off-the-shelf data on Americans for surveillance purposes and criticized the Pentagon for not silencing Anthropic’s privacy concerns.
“The Department of Defense is furious over Anthropic, which requires minimum ethical guardrails for how the Department of Defense uses its products,” Wyden said in an emailed statement. “This is a serious concern given AI’s ability to turn a wide range of public and commercial data into extremely revealing profiles of Americans. Information about location data, web browsing records, mental health, political activity, and religious affiliations are all available for pennies on the open market, potentially making Americans targets for doing perfectly legal things.”
“Creating AI profiles of Americans based on that data would represent a terrifying expansion of mass surveillance that should not be allowed, regardless of what current outdated laws say.”
Anthropic CEO Amodei has repeatedly said that a stronger commitment from the Department of Defense not to use AI to monitor Americans is needed because the law has not kept pace with AI’s increasingly powerful ability to analyze or parse vast amounts of data. Recent research has also shown that today’s AI systems can identify individuals even when the underlying data is supposedly anonymized.
Protesters against OpenAI’s initial deal with the Department of Defense surrounded OpenAI’s San Francisco headquarters this weekend, holding up chalk messages urging employees to remain skeptical of the company’s terms, while uninstalls of OpenAI’s ChatGPT app spiked following news of the agreement.
Michael Horowitz, a former assistant secretary of defense for emerging capabilities and now a political science professor at the University of Pennsylvania, told NBC News that the dispute between the Pentagon and Antropic went beyond simple contract terms.
“This dispute reflects a breakdown in trust between Anthropic and the Department of Defense. Anthropic does not trust the Department of Defense to use its technology responsibly, and the Department of Defense does not trust that it will allow Anthropic to use its technology for what the Department of Defense considers critical national security use cases,” Horowitz said. “Part of it is cultural differences, part of it is politics, and part of it is personality.”
