Anthropic and OpenAI Face NYC AI Transparency Push
Former lab employees warned of weak oversight as New York City considered third-party validation, incident reporting, whistleblower protections and civil liability.
Listen
AI narration
12:56
0:00 / 12:56
AI SummaryGenerated from this article
Former employees of Anthropic, OpenAI and Google DeepMind warned New York City lawmakers on October 5, 2026, that frontier AI development lacks sufficient oversight and transparency. Current company representatives emphasized safety work but did not provide numerical estimates of catastrophic risk.
The City Council proposed six bills to address the gap: third-party validation before deployment, disclosure requirements, rapid incident reporting within 24 hours, whistleblower protections for city contractors, civil liability for foreseeable harms from jailbreaking, and an AI emergency response plan. These remain proposals under consideration and are not yet law.
Former employees of Anthropic, OpenAI and Google DeepMind told New York City lawmakers that frontier AI development lacks sufficient oversight and transparency. Current OpenAI and Meta representatives emphasized safety work and potential benefits, but did not provide a numerical estimate of catastrophic risk, according to Associated Press reporting.
At the City Council’s October 5, 2026, Committee of the Whole hearing, the central question was how the public should evaluate safety assurances when much of the evidence remains inside the companies building the systems.
The hearing brought major AI labs and former insiders onto the same public record under oath. It offered an opportunity to examine whether voluntary assurances should be backed by disclosure requirements, outside validation and legal protections for people reporting safety concerns. It did not establish that catastrophe is inevitable or that a particular product had failed.
Former Insiders Warned About Oversight, Not Just Capability
AP reported that former employees from OpenAI, Anthropic and Google DeepMind warned lawmakers about inadequate oversight and transparency in frontier AI development. Their concern was institutional as well as technical: who can inspect the evidence, challenge a safety assessment and act when they believe safeguards are insufficient?
Before the hearing, the Council’s September 16 announcement cited departing Anthropic researcher Jacob Coxon’s warning that the technology “could kill us all by the end of the decade” without additional safeguards.
That was a former researcher’s forecast, quoted by the Council as part of its rationale for convening the hearing, not an established scientific prediction. The announcement also does not establish that Coxon repeated those exact words during testimony.
Former employment gives a witness relevant experience, but does not automatically validate every conclusion about future systems. Predictions about loss of control or extinction remain risk judgments whose evidentiary weight depends on the observations, assumptions and reasoning supporting them.
For lawmakers, the more actionable issue was whether outsiders have enough information to assess those judgments. A transparency requirement can be evaluated without first agreeing on a probability of extinction.
OpenAI and Meta Emphasized Safety and Benefits
OpenAI and Meta representatives highlighted their safety efforts and AI’s potential social benefits, AP reported. These were company positions, not independent verification that their safeguards are sufficient.
Work with Zeniteq
Let’s work together
We’re open to thoughtful collaborations with teams building in AI. Explore the ways we can work together.
The representatives did not give lawmakers a numerical estimate of catastrophic risk. That omission establishes neither that the systems are safe nor that a catastrophe is likely. Even a single number would be difficult to interpret without a defined outcome, time horizon and methodology.
Lawmakers can ask more concrete oversight questions: which risks were evaluated, what the tests found, what would trigger a deployment restriction, and who can independently examine those decisions. A company’s confidence in its safety work is different evidence from an external assessment of that work.
Testifying under oath added accountability to the exchanges, though it did not turn them into a technical audit. The hearing created a public record against which subsequent statements and disclosures could be compared.
The Council’s September 28 attendance announcement identified Anthropic, OpenAI, Google and Meta as participating companies. It said that, apart from Meta, they confirmed attendance after the Council explicitly threatened subpoenas.
Although the Council initially requested participation from chief executives, those requests are not proof that every named CEO personally appeared. The reporting cited here describes company representatives.
NYC’s Proposals Would Turn Assurances Into Obligations
The Council’s September 25 legislative package outlined several possible routes to enforceable accountability. These remained proposals under consideration; holding the hearing did not create the obligations.
Their scopes differ substantially. Some would reach AI businesses operating in the city; others concern city contracts, municipal workers or government infrastructure. Describing them as a single AI regulation would obscure who must comply and what each measure would require.
Third-Party Validation Before Sale or Deployment
Introduction 2602 would make it unlawful for a business to market, offer for sale or deploy an AI system in New York City without third-party validation.
According to the Council’s summary, validators would examine data quality, bias, decision outputs, privacy and security, along with additional validation requirements set by the city’s Cyber Command. They would also have to disclose relevant conflicts of interest.
The proposal would require a human override, described as a “kill switch,” and require validators to verify that it exists. Businesses and validators could face a $25,000 penalty for each instance of marketing, selling or deploying a system without validation, or where validation was falsified.
This is the package’s clearest move toward external scrutiny. The announcement, however, does not provide a complete audit methodology or establish that every validation would include frontier-model catastrophic-risk testing.
A privacy review, a bias assessment and an evaluation of autonomous cyber capabilities answer different questions. Confirming that an override exists is also narrower than demonstrating that people could reliably stop every harmful activity in a complex deployment. The eventual standards would determine how much assurance validation actually provides.
Disclosure and Rapid Incident Reporting
Introduction 2603 would require certain disclosures about AI tools and prohibit false or misleading representations about their safety. The Council’s announcement does not specify the full disclosure scope, so it should not be described as requiring publication of every internal evaluation, training dataset or safety incident.
Introduction 2601 is more specific, but narrower. It would require Cyber Command to establish standards and procedures for identifying AI safety incidents related to covered city contracts.
If a contractor or the contracting agency became aware of such an incident during the contract term, it would have to notify Cyber Command in writing within 24 hours. Cyber Command would then have to publicly disclose the occurrence of a reported incident within 24 hours.
This would create an incident-reporting channel with deadlines. Based on the Council’s summary, it would not impose the same reporting requirement on every incident at every AI lab. A city-contract reporting rule could provide useful public evidence about government deployments without becoming a general disclosure regime for the entire AI industry.
Whistleblower Protections and Financial Incentives
Two proposals address the people who might bring otherwise inaccessible information to authorities.
Introduction 2604 would clarify that the city’s whistleblower protections cover New York City employees and covered city contractors and subcontractors who report AI-related conduct they know, or reasonably believe, presents a public safety threat. That scope does not support a claim that the bill protects every private-sector employee at Anthropic, OpenAI, Google or Meta.
Introduction 2605 would allow individual whistleblowers to receive a portion of fines or penalties recovered from AI companies that violate applicable laws.
The Council’s summary does not specify an award percentage or establish that reporting a concern would automatically produce a payment. The incentive is tied to recovered penalties for legal violations.
Protection concerns the consequences of speaking up; a financial incentive offers a potential reward for information leading to enforcement. Neither, by itself, guarantees that a report will be investigated effectively.
Civil Liability for Foreseeable Harm
Introduction 2600 would establish a private right of action against AI companies for foreseeable harms arising from malicious use or circumvention of safety controls, commonly called jailbreaking.
The Council described three conditions: the harm was foreseeable to the company, the company failed to implement reasonable safeguards, and a third party caused the harm by exploiting that failure.
Affected individuals would have a possible legal remedy, though the proposal is not described as imposing automatic liability for every harmful output or every misuse of an AI tool. Foreseeability, the adequacy of safeguards and the connection to the harm would be central questions.
Introduction 2606 would separately require Cyber Command, working with New York City Emergency Management, to develop a response plan for AI-related events that compromise city information systems or infrastructure, or disrupt government operations and public safety.
The Enforcement Details Will Determine the Impact
The hearing exposed a gap between insiders’ warnings about development practices and companies’ assurances about safety work. Neither should be treated as conclusive simply because a witness spoke under oath.
The proposed legislation offers ways to test parts of those competing accounts. Validation can require an outside assessment, and incident reporting can establish a dated public record. Whistleblower measures can help information reach authorities, while civil liability can give injured people a route to challenge inadequate safeguards.
Whether those mechanisms work will depend on their final scope, standards and enforcement. A requirement to obtain validation is only as useful as the assessment it demands; a disclosure rule is only as informative as the material it covers.
New York City does not need to settle an extinction forecast to consider those measures. It does need to specify what evidence companies must provide, who can assess it and what happens when a legally required safeguard is missing.
Frequently Asked Questions
3 questions
1
Are NYC’s proposed AI safeguards already law?
No. The Council’s September 25 announcement described proposed bills covering validation, disclosures, incident reporting, whistleblower measures and civil liability. Holding the October 5 hearing did not itself make those obligations enforceable against Anthropic, OpenAI, Meta or other AI businesses.
2
Would the whistleblower proposals protect every AI lab employee?
No. Introduction 2604 concerns New York City employees and covered city contractors and subcontractors reporting AI-related public safety threats. Introduction 2605 separately proposes financial incentives tied to penalties recovered from AI companies that violate applicable laws. Neither summary establishes blanket protection for every private-sector AI lab worker.
3
Did the hearing establish the probability of an AI catastrophe?
No. Former insiders’ warnings were testimony and risk judgments, not verified predictions. AP reported that OpenAI and Meta representatives did not provide a numerical catastrophic-risk estimate. That omission establishes neither safety nor danger. Any probability would need a defined outcome, time horizon and supporting methodology to be meaningfully assessed.
