Feed: Slashdot
Feed Link: https://slashdot.org/
---

Title: California AI Policy Report Warns of 'Irreversible Harms'

Link: https://yro.slashdot.org/story/25/06/17/21421...

An anonymous reader quotes a report from Time Magazine: While AI could offer
transformative benefits, without proper safeguards it could facilitate
nuclear and biological threats and cause "potentially irreversible harms," a
new report commissioned by California Governor Gavin Newsom has warned. "The
opportunity to establish effective AI governance frameworks may not remain
open indefinitely," says the report, which was published on June 17 (PDF).
Citing new evidence that AI can help users source nuclear-grade uranium and
is on the cusp of letting novices create biological threats, it notes that
the cost for inaction at this current moment could be "extremely high." [...]
"Foundation model capabilities have rapidly advanced since Governor Newsom
vetoed SB 1047 last September," the report states. The industry has shifted
from large language AI models that merely predict the next word in a stream
of text toward systems trained to solve complex problems and that benefit
from "inference scaling," which allows them more time to process information.
These advances could accelerate scientific research, but also potentially
amplify national security risks by making it easier for bad actors to conduct
cyberattacks or acquire chemical and biological weapons. The report points to
Anthropic's Claude 4 models, released just last month, which the company said
might be capable of helping would-be terrorists create bioweapons or engineer
a pandemic. Similarly, OpenAI's o3 model reportedly outperformed 94% of
virologists on a key evaluation. In recent months, new evidence has emerged
showing AI's ability to strategically lie, appearing aligned with its
creators' goals during training but displaying other objectives once
deployed, and exploit loopholes to achieve its goals, the report says. While
"currently benign, these developments represent concrete empirical evidence
for behaviors that could present significant challenges to measuring loss of
control risks and possibly foreshadow future harm," the report says. While
Republicans have proposed a 10 year ban on all state AI regulation over
concerns that a fragmented policy environment could hamper national
competitiveness, the report argues that targeted regulation in California
could actually "reduce compliance burdens on developers and avoid a patchwork
approach" by providing a blueprint for other states, while keeping the public
safer. It stops short of advocating for any specific policy, instead
outlining the key principles the working group believes California should
adopt when crafting future legislation. It "steers clear" of some of the more
divisive provisions of SB 1047, like the requirement for a "kill switch" or
shutdown mechanism to quickly halt certain AI systems in case of potential
harm, says Scott Singer, a visiting scholar in the Technology and
International Affairs Program at the Carnegie Endowment for International
Peace, and a lead-writer of the report. Instead, the approach centers around
enhancing transparency, for example through legally protecting whistleblowers
and establishing incident reporting systems, so that lawmakers and the public
have better visibility into AI's progress. The goal is to "reap the benefits
of innovation. Let's not set artificial barriers, but at the same time, as we
go, let's think about what we're learning about how it is that the technology
is behaving," says Cuellar, who co-led the report. The report emphasizes this
visibility is crucial not only for public-facing AI applications, but for
understanding how systems are tested and deployed inside AI companies, where
concerning behaviors might first emerge. "The underlying approach here is one
of 'trust but verify,'" Singer says, a concept borrowed from Cold War-era
arms control treaties that would involve designing mechanisms to
independently check compliance. That's a departure from existing efforts,
which hinge on voluntary cooperation from companies, such as the deal between
OpenAI and Center for AI Standards and Innovation (formerly the U.S. AI
Safety Institute) to conduct pre-deployment tests. It's an approach that
acknowledges the "substantial expertise inside industry," Singer says, but
"also underscores the importance of methods of independently verifying safety
claims."

Read more of this story at Slashdot.

---
VRSS v2.1.180528