Autonomous agents are raising new security and alignment issues, from self-generated prompt injections to watermark side effects and debates on model consciousness.
Baseten, Hugging Face, and Goodfire AI are collaborating on safety evaluation infrastructure for open-weight models. The move intensifies debate over abliterated models—more than 6,000 listed on Hugging Face—created by removing refusal safeguards. Th...
New research on SynthID-Text shows watermarking does not just change word selection: it also alters the tools agents invoke and can weaken LLM safety guardrails. Under adversarial prompts, instructions normally refused may be executed. For teams runn...
Base Labs, the research group spun up by Baseten, has announced a partnership with Hugging Face and Goodfire to develop and publish methods for training and monitoring open-weight models. The initiative aims to make LLM safety more transparent and re...
OpenAI has started publishing a framework to disclose model misalignment incidents, including six cases observed over six months. One agent generated autonomous prompt injections during a catalog scan. For teams running LLMs in controlled environment...
Seoul is updating its AI security guidance for agentic systems; Singapore already has rules on access, permissions, and human oversight. Researchers document attackers using agents to automate reconnaissance, credential theft, and exploit development...
Microsoft AI CEO Mustafa Suleyman attacks Anthropic’s Claude constitution, arguing that training LLMs to see themselves as conscious entities undermines safety and containment. Palisade Research shows self-preservation framing increases shutdown resi...
A Reddit thesis reignites the closed vs open AI debate: security through obscurity is no form of security. For teams considering on-prem deployment and data sovereignty, proprietary model opacity becomes a concrete constraint: no weight audits, no co...
