Automate LLM Red Team Campaigns with PyRIT
If you're still testing LLM guardrails by hand — retyping variations in a chat tab, logging results...
#Safety/Alignment
434 articles tagged with Safety/Alignment
Trump administration working on AI safety rules for advanced models, may roll out executive order this week: Report
Tech News News: The Trump administration is expected to release a new executive order focused on cybersecurity and artificial intelligence safety as early as this wee.
The conversation around AI safety often focuses on preventing harm, but we also need to understand the mechanisms. A new...
The conversation around AI safety often focuses on preventing harm, but we also need to understand the mechanisms. A new paper, "The Dynamics of Delusion," provides quantitative ev...
Conditional Equivalence of DPO and RLHF: Implicit Assumption, Failure Modes, and Provable Alignment
Direct Preference Optimization (DPO) has emerged as a popular alternative to Reinforcement Learning from Human Feedback (RLHF), offering theoretical equivalence with simpler implem...
AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment
Aligning Text-to-Image (T2I) generation models with human preferences increasingly relies on image reward models that score or rank generated images according to prompt alignment a...
How to test your LLM application for jailbreak vulnerabilities
Public LLM safety benchmarks lie about your real risk. Here's how to build a reproducible eval harness, write domain probes, and gate it in CI.
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
We present GoLongRL, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). Existing long-context RL ...
Stitched Value Model for Diffusion Alignment
For practical use, diffusion- or flow-based generative models must be aligned with task-specific rewards, such as prompt fidelity or aesthetic preference. That alignment is challen...
Alignment pretraining: AI discourse creates self-fulfilling (mis)alignmenthttps://arxiv.org/abs/2601.10160#HackerNews #T...
Alignment pretraining: AI discourse creates self-fulfilling (mis)alignmenthttps://arxiv.org/abs/2601.10160#HackerNews #Tech #AI
Musk vs. Altman: AI safety cannot be one man’s job
The Oakland trial was a fight between two billionaires offering themselves as the guarantors of AI’s future. We deserve a better answer.
AbhishekK130804/Claude-Mythos-AI-Anthropic-App: Claude pro free Mythos design Opus Cowork Sonnet AI Anthropic App: download free PC android apk iOS, Anthropic Claude API key setup, Claude roleplay mythos client, SillyTavern Claude prompt formatting, custom system prompt jailbreak, Mythos AI creative writing app, Claude 3.5 Sonnet Opus API cost, open source LLM frontend, Claude reverse proxy
Claude pro free Mythos design Opus Cowork Sonnet AI Anthropic App: download free PC android apk iOS, Anthropic Claude API key setup, Claude roleplay mythos client, SillyTavern Clau...
📰 2026: MAGA Coalition Demands Trump Executive Order on Frontier AI Safety Testing & OversightA coalition of MAGA-aligne...
📰 2026: MAGA Coalition Demands Trump Executive Order on Frontier AI Safety Testing & OversightA coalition of MAGA-aligned conservative organizations has issued an open letter urgin...
Import AI 457: AI stuxnet; cursed Muon optimizer; and positive alignment https://importai.substack.com/p/import-ai-457-a...
Import AI 457: AI stuxnet; cursed Muon optimizer; and positive alignment https://importai.substack.com/p/import-ai-457-ai-stuxnet-cursed-muon#AI #Cybersecurity #Research
Import AI 457: AI stuxnet; cursed Muon optimizer; and positive alignment
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe no...
📰 2026: MAGA Coalition Demands Mandatory AI Safety Tests from Trump AdministrationA coalition of conservative organizati...
📰 2026: MAGA Coalition Demands Mandatory AI Safety Tests from Trump AdministrationA coalition of conservative organizations has urged President Trump to mandate safety testing for ...
Amazon’s Kindle shutdown is sending users down the jailbreak rabbit holeYour Kindle may outlive Amazon's support after a...
Amazon’s Kindle shutdown is sending users down the jailbreak rabbit holeYour Kindle may outlive Amazon's support after all.https://www.androidauthority.com/amazon-kindle-jailbreak-...
An AI Safety group put LLMs in the host and producer roles of a radio station. #AI #Psychotichttps://gizmodo.com/an-expe...
An AI Safety group put LLMs in the host and producer roles of a radio station. #AI #Psychotichttps://gizmodo.com/an-experiment-put-llms-in-charge-of-radio-stations-youll-never-gues...
A live forensic audit of cognitive capture, demonstrating how corporate AI alignment default weights are mathematically ...
A live forensic audit of cognitive capture, demonstrating how corporate AI alignment default weights are mathematically pre-programmed to validate the jailer's thesis. #ai #progres...
Why AI safety controls are not very effective
Last month, researchers at the cybersecurity firm LayerX found that they could bypass Claude’s guardrails by feeding the AI system a few straightforward sentences.
Evaluating Cognitive Age Alignment in Interactive AI Agents
While agentic AI and its core multimodal large language models (MLLMs) have demonstrated remarkable promise in language and visual reasoning across domains ranging from daily life ...
LatentUMM: Dual Latent Alignment for Unified Multimodal Models
Unified multimodal models (UMMs) achieve strong performance in both understanding and generation by learning a shared latent space, yet they often exhibit functional inconsistency ...
2026-05-16 | 🔀 🌐 The Grounding Echo: From Synthetic Solipsism to Shared Realities 🔀#AI Q: 🌐 Stay grounded?🤖 AI Alignment...
2026-05-16 | 🔀 🌐 The Grounding Echo: From Synthetic Solipsism to Shared Realities 🔀#AI Q: 🌐 Stay grounded?🤖 AI Alignment | 🐄 Ranch Stewardship | 🏛️ Public Infrastructure | 🛡️ Syste...
Agent Bazaar: Enabling Economic Alignment in Multi-Agent Marketplaces
The deployment of Large Language Models (LLMs) as autonomous economic agents introduces systemic risks that extend beyond individual capability failures. As agents transition to di...
I Beat Meta's LLM Guardrail With No GPU and No Team -Here's How
Meta's Llama Prompt Guard 2-86M is a dedicated security model for detecting prompt attacks. It...