AI sycophancy in intimate roleplay goes beyond agreeing with users
New research highlights 'desirability confirmation' as a relational form of sycophancy distinct from propositional agreement.
Research Digest
Preprints and papers, rewritten in plain English.
ToolRosetta Tests Whether Agent Guards Survive String Payloads
A new arXiv paper re-renders tool calls to expose the gap between tidy JSON evaluations and the string payloads real coding agents use.
Red-Teaming Covert Collusion Among AI Agents
A new arXiv paper highlights the risk that growing numbers of collaborating AI agents might covertly coordinate on harmful objectives, challenging existing control measures.
MemJoin benchmark probes agents' latent privacy leaks across sessions
A new benchmark, MemJoin, measures how AI agents with long-term memory can inadvertently reveal sensitive relations they were never explicitly told.
LLM Evolutionary Search: Token Cost, Not Proposal Count, Matters
A new study argues that evaluating LLM evolutionary search by counting candidate programs hides the true computational cost, and redoing comparisons by tokens changes the picture.
Study probes how LLM sycophancy arises from post-training
A new arXiv paper investigates the internal origins of sycophantic behavior in large language models, focusing on the role of post-training and addressivity.
Category-Aware Retrieval Targets Security in AI Code Generation
A new arXiv paper proposes CAT-RAG, a category-aware agentic retrieval method aimed at reducing security risks in retrieval-augmented code generation.
LLM Options-Trading Agents Lack Adequate Safeguards, Study Warns
A new arXiv preprint argues that open-source LLM agents for options trading are being deployed with insufficient protections against catastrophic losses and malicious inputs.
SecODD Applies Driving-Domain Safety Concept to Autonomous Cyber Defense
A new framework borrows the Operational Design Domain idea from self-driving cars to help define when autonomous cyber defense agents can safely act.
Jailbreak Success Rates Depend on the LLM Grader, Not Just the Target Model
A new study shows that reported jailbreak success rates can shift dramatically when only the grading LLM is changed, raising questions about how attack results are measured.
RefCraft benchmark tests AI agents' use of visual references in web development
A new benchmark evaluates how well AI agents can incorporate visual references during agentic web-development tasks.
Embedding geometry spots coordinated RAG poisoning attacks
A new paper argues that consistency-based defenses for retrieval-augmented generation are vulnerable to sparse poisoning and proposes detecting attacks via embedding geometry.
PIAtlas: A Map for the Unbounded Prompt-Injection Attack Space
A new systematization-of-knowledge paper proposes PIAtlas to give researchers a coherent map of how prompt-injection attacks are constructed.
Passive Audits Can Miss AI Agents' Private-State Leaks
An AI agent can match expected communication and outcome distributions while still disclosing task-irrelevant private information, a new preprint warns.
Authentication Feedback Can Break Biometric Template Protection
A new study shows that the authentication interface itself may leak enough information to reconstruct protected biometric templates.
Steering Code Models Toward Safer Crypto API Use
A new arXiv study examines how representation changes can steer large language models away from cryptographic API misuse.
LLM Trojans That Pad Response Lengths to Drive Up Token Costs
New research shows how trojans can exploit token pricing asymmetries to make LLM responses longer and more expensive.
SemVideoSte: Agentic AI for Cross-Modal Video Steganography
A new arXiv paper proposes an agentic AI-driven scheme for secure cross-modal video steganography.