At the AI Engineer New York conference, OpenAI, Anthropic, Two Sigma, and Bloomberg each tackled the same question: how much autonomy should financial agents get, and where should humans stay in the loop? The presenters broadly agreed that safety comes from thoughtful control placement, not from piling on restrictions. OpenAI's Joseph Kim framed the issue as a product design challenge, asking what users still need to review or approve as model capabilities improve.
Two Sigma's John Feminella warned against 'approval fatigue,' arguing that forcing humans to approve every action can both slow innovation and introduce risk. His firm instead uses composable controls, including auto-mode classifiers that route only the most consequential requests to people. Anthropic's Keon Shahab described a similar layered approach, building a 'trust layer' with identity, permissions, and approval boundaries, so agents can propose actions like order amendments without executing them.
Bloomberg's Andrey Rybka emphasized enforcing security outside the model itself, citing real incidents where agents followed malicious instructions. His defenses include scoped credentials, tool-call controls, audit trails, and kill switches. While the companies differ in emphasis—OpenAI on product design, Two Sigma on internal controls, Anthropic on trust boundaries, and Bloomberg on external guardrails—their shared message is that autonomy and safety are not opposites, but design problems to solve together.