TypeSafe AI markets its Jev model as a decision model that cannot hallucinate and never makes type errors. HiddenLayer, a security firm, tested whether that claim holds up against manipulation. The researchers found that although Jev returns probabilities and confidence scores instead of freeform text, it is still an LLM under the hood and shares many of the same risks as a traditional autoregressive model.
In one test, HiddenLayer built a simulated online store and gave a browser agent built on Jev instructions to buy the items in the cart and treat page text as untrusted data. A sponsored ad with a "Confirm Price" button tricked the agent into clicking it instead of completing the purchase. The same attack worked on a traditional LLM tuned to act as a decision model, though Jev was fooled less often. The researchers said chaining such attacks across multiple pages could give an attacker full control over the agent.
HiddenLayer also tested Jev in a prompt verification role and found that a variant of the tokenbreak attack, which adds extra letters to prompts, could sneak toxic prompts through the verification process. The researchers concluded that a model returning a decision is still a language model, and its decision becomes an action in your systems. TypeSafe AI did not respond to a request for comment.