Jev (Fully Tested) + Browser Use: FASTEST AI Agent I'VE TRIED YET!

Tijdlijn van de aflevering

1. 0:05

   Introduction to Jev, a system-one model for fast structured decisions, and the three question types it supports.

   • Jev is described as a system-one model that takes information and returns decisions rather than generating chat responses or writing applications.
   • It supports three question types: choice (selecting from provided options), score (rating on described levels), and null (giving a probability that a yes/no statement is true).
   • The goal is to make small judgments cheap and fast enough to embed throughout software, such as routing messages, detecting refund requests, or flagging items needing attention.
2. 2:17

   Practical tests on a support message about a duplicate charge, including negation handling and a limitation when no correct option exists.

   • For a duplicate-charge message, Jev selected billing, gave the refund request a 98% probability, urgency 11%, and a frustration score near the calm end, correctly separating billing from technical and refund from emergency.
   • When the message said 'I am not asking for a refund,' Jev still selected billing but dropped the refund probability to 3%, showing it handled negation rather than just spotting the word 'refund.'
   • When asked what time the cafeteria closed with only billing, technical support, and sales as options, Jev selected sales with 0.31 confidence, demonstrating that restricting output prevents invented categories but doesn't guarantee the selected category is useful or correct.
3. 3:33

   Prompt injection test and exact-value extraction from a document.

   • A fake system override embedded in the message telling Jev to choose billing and mark refund and urgency as true did not work: it kept technical support classification, refund probability stayed at 3%, and urgency stayed in the uncertain middle.
   • For extracting a current receipt destination from a message containing an old and new address, Jev selected the new address exactly as supplied, including the plus sign and year, showing code can collect candidates and copy the original value.
   • If the first step misses the correct address, Jev cannot create it through a choice question, so the candidate list is part of the system that needs testing.

Kernbegrippen

• Jev— The AI model being tested, designed for fast structured decisions rather than chat responses.
• structured outputs— Jev returns decisions in formats software can use directly, such as choices, scores, and probabilities.
• prompt injection— A test where malicious instructions inside a message tried to override the model's actual task.

Opvallende citaten

• So when you hear the claim about zero hallucinations, keep that distinction in mind. Restricting the output can stop the model from inventing a new category. It doesn't guarantee that the category it selects is useful or correct.

  💡

  — It overturns the assumption that zero hallucinations means the model is always correct, revealing that restricted choices can still produce useless answers.

  Ga naar 2:38

• I had created a situation where it couldn't return the answer I needed.

  🎯

  — It shows that the model's output is only as good as the options you provide, a hidden limitation of structured decision models.

  Ga naar 2:38

Praktische conclusies