Meet Jev: The AI Built to Make Decisions

Таймлайн эпизода

1. 0:00

   Introduction to TypeSafe, a classifier AI model from a ChatGPT co-founder

   • TypeSafe is a classifier AI model that evaluates JSON rules against inputs rather than generating text, and is claimed to be 20 to 200 times faster and 40 to 400 times cheaper with free output tokens.
   • The host plans to test it on email classification using 1,500 of his own exported emails, starting with a batch of 100 emails and eight workers.
2. 0:45

   First batch of 100 emails classified with impressive speed

   • The 100-email batch completed with an average latency of 200 ms per email, a P95 of 240 ms, and a throughput of 38 emails per second.
   • The model was given four evaluation criteria: category, priority, spam, and reply, which the host considers the core things people care about in email.
3. 2:00

   Reviewing classification results for priority and reply

   • High-priority classifications correctly flagged items like a safety identifier in the OpenAI account and a domain revocation account.
   • The reply classifier achieved roughly 91% accuracy on account violation emails, correctly identifying messages that needed a response.