Deepseek did it again...

Tijdlijn van de aflevering

1. 0:00

   DeepSeek V4.1 Flash launches as a cheap, fast open-weights model that benchmarks near frontier models like Opus 5 and GPT 5.6 Soul.

   • DeepSeek V4.1 Flash is incredibly cheap and fast, and benchmarks show it matching Opus 5 and GPT 5.6 Soul, continuing the pattern where open-weights models reach previous-generation frontier quality about six months behind the absolute frontier.
   • The model is a 552 billion parameter mixture of experts with only 8 billion active parameters for input and 16 billion for output, making it relatively small compared to GPT 5.6's trillion parameters and Astra's estimated 7-10 trillion.
   • On benchmarks it scores 30 on Terminal Bench 3.0 (only Opus 5 beats it) and 74.2 on Deep Suite, beating both Opus and GPT 5.6, though the host notes his own tests did not match the benchmark hype.
2. 3:04

   DeepSeek's efficiency gains slash memory requirements, cutting KV cache HBM needs to a quarter and SSD storage to an eighth.

   • The KV cache only needs a fourth of the high bandwidth memory and an eighth of the SSD storage, which matters because HBM prices have been spiking from around $25 per gigabyte up to $10 per gigabyte as AI consumes all available memory.
   • The memory footprint shrank eight times from DeepSeek V1 to V3.2, then 13 times smaller to V4 Flash, and another 4x smaller from V4 to V4.1, reflecting China's strength in making existing technology more efficient and faster rather than necessarily better.
   • This efficiency is DeepSeek's answer to the global memory crunch, letting them serve many more tokens without paying for all that memory, and it shows up as extreme output speed reminiscent of the early Groq days.
3. 6:00

   Pricing is extremely low with peak and off-peak tiers, and the model is fully open weights so users can self-host.

   • Off-peak pricing is 15 cents per million input tokens and 60 cents per million output tokens, doubling during peak hours to 30 cents input and $1.20 output, with cache hits costing a fraction of a penny.
   • Because it is open source and open weights, users can download it, avoid giving DeepSeek their data, use any NeoCloud, or potentially run it locally once quantized if they have enough VRAM.
   • The host contrasts this with frontier models like OpenAI and Anthropic charging around $50 per million output tokens, an orders-of-magnitude difference, though the vast majority of the economy only needs cheap models capable enough for 95% of use cases.

Kernbegrippen

• DeepSeek V4.1 Flash— The newly released open-weights model that is the main subject of the episode.
• Mixture of Experts— Architecture that activates only a small subset of weights per query, enabling extreme efficiency.
• Efficiency gains— The core theme: DeepSeek's ability to cut memory and compute requirements dramatically.

Opvallende citaten

• And there's one big butt here. I've put it through a few tests and it didn't perform as well as I would have hoped, especially compared to the benchmarks, but I'm going to show you that later.

  💡

  — Reveals a gap between benchmark scores and real-world performance, challenging the assumption that high benchmarks mean practical capability.

  Ga naar 0:36

• So of a 552 billion parameter model, only a tiny fraction of the weights are actually being used.

  🤯

  — Highlights the extreme sparsity of mixture-of-experts models, showing how a huge model can run efficiently.

  Ga naar 2:29

Praktische conclusies