Topic
Claude Sonnet 4 6
3 episodes
-
I Spent May Evaluating Different Engines for OCR | Towards Data Science
Justy and Cody react to a hands-on OCR engine shootout across 93 messy real-world documents. The author’s core claim: OCR is now a routing problem, not a single-engine race—specialist models excel in their niche but break on out-of-domain docs, while paid structured APIs may be overkill for many use cases. They debate the economics, practicality of ‘classify-then-route,’ and whether most teams should just test on their own data.
-
Replacing RAG with bash cut AI retrieval costs 30%
Justy and Cody dig into the argument behind direct corpus interaction, where agents use terminal tools like grep and find instead of relying only on vector search. They like the core point that retrieval interfaces can bottleneck reasoning, but they keep it grounded: this looks strongest for exact-evidence tasks in changing workspaces, and weakest as a blanket replacement for broad recall across huge corpora.
-
ClawMark: A Living World Benchmark for Multi Turn, Multi Day, Multimodal Coworker Agents
ClawMark is a benchmark for evaluating AI agents as persistent coworkers across multi-day workflows with dynamic, stateful environments. Unlike existing benchmarks that run single-episode tasks in static environments, ClawMark spans multiple in-universe workdays with exogenous state changes (emails arrive, calendars shift, files update) between turns, multimodal evidence (PDFs, audio, video, spreadsheets), and deterministic rule-based scoring via 1,537 Python checkers. The benchmark contains 100 tasks across 13 professional scenarios running against five sandboxed services (filesystem, email, calendar, knowledge base, spreadsheet). Current frontier models reach 75.8 weighted score but only 20% strict task success, revealing that adaptation to changing state remains a core unsolved challenge.