Topic
Positional Encoding
2 episodes
-
Thinking Machines open sources first multimodal language model, Inkling, focused on low cost and 'resistance to censorship'
Inkling, Thinking Machines' open-source multimodal MoE model (975B total / 41B active parameters), lands as a broad, balanced generalist with a standout feature: a controllable 'thinking effort' knob to dial cost vs. performance from 0.2 to 0.99. Enterprises get native text+image+audio fusion, Apache 2.0 weights, and a lighter Inkling-Small preview, but benchmarks show it trails specialized open and closed models on coding and pure reasoning, while remaining competitive on multimodality and agentic workflows. The episode debates whether the real win is the runtime control surface (Tinker platform) and a cautious, non-censoring epistemics posture — not the headline parameters.
-
Overview: Transformer Architecture
We finally sit down and define transformer architecture from the ground up, because we keep throwing the term around like it’s obvious and it really isn’t. We use the attention-as-a-room-of-index-cards picture to make the mechanism click, then connect it to why Transformers became the backbone of modern language models.