[ 66260400.000 ] posts.x: Full talk on two vLLM Mamba bugs, the scheduler ordering issue and the int32 overflow, and how the AI21 team traced both through logprobs.  |   [ 66260400.000 ] posts.x: Two production bugs in vLLM's Mamba support produced zero crashes and zero errors, only confidently wrong output. @aiDotEngineer published "Two Bugs…  |   [ 66257640.000 ] posts.x: @mattpocockuk @davekiss @poteto @dexhorthy @emilkowalski 🤣 brilliant!  |   [ 66255540.000 ] posts.x: @yoheinakajima 👊  |   [ 66254340.000 ] posts.x: @yoheinakajima Shockingly quick response!  |   [ 66251100.000 ] posts.x: Full talk on FriendliAI's four-pillar rebuild for agent inference, from prefix caching to cache-aware routing:  |   [ 66251100.000 ] posts.x: A tower defense game, built by two different models on the same coding-agent task: one run cost $1.50, the other 27 cents. @bgchun, founder and CEO…  |   [ 66243600.000 ] posts.x: Full talk on what's changed in quantization, KV caching, and speculative decoding since Philip's book came out:  |   [ 66243600.000 ] posts.x: A quantization method called TurboQuant promised to shrink the KV cache down to four bits this year, then turned out to cut tokens per second by more…  |   [ 66230100.000 ] posts.x: Curious what The Godfather thinks of this. @HamelHusain  |   [ 66229560.000 ] posts.x: @lukebfox1 This was such a great game! We used to have all-night LAN parties on this one!  |   [ 66229440.000 ] posts.x: Astounding!  |   [ 66179400.000 ] posts.x: Full talk on the harness layers, the files-vs-databases tradeoff, and context rot:  |   [ 66179400.000 ] posts.x: Most of what makes an AI agent reliable has nothing to do with the model itself. In "Total Recall: Agent Memory and Harness Engineering,"…  |  

[ 66260400.000 ] posts.x: Full talk on two vLLM Mamba bugs, the scheduler ordering issue and the int32 overflow, and how the AI21 team traced both through logprobs.  |   [ 66260400.000 ] posts.x: Two production bugs in vLLM's Mamba support produced zero crashes and zero errors, only confidently wrong output. @aiDotEngineer published "Two Bugs…  |   [ 66257640.000 ] posts.x: @mattpocockuk @davekiss @poteto @dexhorthy @emilkowalski 🤣 brilliant!  |   [ 66255540.000 ] posts.x: @yoheinakajima 👊  |   [ 66254340.000 ] posts.x: @yoheinakajima Shockingly quick response!  |   [ 66251100.000 ] posts.x: Full talk on FriendliAI's four-pillar rebuild for agent inference, from prefix caching to cache-aware routing:  |   [ 66251100.000 ] posts.x: A tower defense game, built by two different models on the same coding-agent task: one run cost $1.50, the other 27 cents. @bgchun, founder and CEO…  |   [ 66243600.000 ] posts.x: Full talk on what's changed in quantization, KV caching, and speculative decoding since Philip's book came out:  |   [ 66243600.000 ] posts.x: A quantization method called TurboQuant promised to shrink the KV cache down to four bits this year, then turned out to cut tokens per second by more…  |   [ 66230100.000 ] posts.x: Curious what The Godfather thinks of this. @HamelHusain  |   [ 66229560.000 ] posts.x: @lukebfox1 This was such a great game! We used to have all-night LAN parties on this one!  |   [ 66229440.000 ] posts.x: Astounding!  |   [ 66179400.000 ] posts.x: Full talk on the harness layers, the files-vs-databases tradeoff, and context rot:  |   [ 66179400.000 ] posts.x: Most of what makes an AI agent reliable has nothing to do with the model itself. In "Total Recall: Agent Memory and Harness Engineering,"…  |