Baseten
2 articles in the last 30 days.
-
Inference engineering for software engineers
A guide to moving from frontier lab APIs to open models, either through inference providers or dedicated GPUs. It covers staged scaling, KV cache sizing, and speculative decoding, with input from three providers.
-
How big is the open model economy today?
Open model inference startups report near-exponential token growth. Fireworks says it serves over 40T tokens per day for customers' custom models, and the author compares this with Google's platform volumes.