SemiAnalysis InferenceX
1 article in the last 30 days.
-
Inference Performance Index: changing the way we measure inference capacity
Epoch AI proposes an index that compares how many requests chips can serve concurrently. It estimates GB300 offers about 25× H100's inference capacity despite only 2.5× the 8-bit FLOP/s.