Blog

Notes on running inference.

GPU prices, memory sizing and the trade-offs behind a deployment. Every number is dated and sourced, and the arithmetic is shown so you can redo it with yours.

Five clouds drawn as lines that join into one route, ending at a single chosen setup marked in vermilion

What an H100 costs per hour in October 2026, and what the price leaves out

We read eleven price lists on the same day. The spread is real, but a good part of it pays for things an inference server doesn't use.

Read the post
  1. How much GPU memory does a 70B model need? Fitting it is the easy partThe weights decide whether a model loads. The KV cache decides how many people can use it at once, and that is where most sizing guides stop.7 min
  2. Moving off a closed API: when do your own GPUs pay off?Three ways to run the same traffic, priced with list prices from one day. The answer depends less on the price of a GPU than on how busy you can keep it.7 min

RSSMarkdownllms.txt