Size the workload. Then choose the GPU.

For inference, specify the model revision, precision, prompt length, output length and concurrent requests. Measure throughput and latency together at that load.

For training, record batch size, sequence length, optimizer and checkpoint interval. These inputs determine the working set and communication pattern; the GPU model comes after them.

A 70-billion-parameter model at two bytes per parameter requires 140 GB for weights. At one byte it requires 70 GB; at four bits, 35 GB. These are decimal estimates before quantization metadata, KV cache, activations and runtime buffers.

Measure peak memory at the intended context and concurrency. Training adds gradients and optimizer states, depending on the training method.

ILION operates GB300 clusters in Canada. NVIDIA’s GB300 NVL72 reference is a rack-scale platform with 72 GPUs and 36 Arm-based Grace CPUs. Those are manufacturer architecture figures, not ILION inventory totals.

Supermicro SYS-822GS-NB3RT uses an eight-GPU HGX B300 platform. Qualify the host architecture, software packages and GPU-to-NIC topology for the selected system.

H200 provides a 141 GB HBM3e reference; RTX PRO 6000 Blackwell Server Edition provides 96 GB GDDR7 ECC. Bandwidth, interconnect, graphics support and software licensing differ. Tokyo allocations identify their exact RTX PRO 6000 board configuration in the project specification.

Multiple GPUs require a compatible sharding or parallelism strategy; adding nominal memory capacities alone does not establish application fit.

Keep model revision, precision, input lengths, concurrency and software versions identical when comparing platforms. Save time to first token, inter-token latency, total throughput and peak memory from the same run.

For training, retain step time and checkpoint/restart results. The final selection records the configuration, observed results and agreed acceptance limits.