GPU (NVIDIA)

Lineage of all NVIDIA’s GPU generations named after scientists, mathematicians, engineers

Architecture	Approx. Release	Named After	Representative GPU / Platform	Specs, brief	Notes
Tesla	2006	Nikola Tesla	Tesla C870 / G80	Early CUDA GPU; GDDR3-class memory; FP32 CUDA compute	CUDA/unified shader era begins; general-purpose GPU compute becomes practical.
Fermi	2010	Enrico Fermi	Tesla M2090	6GB GDDR5, ~178GB/s; ~1.33 TFLOPS FP32, ~665 GFLOPS FP64	ECC, cache hierarchy, stronger FP64, serious HPC compute.
Kepler	2012	Johannes Kepler	Tesla K80	24GB GDDR5 board, 480GB/s; ~5.6 TFLOPS FP32 board	Hyper-Q, dynamic parallelism; K80 was dual-GPU.
Maxwell	2014	James Clerk Maxwell	Tesla M40	24GB GDDR5, 288GB/s; ~7 TFLOPS FP32	Efficiency-focused architecture; common before Tensor Core GPUs.
Pascal	2016	Blaise Pascal	Tesla P100	16GB HBM2, ~732GB/s; ~10 TFLOPS FP32, ~21 TFLOPS FP16	HBM2 and NVLink; strong FP16, but no Tensor Cores yet.
Volta	2017	Alessandro Volta	Tesla V100	16/32GB HBM2, ~900GB/s; ~125–130 TFLOPS Tensor mixed precision	First Tensor Cores; V100 became foundational for AI training.
Turing	2018	Alan Turing	NVIDIA T4	16GB GDDR6, ~320GB/s; 65 TFLOPS FP16, 130 INT8 TOPS	RT Cores + Tensor Cores; very important inference GPU.
Ampere	2020	André-Marie Ampère	A100 80GB	80GB HBM2e, ~2TB/s; up to 624 TFLOPS FP16/BF16 sparse	TF32, MIG, structured sparsity, third-gen Tensor Cores.
Ada Lovelace	2022	Ada Lovelace	RTX 4090 / RTX Ada	24GB GDDR6X-class flagship; ~1TB/s-class bandwidth; strong FP32/RT/Tensor throughput	Client/workstation graphics + AI; DLSS 3, stronger RT/Tensor Cores.
Ada Lovelace	2022	Ada Lovelace	L40S	48GB GDDR6 with ECC, 864GB/s; 91.6 TFLOPS FP32, 362.05 TFLOPS FP16/BF16 Tensor, 733 TOPS INT8 Tensor	Data-center universal GPU for inference
Hopper	2022	Grace Hopper	H100 SXM	80GB HBM3, 3.35TB/s; ~1,979 TFLOPS FP16/BF16 sparse, ~3,958 TFLOPS FP8 sparse	FP8 Transformer Engine, TMA, WGMMA, fourth-gen Tensor Cores. ([NVIDIA][3])
Hopper	2023	Grace Hopper	H200 SXM / PCIe	141GB HBM3e, 4.8TB/s; FP8 Tensor Core performance ~4 PFLOPS	Same Hopper generation; H200 is primarily the memory-capacity/bandwidth upgrade over H100.
Blackwell	2024	David Blackwell	B200 / GB200	DGX B200: 1.44TB HBM3e total, 64TB/s; 72 PFLOPS FP8, 144 PFLOPS FP4 system-level	Fifth-gen Tensor Cores, FP4/NVFP4, second-gen Transformer Engine, large NVLink scale-up.
Blackwell	2025	David Blackwell	RTX PRO 6000 Blackwell Server	96GB GDDR7, 1,597GB/s; 120 TFLOPS FP32, 2 PFLOPS FP8, 4 PFLOPS FP4	Blackwell also spans professional RTX/server GPUs, not just B200/GB200.
Blackwell Ultra	2025/2026	David Blackwell	B300 / GB300	DGX B300: 8× B300, 8×288GB = 2.3TB total; 72 PFLOPS FP8 training, 144 PFLOPS FP4 inference	Higher-memory Blackwell refresh for reasoning/inference; NVIDIA also describes 288GB HBM3e per GPU.
Vera Rubin	2026 platform	Vera Rubin	Vera Rubin NVL72	NVL72: 20.7TB HBM4, 1,580TB/s rack memory bandwidth; per GPU options shown as 576GB/44TB/s and 288GB/22TB/s	Rubin GPUs + Vera CPUs, NVLink 6, ConnectX-9, BlueField-4, Spectrum-6; NVIDIA also references Groq 3 LPX/LPU integration in the platform.
Feynman	Future roadmap / 2028 target	Richard Feynman	Feynman + Rosa CPU platform	Public roadmap only; final memory/FLOPS not productized	Next major architecture beyond Rubin.

GPU to GPU inference

How 152,384 and 0.0003456 are represented in different GPU number formats

Format	Total Bits	Exponent Bits	Bias	Mantissa Bits	Approx Precision	152,384	0.0003456
FP32	32	8	127	23	~7.2 decimal digits	152,384, exact integer	0.0003456000122241676
FP16	16	5	15	10	~3.3 decimal digits	+inf, overflow; max finite 65,504	0.0003457069396972656
BF16	16	8	127	7	~2.4 decimal digits	152,576	0.0003452301025390625
FP8 E4M3	8	4	7	3	~1.2 decimal digits	out of range; max finite ±448	0.0, underflows
FP8 E5M2	8	5	15	2	~0.9 decimal digits	out of range; max finite ±57,344	0.0003662109375
Raw INT8	8	none	none	none	integer only	overflow if raw signed INT8	0 if directly cast/truncated