Are failed HG's really all that common though...?

That is one indication. But, if you just moving your car a few feet, and let the
engine start, and turn the car off before it finished, you might get a knock from unburnt
fuel. If the engine start, and you didn't want it too, turn the car quickly.
If you can't turn it off, let it run until it stops.
Sometime if it's cold outside, you might get a knock after the car has sat overnight.
Because of the cold/hot air causing condensation in the intake manifold.

Usually you'll get the knocking when the engine has stopped and is cooling off.
The leaking head gasket will allow coolant too seep inside the cylinders for the next
time the engine starts. But you will also notice the coolant level slowly going down,
depending on how damaged the head gasket is.

 

Good thinking to ask. There are also other causes of knocking on start up besides the head gasket.

Which means you're in sound logical territory if you say "failed head gasket" ⇒ "probably going to knock on start up" ...

but not sound to say "knocks on start up" ⇒ "it's the head gasket", without further confirmation.