Normal bikes that you just push aren’t that stable without a rider, but you can get it some distance. They still fall over rather quickly. That’s mostly the form of the handlebars like gnu commented. And yes, without a rider, the gyroscopic effect is relevant. A bike weighs let’s say 15 kg, and a rider is commonly like 75kg. Of course removing like 80% of the weight changes if the gyroscopic has a meaningful influence. Add the rider back, and it becomes negligible again.

This is of course even more pronounced if you push only a wheel with nothing else, then there’s nothing left but momentum and the gyroscopic effect.

The reason you lean into a turn is exclusively the centrifugal force (not sure that’s the right twin), if you don’t you fall over because you have nothing to turn against. Changing direction needs something to push against.