The mining is also usually a really polluting affair for the region, much more than the what power generation might suggest. And overall, in many countries there is a lot of subsidies going on for hidden costs, especially relating to the waste and initial construction. So it is not as cheap as a first look might suggest.
I’m not against it per se, it is better than fossil fuels, which simply is the more urgent matter, but it’s never been the wonder technology it has been touted as ever since it first appeared.
One thing to remember about the mining issue is that coal mining is just as bad, and coal is often radioactive as well. More people have died from radiation poisoning due to coal power/mining than have died from radiation poisoning due to nuclear power, even when you include disasters like Three Mile Island and Chernobyl.
Of course, we’ve also been mining and using coal a lot longer, but the radioactive coal dust and possibly radioactive particles in the smoke from coal plants is something that many people are unaware of.
But, like you said, the big thing is to move away from fossil fuels entirely, and nuclear power has its own issues. It doesn’t so much matter what we go with so long as we do actually go with something, and renewables are getting better and better all the time.