How does it damage the PN junction of the panel is open circuit or barely loaded? It doesn’t seem logical that this would damage the panel, but I’m open to being proven wrong.
There are all kinds of follow up questions to ask as well, but I think the main one is how big an effect are we talking?
Not a huge effect now with smart systems but if you leave solar panel disconnected from everything and out in the sun for weeks at a time you will damage the panel. Open circuit voltage is higher than operating voltage and higher voltage will break down insulation. PN depends on the insulating properties of a doped layer. If I remember correctly electron tunneling causes damage by making the band gap smaller