It’s depends purely on how it’s used. Used blindly, and yes, it would be a serious issue. It should also not be used as a replacement for doctors.
However, if they could routinely put symptoms into an AI, and have it flag potential conditions, that would be powerful. The doctor would still be needed to sanity check the results and implement things. If it caught rare conditions or early signs of serious ones, that would be a big deal.
AI excels at pattern matching. Letting doctors use it to do that efficiently, to work beyond there current knowledge base is quite a positive use of AI.