Well, the H.P. fandom stands in the center not Rowling or her books. I think one should differentiate with that.
Also: yes, the books contain antisemitic stereotypes, but Rowling herself hasn’t said herself anything antisemitic to my knowledge. Some of her statemets are transphobic, but the Fediverse has enough room to carry out this conflict.
We are not in centralized social media anymore, where you have to be on the same site as everybody else. If you are on the same instance as a transphobe and you oppose that: change instances. If you don’t want to be on an instance that federates with any transphobe content: change to an instance that de-federated with any of those instances. That’s all possible in the Fediverse.