But as soon as you interact with literally anyone (or anyone interacts with you) your data is still replicated on other servers.
No personally identifiable information or private account information is transmitted between instances. The only thing that is synced is the content of your posts, reports and up- and downvotes. And all of that serves a purpose and is shared willingly.
Your posts are all public and discoverable by web crawlers even if your instance didn’t federate at all. That’s kind of the point of activityPUB
ActivityPub does not have support for private messages. Services that try to shoehorn a direct message as “private” are misusing the protocol. You could configure client-side encryption but thats outside the scope of ActivityPub