Probably better to post in the github issue rather than replying here.
You can have your client talking to all the servers and grabbing votes for whatever you’re subscribed to, and losing votes for anything you’re not subscribed to. It works basically exactly that way for one-user instances already.
It works like that for servers because servers are assumed to have high uptime, so (in theory) push-based communication should be enough. However, we see that this is not true even for servers (e.g, medium-sized instances getting out of sync with LW because they can not keep up with all the data being sent to them) and this will be specially true in the case of a network with tens/hundreds of thousands of separate clients. No server will be willing to push activities to all those inboxes, so we will need to have some pull-based form of communication as well.
Oh, yeah, at that point it’ll be a scalability clusterfuck. No idea what the solution is. Maybe something with persistent caches run by third parties or something? That actually would be fine, since all the actions are signed with the private key of the actor, I think.
ActivityPub is not to me a real great designed protocol but it’s whatever. Usually the key part for social networks is the “social” part of it; the protocol or the web site can be pure shite and if people like interacting with the other people there then it’s fine. But yes, you are correct that beyond a certain point of scalability there are some dragons lurking that don’t have obvious weak spots.
The problem is not with ActivityPub, but the implementations. No one ever claimed that it should be only a push-based system, but it seems that everyone working on AP software can only think in terms of server-to-server interactions to get the data and then reinvent the wheel by developing their ad-hoc API.
AP is fine if we treat it as a messaging protocol and use it to power offline-first applications. The devices do not need to have all the network’s data, just the one that the user has actively interacted with.