“You all” or “you both” or “you”or “you guys” seems less icky, like they don’t view you as a couple saying “you two”, like you are up to no good.
Like most of English, it’s very situational and not automatically rude. It’s mostly rude when being declarative (with no authority) and/or dismissive.