Because to me, they seem like de facto "Agree and “Disagree” buttons, whether or not it was the intent.
Not quite. Upvote means “I agree I like butt licking” and downvote means “I agree I like butt sex.” Easy to confuse the two.
Now, just to be clear, does this apply to both the upvoter/downvoter and the comment being left or just the upvoter/downvoter alone?