I totally agree. The only time I mention to someone that they are larger is when I am trying to squeeze them into an XL condom lol. If they are not above average, I wouldn't tell them they are. If their skin is particularly smooth and silky or nicely shaped, etc, I would mention it. I am not an overly effusive person; I limit my compliments to real compliments and do not overembellish as I too do not want to hear fakery.
If more than one person is saying the same thing, take it as a compliment and as true, too. If I meet up with someone with little experience, not used to condoms, etc, and they are well above average in size, part of why I would mention something is to let them know that when they buy condoms for themselves, to make sure they get large sizes, to make sure any sps they visit have larger sizes too.