Normally people that are not that familiar would use the surname to refer to each other, and that is especially true for Japan (and maybe some others, but idk).
In my country though, you will get weird faces if you call anyone with the family name. You probably know why.