Why is it that Japan has a population of over 120 million, making it the most populous country among all developed countries apart from the U.S., with a larger population than the Philippines and Vietnam and about twice the population of the UK, but Japanese people still come across as a rarity?

Why is it that Japan has a population of over 120 million, making it the most populous country among all developed countries apart from the U.S., with a larger population than the Philippines and Vietnam and about twice the population of the UK, but Japanese people still come across as a rarity?