Japan isn't that huge. It's 2 hours by train between Tokyo and Kyoto. It's not the US.
http://i.imgur.com/cjkvtmd.png
I dunno how to embed images here but you're fairly off. Japan may not be as big as the US but it's still very large. Traveling from the Canadian border to the Mexican border is approximately the same distance. I live about 2/3rds that distance from Los Angeles and it took my family about 16 hours to drive there by car without taking a break past stopping to use the restroom or get food.
"Too far away" isn't really the issue. It's just that it takes extra effort to maintain the relationship.
But that is clearly at least part of the issue here. Since Takemiya didn't bother actually going in-depth into anything and just generalizing it instead we can't say what their dreams were or anything or how much of a factor they were in comparison to the distance which was clearly a factor. Long distance relationships are not for everyone. It's not a matter of effort. Some people simply cannot do it. On top of that it's a different culture. You cannot compare it to western culture where long distance relationships are a lot more common.
A lot of manga are based on this : once you graduate, once you change job, once you quit a club, if nothing connects you, then all is left are the feelings. And if they are not strong enough, you drift apart. They even willingly put an end to things so as to no be "held back" by these feelings.
Which as someone pointed out earlier is part of Japanese culture... Again: Different cultures. Trying to judge it by western standards is not fair.