This is pretty much where harems all fall apart: what reason do these girls actually have for being attracted to essentially a piece of cardboard?
If you're looking for realism in entertainment, you're going to have a bad time.
Not so much realism as suspension of disbelief... and I too have a hard time suspending that when the MC has all the personality of a wet noodle. I suppose they do this so the male watchers can insert themselves into the MC. But at the same time those of us who aren't trying to do self-insertion it makes the character look boring and flat.
I'm a guy and even I'm getting, actually I've been tired for a few years already, of the Beta-to-the-max type MCs in shounen/harem/(insert anything) manga/LN scene. The ones where the male MC actually kick ass is so far in between, although there's been a surge this last 2 years it's still not enough to counter balance the weak MC galore.
It didn't used to be this way... I blame it all on you Shinji, you ruined everything.