I still strongly suspect that his feelings of affection towards Kaoru are not the main reason he is with her. He likes her, he's known her forever, but I think he married her and stays with her out of some kind of guilt. Time will tell if I'm right on this, but it seems pretty likely he doesn't love her the way she loves him, or probably the way he loves Risako.
I think you make a bunch of really good points that I've never really thought about before. Maybe he really is staying with her out of guilt- that's a possibility that I think isn't much of a stretch. Reminds me of the Commander in The Handmaid's Tale- he was kind to Offred only because he felt guilty about the previous handmaid being so miserable and killing herself. Maybe it's something like that? Or even something else, like pity; or maybe they just didn't think their marriage through very well.
Whatever it is, I don't think their relationship is based on just love. And even if it is, a very watered down version of it.
Well, guess we just have to wait until more chapters to find out more, won't we. I'm sure it'll be well worth the wait though. Great series.