That's probably the worst thing about waifuism as a whole - the knowledge that "if they were real, why would they choose us?" and the probable answer of "they wouldn't".
Above all else, at least for me, this is the most debilitating part of it all. The entire point of waifuism is to admit and accept your romantic, or platonic love for that which doesn't exist - your love for a concept. However, even after recognizing how they are just a concept, the knowledge that said concept would, if given the opportunity, not return the love you've poured into it is truly scarring.
Even in the case of a former husband/wife who's partner has died, they still know and remember that someone loved them in "that way". This gives them, if nothing else in what is probably a bleak and depressing life, security. They know that their husband or wife is still with them for as long as they can remember their love. They know this, and none can take this away.
However, for someone with a waifu, there is, at best, uncertainty, at worst, definitive knowledge. Aside from predicting and fantasizing, you can not be 100% one way or the other that your waifu would hate you. Chances are, they wouldn't outwardly hate you, which is something, at least. But this begs the question - when does it reach the degree in which feelings stop being neutrality and start becoming dislike? What, exactly, differentiates between not caring whatsoever for someone, and being madly in love with them? For a waifu-bro, there's even further confusion on this; "What differentiates between indifference and love, yes, but more importantly, am I doing enough to differentiate it actively? Would she return my gestures?". The issue here, again, is uncertainty in the millions of possible responses. Fortunately for me, my waifu is one who probably outwardly "hates" no one in specific (pic related, though maybe she's the most pissed off by Konata, heh), and I doubt someone with her personality would dislike me intently. If anything, we'd either see each other as faces in the crowd, or as acquaintances - I think we'd at least get along semi-well.
But for those with more, erm, "emotionally potent" or "volatile" waifus, or when their personalities, interests or actions conflict with those of their waifus, the effect of the uncertainty is magnified - there is no longer uncertainty, but straightforward doubt. You often can tell, despite the cliché stating otherwise, how a person "is" generally from your first impression, and after all, wouldn't your waifu be able to do this semi-reliably, as well? Say, if your waifu is very athletic or "petite" and tsundere to begin with, and you tend to resemble a walking meatball on a good day, would she honestly take you seriously? "Possibly", if you're being generous. This is the most depressing because even the concept you've half-created in your head to keep yourself happy with your life has turned on you.
However, this carries a slight bit of advantage. In a "true" relationship, both parties are supposed to respect each other unconditionally - "who is she to tell me to lose weight, it's my body", etc. However, with a waifu, it gives a certain goal to work toward: to be the hasubando she deserves. If your waifu is Emi from Katawa Shoujo, it may be a good idea to take up running/jogging. Aside from taking a step to becoming the ideal hasubando, this also can help you meet people (romantically or otherwise depending on your faith in the waifu system), as it is, after all, a hobby, as any other. If your waifu is an aspiring actress (are there any anime about this? sounds interesting), audition for a few musicals. If your waifu is from K-On!, learn their instrument. It gives a way to improve yourself, your "relationship", your self-image, and it's a good way to meet people. Even if your waifu is Nagato and you start reading everything you can and get soopah-skillz on the computer, someone out there is going to have the same interest, and you will be able to discuss it with them. Bam, acquaintance count +1.
So, in short, really, it's what you make of it. You can turn that knowledge of your waifu's probable opinion of you into a parasite on your self-esteem, or you can do something about it, inspired by your waifu. See, unlike in real life, your waifu will always be there. Most people tend to categorize others between "Friends", "enemies", and "possibly more" subconsciously, and judge others based on that, but your waifu will always be there - in spirit. Unless you, for some reason, simply will your waifu away, she will, if anything, be pleased with how you're bettering yourself and will probably forget how you were before, if you really wouldn't've gotten along well with her before. Your waifu is in your mind and heart, and both are always with you; they're just not going anywhere, and neither is she. But people come and go. People move away. People grow and change. You may never see your best friend again, or maybe you'll move in together. You simply never know. But the one thing you can take comfort in is that she'll always be with you and will appreciate how you're trying to better yourself, if not for you, then for her.
It's all what you make of it. Really.