Good question, actually, I find myself having these troubles as well. Even for usual style, not super-deformed, I am unable to "project" anime character looks at imaginable 3D girl, and for some reason without that I can't fully immerse myself into thinking about any interaction. But then, my imagination suck anyway.
Does this need for ability to project 2D onto 3D in order to feel makes me inferior? Maybe the reason is that I am trying to project 2D image on some generic japanese girl like cosplayer, and that is doomed to fail, they just don't share attractiveness.