Nope; no soap, no cleanser, no nothing. I just massage in a quarter-sized amount of regular extra virgin olive oil, let it sit for a minute, and rinse it off in the shower. The idea (if I understand correctly) is that it dissolves the dried oils and dirt, without overdrying or removing the sebum your skin needs to be healthy, which is what most soaps and oil-free cleansers do.
Google "oil cleansing method". There are a few different versions, some of which involve mixing in castor oil, exfoliating with baking soda beforehand, etc. I just go for the quick and simple way, and it works like a charm. Also makes it much easier to shave, which is important for me since I have a fairly dark, coarse beard.
Fair warning, I am not a dermatologist, and this may all turn out to be total bunk. But a lot of people, myself included, seem to have been successful with it.
Might be worth a try. I've never used it myself.