Exactly this! I have been using this for a long time - I'm on my second one and looking into a third (unless my husband carries out his threat to buy me that new phone): http://www.senacases.com/apple/iphone-4s-cases/walletskin/#Yeah, what is so nice is I REALLY do not need my purse as long as I have my phone.
Inside the wallet I can keep my main credit cards and DL; I keep a few bills in the fold; the clip on the back (which is removable) acts as a stand or to hood into my cleavage to hold the phone and leave my hands free. When I need my keys with me, I clip them onto the wallet too.