Edoyu is a great spot for people looking for an onsen experience without leaving Tokyo. Though on the pricey side, Edoyu offers a variety of baths and a variety of other services (massage, etc). Most unusual, it is open all night -- offering those willing to forgo a hotel a very reasonable overnight option.
Getting to Edoyu is both simple and complicated -- simple because it's just steps away from the A3 Exit of Ryougoku Metro station. Complex because the online maps make it appear directly across the street when really it's several buildings back, not on Kiyosumi-dori but on Hokusai-dori. Google Maps has it listed as 両国湯屋 江戸遊. Our OnsenJapan.net icon shows the exact entrance of the building (zoom down to the highest level and use "Map" not "Terrain" setting.)
Please note: As in many Japanese onsens and spas, people with TATTOOS of any kind (even stick-on or temporary!) are strictly forbidden to enter.