We stayed in District 1 and everything was in walking distance from our hostel (within a 2-4km radius). Grab yourself a good map, a bottle of water, maybe even a fan and away you go.
Taxis are readily available as well, or if you are brave enough you can even hire motorbikes. But please be careful. The roads in HCMC are some of the craziest and busiest we have seen.