I'm basically a Troglodyte in survival and most of my bases are underground. Currently in a medeval style mansion, but that's an exception. Once built a base in side of a ravine with lots of balconies. Most important lack is you either have to have outside space for a lot of gardening and other stuff like Iron and XP farms or keep them smaller and use a lot of lights. Advantage is you don't mess up the scenery as much although some like an urbanized setting
Biggest advantages are it's easier to arrange inside of rooms only without worrying about outside, and you can spread out anyway you want in all 3 dimensions without worrying about overall looks or plan.
A really easy and fun thing to build in a cave base is a mob trap described in the following video by paulsoaresjr. The mobs are carried by a strategically placed stream of water into a hole in the base's ceiling and burn to death on a campfire. paulsoaresjr even built a tripwire doorbell (I think in a subsequent video) so that when a mob was being carried along in the stream a doorbell rang to announce the "visitor". It really cracked me up.
If you decide to do this, don't make the mistakes I did. Make sure that the mobs you want to catch are far enough away from your base's living and working quarters so you won't hear them and let them be swept through a long tunnel into the trap.
When I discovered a village on the side of a hollowed-out mountain, I decided to take advantage of the natural cave to build one of these traps, but I didn't do it right. I built a huge hall with storage and sleeping quarters for my villagers, and simply left the rest of the cave in the dark above the 4-block high ceiling I built. In this huge area I added a 8-block long trough that would sweep mobs into the ceiling hole if they stepped in the water. This was a very faulty design. First of all, mobs can wander freely in the dark space above my ceiling, meaning that the mob noises and groans are constant and unbearable. Second, the space where they can wander is huge and very seldom does a mob happen to step into the water and get swept into the trap. When one does, it is very satisfying, though! The only good thing I did: Instead of removing the loot from a chest or hopper below the trap, I have a hopper going off to the side leading to a chest in a separate loot storage room. Some day I will re-visit this design and change it so that I don't hear mobs moaning and groaning all the time.