Terrain generation for caves looks very much like it's been griefed when the cave opens up on the edge of a river or side of the hill. The resulting overhang is ugly, and the terrain disruption looks like someone set off 500 dynamic to the place. It's visually and aesthetically jarring, but it also makes it difficult to built. Recommend covering most caves or making their entrances inbetween hills.