You can never have too much! My altar has a full layer of 15x15 mixed crystal clusters 1 layer underneath my altar. And instability becomes essentially a non-issue at that point. The dangerous ones might do something once in awhile, but generally never a problem.
*edit: Loaded up world to get actual dimensions and screenshot