We recently found that crystallization of monodisperse hard spheres from the bulk fluid faces a much higher free energy barrier in four than in three dimensions at equivalent supersaturation, due to the increased geometrical frustration between the simplex-based fluid order and the crystal [J.A. van Meel, D. Frenkel, and P. Charbonneau, Phys. Rev. E 79, 030201(R) (2009)]. Here, we analyze the microscopic contributions to the fluid-crystal interfacial free energy to understand how the barrier to crystallization changes with dimension. We find the barrier to grow with dimension and we identify the role of polydispersity in preventing crystal formation. The increased fluid stability allows us to study the jamming behavior in four, five, and six dimensions and compare our observations with two recent theories [C. Song, P. Wang, and H. A. Makse, Nature 453, 629 (2008); G. Parisi and F. Zamponi, Rev. Mod. Phys, in press (2009)].