The crystal structure of a solid largely dictates its electronic, optical and mechanical properties. Indeed, much of the exploration of quantum materials in recent years including the discovery of new phases and phenomena in correlated, topological and two-dimensional materials—has been based on the ability to rationally control crystal structures through materials synthesis, strain engineering or heterostructuring of van der Waals bonded materials. These static approaches, while enormously powerful, are limited by thermodynamic and elastic constraints. An emerging avenue of study has focused on extending such structural control to the dynamical regime by using resonant laser pulses to drive vibrational modes in a crystal. This paradigm of ‘nonlinear phononics’ provides a basis for rationally designing the structure and symmetry of crystals with light, allowing for the manipulation of functional properties at high speed and, in many instances, beyond what may be possible in equilibrium. Here we provide an overview of the developments in this field, discussing the theory, applications and future prospects of optical crystal structure engineering.