The gravitational wave (GW) spectrum at frequencies above a kHz is a largely unexplored frontier. We show that detectors with sensitivity to single-phonon excitations in crystal targets can search for GWs with frequencies, , corresponding to the range of optical phonon energies, . Such detectors are already being built to search for light dark matter (DM), and therefore sensitivity to high-frequency GWs will be achieved as a byproduct. We begin by deriving the absorption rate of a general GW signal into single phonons. We then focus on carefully defining the detector sensitivity to monochromatic and chirp signals, and compute the detector sensitivity for many proposed light DM detection targets. The detector sensitivity is then compared to the signal strength of candidate high-frequency GW sources, e.g., superradiant annihilation and black hole inspiral, as well as other recent detector proposals in the frequency range. With a judicious choice of target materials, a collection of detectors could optimistically achieve sensitivities to monochromatic signals with over .