Dark state in a nonlinear optomechanical system with quadratic coupling*

We consider a hybrid system consisting of a cavity optomechanical device with nonlinear quadratic radiation pressure coupled to an atomic ensemble. By considering the collective excitation, we show that this system supports nontrivial, nonlinear dark states. The coupling strength can be tuned via the lasers that ensure the population transfer adiabatically between the mechanical modes and the collective atomic excitations in a controlled way. In addition, we show how to detect the dark-state resonance by calculating the single-photon spectrum of the output fields and the transmission of the probe beam based on two-phonon optomechanically induced transparency. Possible application and extension of the dark states are also discussed.