We examine the response of a dense colloidal suspension to a local force applied by a small magnetic bead. For small forces, we find a linear relationship between the force and the displacement, suggesting the medium is elastic, even though our colloidal samples macroscopically behave as fluids. We interpret this as a measure of the strength of colloidal caging, reflecting the proximity of the samples' volume fractions to the colloidal glass transition. The strain field of the colloidal particles surrounding the magnetic probe appears similar to that of an isotropic homogeneous elastic medium. When the applied force is removed, the strain relaxes as a stretched exponential in time. We introduce a model that suggests this behavior is due to the diffusive relaxation of strain in the colloidal sample.