It's often beneficial to experiment with the intensity and photon count sliders in the Caustics box of the Rendering tab. The default settings aren't necessarily the best for verisimilitude. 

 I wish I could figure out how to completely eliminate flicker of caustics during animations. Perhaps it can't be done, but I suspect it's a matter of finding the proper settings (and enduring slower render times.)