It can be affected my many things so there is no one "right" way to save files. However, it's always a good thing to delete geometry and shaders that are no longer in your scene prior to saving. (Edit > Delete Unused Shader/Geometry)

Features like hair, saving texture files internally, complex replicators and more will extend time.