You speak about displacement, I suppose that it is in the texture room… have you tested displacement with the brushes ?, they are very precise.
You could also try to apply the texture of a wall, to see what is the result (the rear wall looks good)…