I generally don’t, except for wargames models — those get a coat of varnish to help them stand up to handling.
Display models, though, I only vanish if I need to for the effect I’m after. For example, if I’ve used gloss paint for some reason, but the model needs to be matt. Otherwise, I don’t really see the use of it, and actually, feel that varnish is one of those things that can too easily go wrong.