It's important to differentiate between types of contamination here.
Things like dust can be removed mechanically without too much effort, and things like soft brushes, lens pens and compressed air work very well for those. It should be the first thing you attempt.
Other contaminations are simply 'stuck' to the lens: think of splats of tap/rain water that have dried up, greasy fingerprints, tar deposits from pollution or smoke and such. These will not budge if you blow compressed air at them or even try to swipe them away with a mild brush.
At that point you need to dissolve them and them wipe the surface clear with a cloth of some kind. For many contaminants a 50/50 mix of isopropanol and distilled water is a good solution: it will readily dissolve greasy residues but not damage coatings or plastic components if applied briefly. Benefit is that any solvent left behind will evaporate without leaving a trace.
Detergents have the downside of leaving traces of their own once they dry out - soap doesnt evaporate despite the water you mixed it with will.