Focusing to a point is usually fine (however if I move the laser too quickly it falls out of focus), but when I focus to infinity it is fine but if I move the laser quickly, it usually comes out of focus and the dot is much larger. I am relatively certain that my eyes aren't being tricked as if I move the laser slower, it stays in focus. Do I need to adjust the laser so that when the knob is all the way in, it is at infinity?
These are very cheap pointers that have no spring or other device to hold the focus knob in place. It is the nature of a <$10.00 laser pointer. I never really tried to fix this as it is common among all of them.
Those housings that have wobbly tops are annoying, I just tighten them all the way down snug, then reach in the front with the right size screwdriver and adjust the lens for infinity and leave it set there.