Too hot can have bad effects on everything. First off is the diode you can kill. High temp can also screw with just about everything else from the crystals to the driver.
You laser may cease to work at all in low temp, but its unlikely to cause permanent damage (IE, bring it back to room temp and it works fine). Only problem I see with very low temps are:
1. condensation getting inside the laser/optics if the laser is itself cold, but in a warmer enviroment
2. thermal expansion if you move it from cold to warmer temperatures quickly, which could cause problems as major as cracking an optic, or cause more subtle damage to the internals.
Most lasers' specifications should state a temperature range for operating and storage. The operating range is usually much more narrow than the storage range, but the actual values vary by laser. However, as Pseudo pointed out a quick change in temperature, even within the stated range, can have negative consequences.
Pointers have a MUCH higher chance of damage than a Lab module. Most lab modules are tmperature controlled. I know on my 400mw 457nm melles griot, when we put it in the bottom of the ocean, the water temp is right around 1-3C, almost freezing. It takes the module a while to warm up, 10-15 mins. Then it's good to go.