If you ever wanted to go blind, by all means take those awesome goggles off and let the optic destruction begin!! That is a joke.
Really if you are pointing it at the sky or a good distance away ( like a gun, always know what you are aiming at and what's around and behind that), you don't need them on. I regularly aim mine up in to the clouds on stormy nights, from my bathroom window. I only do it when I know planes will not be flying. Otherwise I use my fog machine in the back yard and aim towards my house, it's dark colored an absorbs most of the light.
I always wear mine in the house, or if there is a chance of reflection. When playing with anything over 5mw, I do it in a locked room, so no one can walk in and lose an eye. I also have all the windows covered so no stray beam can find it's way outside. I play with mirros and lenses in my laser room. I had a 450nm bouncing off 100 1/2 round mirrors one time. Took me almost 4 hours to get it set up. It was insane!!! Was like a movie!!