I did this on one of my "poorly focused" green keychain pointers recently.
If there is glue, it shouldn't be that hard to unscrew. Just make sure you don't scratch the lens or the coating on the lens.
And, if the lens becomes "loose" after you remove the glue, you can use teflon (pipe-thread) tape to create tighter threads so it's hard to "knock out" of focus.