Making the cavity mirrors is just a practical hurdle, and strictly not even required to make a laser: supperradiant lasers work perfectly well without any cavity mirrors.
The problem is in the laSEr - it requires stimulated emission, and that is very difficult to achieve using current x-ray or gamma sources. The problem is you would need to have a nuclear reaction that is somehow stimulated by the same energy it produces. This is very diffrent from the typical energy generating fission reactions, where a slow neutron results in delayed emission of several photons and a couple of fast neutrons. For something like this to work, the exciting particle would have to be a photon, and the resulting output a photon of equal energy and phase.