Even if you can calculate the treshold at all, which one would you call the 'actual' damage?
Its similar to 'how poisonous is that substance?', and 'what is a lethal dose?'.
With poisons, the lethal dose is often stated as the LD50, which is the dose that killed 50% of test subjects in research. Ingesting this dose isn't a guarantee you will die from it though, its the point where the odds are 50-50. With chemicals other doses are sometimes determined as well, such as LD10 or LD90.
Getting back to the lasers: the '5 mW into the eye' gives a small, but accetable, chance of eye damage. The 'safe limit' where no damage occurs is much lower, in the order of 0.5 to 1 mW into the eye for visible wavelengths. I'm not sure about the exact definition of safe here, but its something like 'less than 1% chance of damage during 15 minute exposure'.
On the other side of things, i don't think that any power level has been defined that is guaranteed to cause eye damage - and i hope no one here will figure that number out the hard way!