The physical effects reducing the damage threshold of dielectric films when exposed to multiple femtosecond pulses are investigated. The measured temperature increase of a Ta2O5 film scales exponentially with the pulse fluence. A polarized luminescence signal is observed that depends quadratically on the pulse fluence and is attributed to two-photon excitation of self-trapped excitons that form after band-to-band excitation. The damage fluence decreases with increasing pulse number, but is independent of the repetition rate from 1 Hz to 1 kHz at a constant pulse number. The repetition rate dependence of the breakdown threshold is also measured for TiO2,HfO2,Al2O3, and SiO2 films. A theoretical model is presented that explains these findings.