Solid-state white light sources gain increasing interest due to their advanced characteristics compared to conventional lighting solutions. New design challenges are introduced in the remote phosphor set-up by the substitution of the efficiency-droop-limited LEDs with laser diodes (LDs) that exhibit peak efficiencies at much higher operating currents. Although laser-excited remote phosphor (LRP) systems have already been employed in some commercial applications, the bottleneck in their performance is identified in the down-conversion process within the phosphor material. The high intensity exciting laser beam in combination with the temperature-dependent properties of phosphors can lead to thermally induced instabilities in the system. For this reason, an opto-thermal simulation framework is developed to investigate the optical and thermal interdependencies and derive the LRPS optimization parameters. The optical analysis is performed with commercial ray-tracing software, where the optical heat losses are computed and subsequently used as the volume heat source in thermal analysis implemented by the finite element method (F.E.M.). The question now arises as to how to properly model the phosphor material in such a simulation scheme. The LED experience has produced a variety of phosphors for lighting applications, most commonly powders in some appropriate resin matrix, which are treated simulation wise as bulk diffusers. As the low thermal conductivity of resins is deemed critical for their use in LRPS, recent research focuses on resin free materials such as glass phosphors, single crystals, polycrystalline dense ceramics, etc. The different modeling approaches of such solutions are investigated here as the scattering properties and surface topology of the samples can vary.