Optically levitated nanoparticles are ideal experimental testbeds for investigating macroscopic superpositions and microscopic thermodynamics. Integrating such levitated nanoparticles with photonic structures can enable strong coupling between their center-of-mass motion and guided photonic modes, facilitating enhanced control and probing of their motion. When coupling a particle to a photonic structure, such as a waveguide, the effects of fluctuations become prominent at nanoscales. In this work, we analyze the classical and quantized center-of-mass motion of a polarizable particle interacting with the fluctuations of the electromagnetic field in the presence of a medium. We derive a position localization master equation for the particle’s quantized center of mass, and examine its classical center-of-mass momentum diffusion, elucidating correspondences between classical and quantum Brownian motion of polarizable particles near media. We study the decoherence rate of the particle in the presence of a planar surface as a function of temperature and distance from the surface, comparing it to common sources of decoherence. Our results are pertinent to experiments aimed at preparing levitated nanospheres in macroscopic quantum states and investigating their Brownian dynamics.