We present a reliable and original experimental technique based on the analysis of beam self-trapping to measure ultrafast optical nonlinearities in planar waveguides. The technique is applied to the characterization of Ge–Sb–Se chalcogenide films that allow Kerr induced self-focusing and soliton formation. Linear and nonlinear optical constants of three different chalcogenide waveguides are studied at 1200 and 1550 nm in femtosecond regime. Waveguide propagation loss and two photon absorption coefficients are determined by transmission analysis. Beam broadening and narrowing results are compared with simulations of the nonlinear Schrödinger equation solved by BPM method to deduce the Kerr n2 coefficients. Kerr optical nonlinearities obtained by our original technique compare favorably with the values obtained by Z-scan technique. Nonlinear refractive index as high as (69±11)×10−18m2∕W is measured in Ge12.5Sb25Se62.5 at 1200 nm with low nonlinear absorption and low propagation losses which reveals the great characteristics of our waveguides for ultrafast all optical switching and integrated photonic devices. © 2017 Elsevier B.V.