Using the nonparaxial scalar diffraction equations to estimate the diffraction efficiency of multiple orders of a sinusoidal reflection grating, we compare the results of transverse electric (TE) and transverse magnetic (TM) vector wave simulations for a reflective surface on glass. Modifications are presented that enable the nonparaxial scalar solution to approximate simulation results in each order to within 1% to 2% across a 0 to 90deg range of incidence angles when the diffracted power is predominantly carried in the first several orders, up to the fourth order. The accuracy of simulations in this regime is particularly important for surface scattering where individual spatial frequencies typically have amplitudes much less than a hundred nanometers. A substantial reduction in the error between vector simulations and scalar estimation of diffraction efficiency is demonstrated with the following modifications: (1) an obliquity factor correction for the specular order based on a first-order power conservation criterion, (2) a cutoff factor modifying the obliquity factor as the diffracted angle approaches 90deg for each order, and (3) nonuniform unallocated power distribution near the low-order cutoffs. The second and third modifications are applied to higher diffraction orders for extending the range of applicable grating heights up to 200nm (peak-to-valley), periods down to twice the wavelength, and slopes up to 0.2 (root mean square).
