© 2019 IEEE. Traditional stochastic optimal control methods that attempt to obtain an optimal feedback policy for nonlinear systems are computationally intractable. In this letter, we derive a decoupling principle between the open-loop plan, and the closed-loop feedback gains, which leads to a deterministic perturbation feedback control based solution to fully observable stochastic optimal control problems, that is near-optimal. Extensive numerical simulations validate the theory, revealing a wide range of applicability, coping with medium levels of noise. The performance is compared against a set of baselines in several difficult robotic planning and control examples that show near identical performance to nonlinear model predictive control while requiring much lesser computational effort.