We show that a simple non-linear transformation of the Adaboost score multiplied with the illumination compensated likelihood leads to a fast robust tracking paradigm. We demonstrate the ability of our method to detect occlusions at the same time ensuring that mis-assignments between the occluder and the occluded does not occur. We present experimental results of our method on low resolution surveillance indoor and outdoor videos using an off the shelf DSP. We also demonstrate the power of the parametric illumination model for pose constrained face recognition when matching across known illumination conditions.