In this paper, a fast head pose tracking system is introduced. It uses iterative closest point algorithm to register a dense face template to depth data captured by Kinect. It can achieve 33fps processing speed without specific optimization. To improve tracking robustness, head movement prediction is applied. We propose a novel scheme that can train several simple predictors together, enhancing the overall prediction accuracy. Experimental results confirm its effectiveness for head movement prediction.