Abstract: Big deviation will be generated by using frame-differential optical flow method on target detection. In order to solve the problem, this paper proposes a spatio-temporal interest point target detection algorithm. The authors extract optical flow from a pedestrian target, apply the method based on spatio-temporal interest point first, and for image sequences Gauss filter and Gabor filter are used to extract interest points set, then the moving objects are extracted from the interest point set, and the optical flow is calculated in a certain area so that the targets are identified and tracked. The experimental results show that the accuracy of this method is higher than that of traditional frame differential optical flow one and the method is relatively simple and effective for actual scenes.