KLT Tracker for human tracking in CCTV


I am trying to use a KLT tracker for human tracking in a CCTV footage. The people are very close to the CCTV. I noticed that some time people change the orientation of the heads and also the frame rate is slightly slow. I have read from Rodrigues et al. paper Section 3.4 that the:

"This simple procedure (KLT tracking procedure) is extremely robust and can establish matches between head detections where the head HAS NOT BEEN continuously detected continuously detected due to pose variation or partial occlusions due to other members of the crowd".

Paper can be found in this link : <a href="http://www.di.ens.fr/willow/pdfscurrent/rodriguez11b.pdf" rel="nofollow">Rodriguez et al.</a>

1). I understood that the KLT tracker is robust to pose variations and occlusions. Am I right?

I was trying to track one single person in footage till now by using the MATLAB KLT as in :

<a href="http://www.mathworks.com/help/vision/examples/face-detection-and-tracking-using-the-klt-algorithm.html" rel="nofollow">MATLAB KLT</a>

However, the points were not being found after JUST 3 frames.

2). Can someone explain why this is happening or else a better solution to this. Maybe using a particle/Kalman filter should be better?


I do not recommend using a KLT tracker for close CCTV cameras due to the following reasons: 1. CCTV frame rate is typically low, so people change their appearance significantly between frames 2. Since the camera is close to the people, they also change their appearance over time due to perspective effects (e.g. face can be seen when person is far from camera, but as he/she gets closer, only the top of the head is seen). 3. Due to closeness, people also significantly change scale and aspect ratio, which is a challenge for some head detectors.

KLT only works well when the neighborhood of the pixel, including both foreground and background, remains similar. The above properties make this less likely for most pixels. I can only recommend KLT as an additional motion based hint for tracking, as a vector of field of part motions.

Most single person trackers do not adapt well to scale change. I suggest you start with some state of the art tracker, like Struck (C++ code by Sam Hare available <a href="http://www.samhare.net/research/struck" rel="nofollow">here</a>), and modify the search routine to work with scale change.


KLT by itself only works for short-term tracking. The problem is that you lose points because of tracking errors, 3D rotation, occlusion, or objects leaving the field of view. For long-term tracking you need some way of replenishing the points. In the <a href="http://www.mathworks.com/matlabcentral/fileexchange/47105-detect-and-track-multiple-faces" rel="nofollow">multiple face tracking example</a> the new points are acquired by periodically re-detecting the faces.

Your particular case sounds a little strange. You should not be losing all the points after just 3 frames. If this happens than either the object is moving too fast, or your frame rate is too low.


  • Is is possible import template tag with ES2015?
  • R lattice 3d plot: ticks disappear when changing panel border thickness
  • pseudocolors in R
  • How to use XSL:Key to get the following-sibling in XSLT 2.0
  • midpoints returned by barplot function do not actually line up with midpoints of bars
  • In zend, how to print a mysql query properly? [duplicate]
  • Parallel cosine distance using clusterapply in R
  • Remove PNG plot margins
  • Javascript bubble sort of an array with objects
  • Non alpha character arrowlabels on a diagram
  • Confusion with the order of execution when `next` with `unless` in ruby
  • Excel - Formula or Macro to fill a cell based on another cell that links to yet another cell
  • SQLite HAVING comparison error
  • Plot ROC curve and calculate AUC in R at specific cutoff info
  • several hexbin plots on one PDF page in R
  • Embedded h2 database: getting connection but table not found
  • ASP Net Core - Mixing External Identity Provider with Individual User Accounts for Audit Tracking
  • Error: java.util.Arrays$ArrayList cannot be cast to java.util.ArrayList
  • Is there a way to read an Excel file using Dataflow
  • Change color of row programmatically in WatchKit
  • Simple command-line app I/O in Dart
  • Using Python objects in C++
  • Error building MEX-files (MATLAB 2012a) on Kubuntu '/usr/bin/ld: cannot find -lstdc++'
  • How to issue a command that produces infinite output and return immediately
  • integration of function in modelica
  • Stitching 2 images (OpenCV)
  • Python function to read variable length blocks of data from file while open
  • How to access recipient on sent messages page with mailboxer
  • Find group of records that match multiple values
  • Possible to get mouse events fired when cursor is outside page?
  • How to detect interior vertices in groups of 2d polygons? (E.g. ZIP Codes to determine a territory)
  • Looking for good analogy/examples for monitor verses semaphore
  • Display images in Django
  • Get one-time binding to work for ng-if
  • Sails.js/waterline: Executing waterline queries in toJSON function of a model?
  • How reduce the height of an mschart by breaking up the y-axis
  • Redux, normalised entities and lodash merge
  • Weird JavaScript statement, what does it mean?
  • Do create extension work in single-user mode in postgres?
  • Traverse Array and Display in markup