Mindsheet Threat Detection Algorithms

Mindsheet Threat Detection Algorithms

Computer assisted threat assessment was weak in many, if not all, all of the MoD Grand Challenge entries. Most of the systems relied heavily on human intervention. In an operational application, this manual approach:

  • fails when communications with the unmanned vehicle fails;
  • fails when the operator is overloaded, or, distracted by other events;
  • fails when the operator loses concentration after a prolonged period of time;
  • overloads the operator at the base station with vast amounts of incoming information.

However, a task, that is so deceptively effortless for alert people, has dogged computer vision researchers for forty years. The problems are immense:

  • six degrees of viewpoint variance on the threat object;
  • object occlusions;
  • object shape and colour variances;
  • elastic objects with no fixed shape;
  • sensor system noise;
  • environmental variations (fog, rain, mist, …).
Occluded Marksman Threat

Occluded marksman, photographed by the Team Mindsheet agile ground vehicle during the MoD Grand Challenge 2008

A straight forward correlation system will quickly get bogged down with a combinatorial explosion of threat correlations. Worse still, most threats will not have been seen before and yet humans effortlessly generalise and correctly classify these unknown threats. In many circumstances, humans can even sense threats, merely by the absence of normality.

And yet, analysis of video imagery could bring many benefits to the Mindsheet surveillance robot platform (Testudo):

  • Detection of suspicious behaviour eg. Peculiar manoeuvring pattern of bomb car during the collision with Glasgow airport
  • Greater stand-off range as explosive molecules do not have to be present in the detector
  • Lower cost sensors which are already present in the form of CCTV cameras
  • Detection of background anomalies that indicate an impending threat
  • Detection of changes, eg. Bag or Case left behind
  • Wide area coverage

Mindsheet have created threat detection algorithms which are based on the recent work of Jeff Hawkins: hierarchical temporal memory (“HTM”) systems.  This new class of algorithm is biologically inspired by the workings of the neo-cortex.  The conceptual framework for this approach is laid out in Jeff Hawkins’ paper: Hierarchical Temporal Memory Systems.

HTMs are organised as a tree-shaped hierarchy of nodes, where each node implements a common learning and memory function. They are ideally suited to spatial and temporal pattern recognition tasks and the algorithm efficiently exploits the fact that features that are close together in space and time are more likely to be more correlated than features that are not.

However, Mindsheet’s original implementation discovered many hurdles with this approach that include:

  • restricted input to binary and 2D patterns;
  • very slow training to allow generalisation to all viewpoint variances;
  • no generalisation to different scales and translations;
  • no clear guidance for setting the right hierarchical topology.

Mindsheet implementation of threat detection algorithm, showing the data fusion with other signal types (such as audio information).

Mindsheet has developed its own version of these recognition algorithms that overcome some of these problems. The new techniques now include:

  • a holographic property that repeats all learnt vectors at every node in the hierarchy for rapid translational and scale generalisation (a variant of object oriented inheritance);
  • faster training by automatically generating rotational variance vector sets;
  • grey scale input by enabling Euclidean distance measurement to learnt grey scale vectors .

The figure below shows preliminary results from this approach, and shows promise. In particular, the initial training very interestingly allowed the subtle discrimination between look-out behaviour and normal civilian behaviour.

Mindsheet Threat Recognition

Preliminary results of pose recognition using the Mindsheet algorithms

Mindsheet proposes to take these algorithms to the next level by collecting training and test sets derived from CCTV feeds and the data that Mindsheet gathered at the MoD Grand Challenge finale. Mindsheet will then evaluate and refine the algorithms for use in the interpretation of images from static sensors, air and ground vehicles.

Aerial shot of Copehill Down MoD Grand Challenge

The original image, resolution degraded to keep the size of this annex less than 1 MByte, taken during the pre-trial at Copehill Down Village


Armed Vehicle

The result of a very large magnification of a small section of the plan view aerial image, circled in yellow, showing a weaponised vehicle.

  • Investigation of the development of a fractal index map of an image, where the inverse of the fractal index of an image is displayed, in false colour, as an overlay on the original image. Man-made objects, such as IEDs, tend to have a low fractal index (low image complexity), which would be displayed as red in an image. This is a promising technology for the display of man-made objects that are partially camouflaged.
  • Difference detection software, to highlight changes that have occurred over time. Any changes detected could be forwarded for more in-depth investigation by the threat detection software, or, by an operator.
  • Automatic detection of circles and ellipses in an image. These objects are frequently associated with man-made objects, such as weapons and IEDs.


Be Sociable, Share!

Tags: , , ,

Facebook Comments:

Leave A Reply (2 comments so far)

The comments are closed.

  1. I am in fact grateful to the holder of this website who has shared this great piece of writing at here.

  2. While hiring a registered patent attorney will increase the investment of the filing process, the USPTO strongly advises seeking professional help due to the
    complexity of the process. To be able to represent you well in any
    case he should have experience working in their chosen technical specialty before studying law and have wide knowledge in writing patents in different fields.
    After filing a patent, the patent is sustained by paying periodic maintenance fee i.


  • The 7 minutes business blog profit cycle workshop delivered a practical “teach a little, do it” – quickly converted theory into practice and was a real boost to my confidence. Excellent session supplied by Mindsheet in Havant.
  • Professional, knowledgable and a great understanding of the web marketing process. Mindsheet are great. Tim Stell

Mindsheet Innovation Lens