Constellation Models for Sketch Recognition

Comments

Summary

In this paper the author develops a constellation model for the recognition of strokes in sketches of particular classes of objects. In this model the author captures the structure of the object based on shape, size and pairwise features. The recognition algorithm determines a maximum-likelihood labeling for an unlabelled sketch by searching through the space of possible label assignments using a multi-pass branch and bound algorithm.

The constellation model consists of features of individual as well as features of pairs of parts. Individual features capture shape and global position of parts and pairwise features capture relative positions of parts. Some features are labeled as mandatory and some are labeled as optional. Individual features are calculated for all features whereas pairwise features are only calculated for the pair which has at least one mandatory feature.

An object class model is represented by a probability distribution over features in object constellation models. This function is learned from a set of example labeled sketches. To support recognition with a minimum number of examples it uses a diagonal covariance matrix.
Labeling Likelihood is the quality of particular matching between labels and strokes is scored using a cost function. Maximum likelihood search procedure finds the most plausible labeling for all strokes that appear in the image.

The algorithm was tested on 5 classes of objects with 7-15 labels and drawings that had 3-200 strokes. Most of the time spent in for the running of the algorithm was in the initialization function which calculated all the features of the stroke. Images with higher number strokes consumed most time so the author developed a technique to reject spurious strokes using a threshold. The recognition went wrong for several cases such as inability to find suitable mandatory features, mislabeling of mandatory strokes and mislabeling of optional strokes.


Discussion

In this paper the author discusses the technique of sketch recognition for complex sketches based on individual features and spatial features. Its application is very limited on objects with fewer features. This algorithm can recognize a stroke with the database of strokes but It cannot help in beautification of strokes.