Monday, September 1, 2008

Visual Similarity of Pen Gestures

Comments

Daniel's blog

Summary


In this paper the author discusses the issues of designing good gestures so that they are easy to remember for the user of the system. The author is trying to develop a tool which will enable the UI designers to improve their gesture set so they are easy to remember and use.

The author investigates the gesture similarity and develops a computable, quantitative model of gesture similarity which will help in creating a gesture designer tool. The author conducts two experiments on the human beings to develop a computable, quantitative model of gesture similarity.

Perceptual similarity is the concept of how human beings perceive two shapes to be similar to each other. Psychologists have conducted investigations on the shapes which are simpler than gestures. Investigations conducted by Attneave found that the similarity of shape correlated with log of the area and tilt for parallelograms.

Multi-dimensional scaling is the technique for reducing the number of dimensions of the data set so that patterns can be more easily seen by viewing a plot of data, in two or three dimensions. Author here uses the MDS version called INDSCAL, that takes as input a proximity matrix of each participant and takes individual differences into account.

In experiment one the author makes a set of 14 gestures which are a widely dissimilar to each other. Each of the twenty participants are shown a set of 3 gestures on the screen and are asked to select the gesture which is mostly dissimilar to the the other two gestures. All possible combination are shown to the participants i.e. a total of 364 screens are shown. After the experiment the author had to analyze two important points from the data collected. 1) to determine what geometric properties of gesture influenced their perceived similarity. 2) to produce a model of gesture similarity, that given two gesture the system could predict the similarity that humans would perceive. The firs point was addressed through MDS plotting. The Euclidean inter-gesture distances corresponded to inter-gesture dissimilarities. The second point was addressed by running the regression analysis to determine which geometric features correlated with reported similarity. Some features were taken from Rubine’s algorithm and some inspired from MDS analysis. The author was able to derive a model which correlated 0.74 with the reported similarities.

In experiment 2 the the author wanted to explore how systematically varying different features would affect the perceived similarity. For this the author made 3 gesture sets of 9 gestures each. First set was to explore total absolute angle and aspect. Second was to explore length and area. Third was to explore rotation-related features. The author then took 2 from each set of gestures and made a fourth set of gestures. Again twenty people were shown the a set of 3 gestures and a total of 538 gesture sets were shown same as in experiment one. The trial was also examined using the techniques used in experiment one. Author was able to determine that length and area are not very significant contributors to similarity judgment. Another finding was that the perceived similarity among gesture is not proportional to the angle rotation of the gesture, instead gestures with horizontal and vertical lines are perceived more similar than those gestures whose components are diagonal.

Author concludes that human perception of similarity is very complicated and there are several cues involved in human perception which determines the similarity and dissimilarity of the gestures. However the authors model correlates 0.74 with the perceived similarity in experiment one which is a fairly good model.

Discussion

The author have conducted an extensive investigation to determine a model for perceived similarity of gestures. Even if we can determine that two gesture are similar to each we still cannot make a gesture set that is easier to remember. Remembrance of a gesture not only depends upon a gesture being dissimilar to another gesture so the user does not overlap the gestures in memory but also on the shapes and actions mapped to a gesture, the complexity of the gesture itself and also similar gestures for similar meanings will also contribute in the remembrance of the gestures.

2 comments:

manoj said...

i agree with his point. The author has not taken into consideration neither the operations mapped to the gesture nor the context of it. The study conducted completely omits these two factors.

andrew said...

I definitely think continued research in this area should look at context and the mappings of shapes to actions. Taking in to account other non-visual features should improve the metrics for finding similarity between gestures. However, this a little outside the scope of the presented research. The authors were focused on finding visual similarities, but perhaps they should have at least raised the point that recall involves more than just visual similarity.