COMMENTS:
Jonathan Hall
SUMMARY:
This paper describes gesture recognition technology. GRANDMA (Gesture Recognizers Automated in a Novel Direct Manipulation Architecture) is a toolkit for developing gesture interface and was used to create a gesture-based drawing program (GDP).
Rubine begins by explaining how the GRANDMA toolkit was used to create new gestures and handlers for a GDP. Both the GDP interface and application were developed using the GRANDMA toolkit. The user created a new gesture class using the gesture designer; 15 training samples provided sufficient variance in the structure of the gesture.
Rubine goes onto discuss the 13 features used for gesture recognition and how these features are used to classify gestures. Each gesture class has weights and in terms of training, the goal is to determine the weight of the sample gestures used. If a gesture is evaluated to more than one class, then the gesture is determined to be ambiguous and is rejected.
Rubine also briefly mentions GDPs developed using Eager Recognition and Multi-finger recognition as extensions. The eager method recognizes gestures as soon as they are unambiguous, and multi-finger recognizes gestures made with multiple fingers simultaneously.
DISCUSSION:
GRANDMA sounds like an awesome toolkit for creating gesture recognizers. After reading this paper, it’s clearer to me how gestures are being used to not only create shapes and sketches, but to perform certain actions in the GDP. However, I was a bit confused by the stroke manipulation phase. I don’t know if it was clear whether or not gestures for manipulation are classified same as other high-level operation gesture.
I find the simplicity of the 13 features used for recognition very appealing, but I still believe that they are explained better in Hammond’s gesture recognition chapter. Also, Figure 6 in the paper should have been divided up to show each feature clearly, but I imagine that would make for a very long paper.
To me, it seems that all gestures are classified the same, and the manipulation phase occurs after recognition has happened. The manipulation that occurs depends on the particular gesture and any applicable attributes of the gesture. For example, the pack gesture is like a selection gesture, and any shapes within the boundary of the pack gesture will be selected and can be further manipulated as one unit. The rectangle creation gesture does not have any manipulation; the rectangle is created by the gesture and that is it. The recognition of the pack and the rectangle create gestures are the same, the manipulation is just a following phase that depends on the recognized gesture. Hopefully that is what you were wondering about.
ReplyDeleteGRANDMA toolkit itself is indeed awesome, but I think the features is more appealing, Figure 6 is a classical show for all the features, for a scientific paper, I think there is really nothing more they can do.
ReplyDelete