Artificial Neural Networks for Wiimote gesture recognition
This software was implemented in Matlab and Java: Matlab for its ANN framework, and Java for building the GUI and communicating with the device (VRPN).
There was a lot of effort in identifying a good set of characteristics on the signal that the ANN could work with, testing different ANN topologies until finding one "enough-well", defining patterns to identify the start and end of a signal, reducing signal noise, and so on.
The current VRPN driver for the Wiimote had a limitation about the feedback that could give to the user (flashlights manipulation), so I modified it to add it.
The signal received from the Wiimote was the 3-axis accelerometer analog values.
Interface of the program in Java. As the user performed the gestures the interface showed the graphical signal and identified gesture. Gestures were of three types: throwing a bowling ball, swinging a tennis racket and punching in boxing.
Some online processing was performed on the accelerometer signals to reduce noise, identify start/end of the signal, and transform it to some set of characteristics to be used as input of the ANN. This graphic shows how the software identified the start of a gesture (in red) and its end (in green). There are three different signals, one for each accelerometer.
