In this paper, we present a proof-of-concept gesture recognition based on Artificial Neural Networks (ANN) for referees participating in the Robocup competitions. Here, we aim at recognizing gestures to interact with an application (Robocup Referee Box) and present the design and evaluation of our sensor-based gesture recognition. As input device we employ the Wii-controller (Wiimote) and the use of the iPhone, both of which recently gained much attention world-wide. We use both devices’ independent acceleration data for gesture recognition. The system allows the training of arbitrary gestures by users which can then be recalled for interacting with the systems. We exploit the devices’ sensor data and employ an artificial Neural Network model for training and recognizing user-chosen gestures. Our evaluation shows that our system is intuitive and easy to use.