We presented an intelligent wheelchair whose motion can be controlled by the user's face direction. In this paper, we propose to add intelligence to our wheelchair when the user is not riding. It can recognize the user's face and can move according to the gestures made by the user. Gesture is a good means to give commands because it can be used in noisy conditions and we would not like to speak loud in some public places. However, environments where wheelchairs are used cannot be controlled. This makes gesture recognition difficult. We propose an interactive way to solve this problem. When the wheelchair is not certain about the meaning of user's gesture, it guesses the meaning and moves a little accordingly to show its guess to the user. Then it observes the user's response, judging whether its guess is correct or not. This guess-action-observation cycle is repeated until the wheelchair can understand the user's intention.