An LED-glove based interaction system is introduced in this paper to provide desktop-free interface for teleconferencing. Using proposed interaction technology, users may collaborate with other remote users in multi-modal display environments. Private displays (LCD or head mounted displays) are used to organize data or refer to classified information, and shared display (projection display) is used to present and share information with other users. Interaction is composed of two modes. In mouse replacement mode, the glove may be used as a traditional mouse. Postures were used for mouse?s click and double-click of left/right buttons. In command mode, gestures were used for frequently used commands such as switching displays, moving information inter-display, fetching data from other displays, etc. Gestures were recognized with high success rate (above 94%) using Hidden Markov Model. In order to solve gesture-spotting problem, we used switches to turn on LEDs during gesture performance. Preliminary studies show that the proposed system is easy to use, and provides most useful interface for multi-modal display teleconferencing.