In this paper, ongoing research pursuing the distinction of online handwriting into textual and different drawing classes is described. In the context of natural pen-based interactions, users will seamlessly switch between such different input modes. Therefore, it is vital for pen input recognition systems to be able to distinguish between these cases, preferably in an early stage of processing. The method described in this paper is tested on data acquired in a multi-modal task setting where users are requested to specify shape and dimensions of bathrooms, using pen and speech. Mode detection in this context yields comparable outcomes to recent findings from the literature. The results presented here elaborate on these findings by examining the possibility to perform early recognition of input modes, so-called incremental recognition. To this end, PENDOWN as well as PENUP trajectories are being explored.