If the collection of training data is costly, we can gain by actively selecting particular informative data points in a sequential way. In a Bayesian decision theoretic framework, we develop a query selection criterion for classification models, which explicitly takes into account the utility of decisions. We determine the overall utility and its derivative with respect to changes of the queries. An optimal query now may be obtained by stochastic hill climbing. Simultaneously the model structure can be adapted by reversible jump Markov chain Monte Carlo.