We propose a novel query-driven lazy learning algorithm which attempts to discover useful local patterns, called support patterns, for classifying a given query. The learning is customized to the query to avoid the horizon effect. We show that this query-driven learning algorithm can guarantee to discover all support patterns with perfect expected accuracy in polynomial time. The experimental results on benchmark data sets also demonstrate that our learning algorithm really has prominent learning performance.