Nowadays, large bodies of data in different domains are collected and stored. An efficient extraction of useful knowledge from these data becomes a huge challenge. This leads to the need for developing distributed data mining techniques. However, only a few research concerns distributed clustering for analysing large, heterogeneous and distributed datasets. Besides, current distributed clustering approaches are normally generating global models by aggregating local results that would lose important knowledge. In this paper, we present a new distributed data mining approach where local models are not directly merged to build the global ones. Preliminary results of this algorithm are also discussed