Nowadays, a major activity on the Internet is the retrieval and browsing of multimedia information. Yet, today?s search engines are not really up to the task. Users often have to query various search engines and browse many web-sites before finding a satisfactory answer. Once users find such an answer, presentation of the results is very rigid, i.e., not tailored to the users? needs, and it does not enable users to manipulate the data. Thus, this retrieval activity often becomes tedious. To improve user-information interaction, a major issue is to give search-engines the ability to access data semantics. In this study, we propose a framework to address the issue. Our framework combines database and multimedia data mining to endow web-based applications with the ability to let users manipulate the data at different levels of interest. As an experimental test-bed, we implemented a holiday planner that supports users in their search for a hotel. By using database technology, web-based multimedia interaction becomes possible, offering as a side effect more feasible and reliable feedback.