The similarity video retrieval problem is traditionally solved by calculating all the possible distances of image matching between each frame v in the database video frame sequences and each frame q in the query frame sequence. Obviously, it is a time-consuming strategy. This paper offers a new technique called the piecewise method to overcome this drawback. The basic concept of this method is that the image matching distance between the pair of frames q and v is computed only if q is similar to v. In most cases, the executive time of this method is less than that of traditional methods. However, the piecewise method cannot accurately retrieve the videos satisfying the requirement of the query frame sequence from the database. This paper provides another more precise video retrieval method based on video shot, that is the piecewise refinement method, to remedy this shortcoming. It is a revision of the piecewise method. Experimental results show that the piecewise refinement method gives an impressive performance not only in processing time but also in storage space.