NOTE: The table of results below has been replaced by our new evaluation, which includes the teddy and cones images and has many new features.

Overall comparison of algorithms

The table below is the current, on-line version of Table 5 of our paper [1]. It compares the performance of different stereo algorithms on four test image pairs, including five algorithms implemented by us and many implementations by other researchers, who have kindly sent us their results. The numbers represent the percentage of "bad" pixels, i.e., pixel whose absolute disparity error is greater than 1. For each image pair, we report percentages for (1) all pixels, (2) pixels in untextured areas (except for the Map images which are textured almost everywhere), and (3) pixels near depth discontinuities. Only non-occluded pixels are considered in all three cases, and we ignore a border of 10 pixels (18 for Tsukuba) when computing statistics. For more details, please see our paper.

All algorithms are run with constant parameter settings across all four images. The rank of each algorithm in each column is indicated with a small red number. The algorithms are listed roughly in decreasing order of overall performance (as measured by cumulative rank), and the minimum (best) value in each column is shown in bold. Each number in the "all" column links to a page containing the corresponding disparity map as well as disparity error maps. The column headers link to the binary maps defining "all" (unoccluded), "textureless", and "discontinuity" regions (only white areas are counted). The image names link to the ground truth maps.

Our implementation:

[1]

D. Scharstein and R. Szeliski. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms, IJCV, 2002. Five algorithm have been implemented: a - Graph cuts using alpha-beta swaps (Boykov, Veksler, and Zabih, PAMI 2001); b - Bayesian diffusion (Scharstein and Szeliski, IJCV 1998); c - SSD + min-filter (i.e., shiftable windows), window size = 21; d - Dynamic programming, similar to Bobick and Intille (IJCV 1999); e - Scanline optimization (1D optimization using horizontal smoothness terms).

Other authors' implementations:

[2]	V. Kolmogorov and R. Zabih. Computing visual correspondence with occlusions using graph cuts. ICCV 2001. a - original submission b - new submission with automatic parameter setting (same as in [21])
[3]	J. Sun, H. Y. Shum, and N. N. Zheng. Stereo matching using belief propagation. PAMI 2003 (also in ECCV 2002).
[4]	O. Veksler. Stereo matching by compact windows via minimum ratio cycle. ICCV 2001.
[5]	Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. PAMI 2001.
[6]	L. Zitnick and T. Kanade. A cooperative algorithm for stereo matching and occlusion detection. PAMI 2000.
[7]	H. Hirschmüller. Improvements in real-time correlation-based stereo vision. CVPR 2001 Stereo Workshop / IJCV 2002.
[8]	J. Shao. Combination of stereo, motion and rendering for 3D footage display. CVPR 2001 Stereo Workshop / IJCV 2002.
[9]	S. H. Lee, Y. Kanatsugu, and J.-I. Park. Hierarchical stochastic diffusion for disparity estimation. CVPR 2001 Stereo Workshop / IJCV 2002.
[10]	C. Sun. Fast stereo matching using rectangular subregioning and 3D maximum-surface techniques. CVPR 2001 Stereo Workshop / IJCV 2002.
[11]	M. Gong and Y.-H. Yang. Multi-baseline stereo matching using genetic algorithm. CVPR 2001 Stereo Workshop / IJCV 2002.
[12]	S. Birchfield and C. Tomasi. Depth discontinuities by pixel-to-pixel stereo. ICCV 1998.
[13]	S. Birchfield and C. Tomasi. Multiway cut for stereo and motion with slanted surfaces. ICCV 1999.
[14]	S. Roy and I. J. Cox. A maximum-flow formulation of the N-camera stereo correspondence problem. ICCV 1998.
[15]	K. Mühlmann, D. Maier, J. Hesser, and R. Männer. Calculating dense disparity maps from color stereo images, an efficient implementation. CVPR 2001 Stereo Workshop / IJCV 2002.
[16]	M. Lin and C. Tomasi. Surfaces with occlusions from layered stereo. Ph.D. thesis, Stanford University, 2002.
[17]	O. Veksler. Fast variable window for stereo correspondence using integral images. CVPR 2003.
[18]	M. Agrawal and L. Davis. Window-based discontinuity preserving stereo. CVPR 2004.
[19]	H. Mayer. Analysis of means to improve cooperative disparity estimation. ISPRS Conf. on Photogrammetric Image Analysis, 2003.
[20]	J. Y. Goulermas and P. Liatsis. A Collective-based adaptive symbiotic model for surface reconstruction in area-based stereo. IEEE Trans. Evolutionary Computation, vol.7(5), pp.482-502, 2003.
[21]	V. Kolmogorov and R. Zabih. Multi-camera scene reconstruction via graph cuts. ECCV 2002.
[22]	(Withdrawn)
[23]	L. Hong and G. Chen. Segment-based stereo matching using graph cuts. CVPR 2004.
[24]	Y. Wei and L. Quan. Region-based progressive stereo matching. CVPR 2004.
[25]	M. Bleyer and M. Gelautz. A layered stereo algorithm using image segmentation and global visibility constraints. ICIP 2004.
[26]	S. Forstmann, J. Ohya, Y. Kanou, A. Schmitt, and S. Thuering. Real-time stereo by using dynamic programming. CVPR 2004 Workshop on real-time 3D sensors and their use.
[27]	J. Sun, Y. Li, S. B. Kang, and H.-Y. Shum. Symmetric stereo matching for occlusion handling. CVPR 2005.
[28]	R. Brockers, M. Hund, and B. Mertsching. A fast cost relaxation stereo algorithm with occlusion detection for mobile robot applications. VMV 2004.
[29]	M. Bleyer and M. Gelautz. Graph-based surface reconstruction from stereo pairs using image segmentation. Proc. SPIE, vol. 5665, January 2005.
[30]	C. Kim, K.M. Lee, B.T. Choi, and S.U. Lee. A dense stereo matching using two-pass dynamic programming with generalized ground control points. CVPR 2005.
[31]	M. Gong and Y.-H. Yang. Near real-time reliable stereo matching using programmable graphics hardware. CVPR 2005.
[32]	O. Veksler. Stereo correspondence by dynamic programming on a tree. CVPR 2005.
[33]	K.-J. Yoon and I.-S. Kweon. Locally adaptive support-weight approach for visual correspondence search. CVPR 2005.
[34]	J. Jang, K. Lee, and S. Lee. Stereo matching using iterated graph cuts and mean shift filtering. ACCV 2006.
[35]	A. Criminisi, J. Shotton, A. Blake, C. Rother, and P.H.S. Torr. Efficient dense-stereo with occlusions and new view synthesis by four state DP for gaze correction. Submitted to IJCV 2005.
[36]	A symmetric patch-based correspondence model for occlusion handling. Y. Deng, Q. Yang, X. Lin, and X. Tang. ICCV 2005.