Flow accuracy and interpolation evaluation
Update (12/29/2009): The technical report version of our upcoming journal paper is now available as Microsoft Research Technical Report MSR-TR-2009-179. It includes an extended related work section and a detailed analysis of the results on these pages. Based on this analysis, we have changed the table organization and now report endpoint errors before angular errors. We have also improved the visual presentation by allowing the repositioning of the result images. Finally, the interpolation evaluation has been finalized and is no longer in "beta" stage.Below are links to the current results. We report two measures of flow accuracy (endpoint and angular error) and two measures of interpolation quality. For each of the 4 measures we report 8 error metrics, resulting in a total of 32 tables. Each table is sorted by average rank across 24 columns of numerical results. Links to the 4 measures are included below, but the tables are also linked among each other.
We want to emphasize that we do not aim to provide an overall ranking among the submitted methods. While endpoint errors should be considered the most appropriate measure of flow accuracy, we do not identify a "default" error metric. In general it will depend on the application which of the 32 metric/statistic combinations might be best suited to compare the algorithms. Also note that the exact rank within any of the tables only gives a rough measure of performance, as there are various other ways that the scores across the 24 columns could be combined.
Note that the images shown are smaller and lower quality (jpgs) than the originals.
The preliminary evaluation results of our ICCV 07 paper are still available here.