Abstract: Trajectories carry rich motion cues and thus have been leveraged to many high-level computer vision tasks. Due to the easy implementation of simple trackers, most previous work on trajectory-based applications utilizes raw tracking outputs without explicitly considering tracking errors. Reliable trajectories are prerequisite for modeling and recognizing high-level behaviors. Therefore, this paper tackles such problems by rectifying raw trajectories, which aims to post-process existing trajectories. Our approach firstly splits them into short tracks, and then infers identity ambiguity to remove unquali ed detection responses. At last, short tracks are stitched via maximum bipartite graph matching. This post-processing is completely scene-free. Results of trajectory rectification and their bene ts are both evaluated on two challenging datasets. Results demonstrate that recti ed trajectories are conducive to high-level tasks and the proposed approach is also competitive with state-of-the-art multi-target tracking methods.