Abstract: To address the problem of frequent trajectory label switching of target vehicles in pure vision-based vehicle target tracking in traffic scenes caused by the lack of depth information, a binocular depth information reconstruction module based on the Deep SORT framework was introduced. The AD-Census binocular stereo matching technique was utilized to accurately reconstruct the three-dimensional information of target vehicles. Meanwhile, by combining with the efficient DETR target detection network, an end-to-end vehicle target tracking system integrating target detection, binocular depth estimation, and target tracking technologies was constructed. Experimental results demonstrate that the number of label switches (IDSW) during vehicle trajectory tracking is reduced to a maximum of 15, while the average tracking precision (MOTP) reaches 79.768%, and the tracking accuracy (MOTA) reaches 84.687%. The method proposed in this paper effectively reduces the switching frequency of driving trajectory labels and improves the tracking accuracy and stability.