Abstract: Panoptic segmentation aims to assign both semantic label and instance ID for things class(countable), and only semantic label for stuff class(amorphous), and attracts much interest in recent years. The existing methods focus on detection-then-segment methods. They cannot handle instance overlaps in real scenarios. This leads to insufficient performance. This paper proposes a panoptic segmentation method based on the convolutional neural network. This method models the mutual relationships inside and outside the parted instance via a tailored representation, namely offset fields. This means the maximum and minimum offset vectors correspond to its instance border. Meanwhile, this paper proposes a CNN-based module for predicting those two offsets. Besides, to further explore the potential of the proposed offset fields, we carried out two different manners to obtain the instance result via the offset fields. To verify the effectiveness of the proposed offset fields on alleviating instances overlaps, we conducted extensive experiments on the challenging cityscapes dataset and obtained the good PQ value.