Instance segmentation and semantic segmentation have long been done using different neural network architectures, although recently there has been a combination of semantics and instance segmentation; "panoramic segmentation" But the classic approach still sees the two as separate tasks. Recently, Facebook AI uses a single neural network architecture to simultaneously perform instance segmentation (identifying people or animals in the foreground of the image) and semantic segmentation (classifying pixels in the background of the image). Their research uses both a unified neural network architecture to implement both instance and semantic segmentation. This new architecture enables efficient use of memory and computing resources and can be used as a benchmark for panoramic segmentation tasks.
Principle: This new architecture & ldquo; Panorama FPN & rdquo; added a semantic segmentation based on Facebook's 2017 R-CNN Branch. This new architecture allows for instance and semantic segmentation of images simultaneously.And the accuracy is comparable to a neural network that only performs instance or semantic segmentation, which is equivalent to halving the computational resources required by traditional methods. Tests on the COCO and Cityscapes datasets show that panoramic FPN performs much better than using two independent neural networks for instance and semantic segmentation under the same computing resources.
Importance: The panoramic FPN architecture is unified, easy to implement and resource utilization high, setting the benchmark for the subsequent panoramic segmentation study. The reduction in computational resource requirements and the depth of understanding of images can also have a profound impact on recognition systems that require complex image real-time processing.The segmentation of objects in the foreground of the image and the segmentation of the background as a unified task are essential for us to understand the scenes in the picture and complete further operations accordingly.