Treffer: Object modeling for instance segmentation

Ph.D. ; Instance segmentation, also known as simultaneous detection and segmentation (SDS), aims at detecting each instance in the given image and predicting corresponding pixel-wise mask. How to model each instance to make the network easier to capture object properties is vital. Several different ways to model objects for the instance segmentation task is explored in this thesis. We predict elementary elements, generalized parts or entire object with specially designed neural network structures. Then we compose them together as the final result or aggregate related information for performance improvement. ; We first present a proposal-free framework, which detects and segments object instances via mid-level patches, covering part of instance. We design a unified trainable network on patches, which is followed by a fast and effective patch aggregation algorithm to infer object instances. Compared with previous proposal-based methods, our method benefits from end-to-end training. Without object proposal generation, computation time can also be reduced. In experiments, our method performs well on PASCAL VOC. ; In the second work, we design Sequential Grouping Networks (SGN) to tackle the problem of object instance segmentation. SGNs employ a sequence of neural networks, each solving a sub-grouping problem of increasing semantic complexity in order to gradually compose objects out of pixels. In particular, the first network aims to group pixels along each image row and column by predicting horizontal and vertical object breakpoints. These breakpoints are then used to create line segments. By exploiting two-directional information, the second network groups horizontal and vertical lines into connected components. Finally, the third network groups the connected components into object instances. The performance on Cityscapes and PASCAL VOC datasets manifests the effectiveness of this method. ; In the third part, we propose PANet, aiming at boosting information flow in prevalent proposal based instance segmentation ...