摘要

we propose a parallel reverse attention network (PraNet) for accurate polyp（息肉） segmentation in colonoscopy images. Specifically, we first aggregate the features in high-level layers using a parallel partial decoder (PPD). Based on the combined feature,we then generate a global map as the initial guidance area for the following components. In addition, we mine the boundary cues using the reverse attention (RA) module， which is able to establish the relationship between areas and boundary cues. Thanks to the recurrent cooperation mechanism between areas and boundaries, our PraNet is capable of calibrating some misaligned predictions, improving the segment-ation accuracy.(1.聚合深层次特征，生成全局图；2. 使用反向注意模块来挖掘边界线索，建立区域于边界线索之间的关系)

介绍

we first predict coarse areas and then implicitly model the boundaries by means of reverse attention. There are three advantages to this strategy, including better learning
ability, improved generalization capability, and higher training efficiency. （首先预测粗糙区域，然后通过反向注意力隐式地对边界建模，该策略具有学习能力强、泛化能力强和训练效率高的优点）

贡献

(1) We present a novel deep neural network for real-time and accurate
polyp segmentation. By aggregating features in high-level layers using a parallel
partial decoder (PPD), the combined feature takes contextual information and
generates a global map as the initial guidance area for the subsequent steps.
To further mine the boundary cues, we leverage a set of recurrent reverse at-
tention (RA) modules to establish the relationship between areas and boundary
cues. Due to this recurrent cooperation mechanism between areas and bound-
aries, our model is capable of calibrating some misaligned predictions.（使用PPD来聚合深层次特征，生成全局图，RA模块建立区域与边界线索之间的关系）

(2) We introduce several novel evaluation metrics for polyp segmentation and present a comprehensive benchmark for existing SOTA models that are publicly available.

(3) Extensive experiments demonstrate that the proposed PraNet outperforms
most cutting-edge models and advances the SOTAs by a large margin, on five
challenging datasets, with real-time inference and shorter training time.

模型框架

Code: [PraNet](PraNet/lib at master · DengPingFan/PraNet · GitHub)

Parallel Partial Decoder(PPD)

class aggregation(nn.Module):
    # dense aggregation, it can be replaced by other aggregation previous, such as DSS, amulet, and so on.
    # used after MSF
    def __init__(self, channel):
        super(aggregation, self).__init__()
        self.relu = nn.ReLU(True)

        self.upsample = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)
        self.conv_upsample1 = BasicConv2d(channel, channel, 3, padding=1)
        self.conv_upsample2 = BasicConv2d(channel, channel, 3, padding=1)
        self.conv_upsample3 = BasicConv2d(channel, channel, 3, padding=1)
        self.conv_upsample4 = BasicConv2d(channel, channel, 3, padding=1)
        self.conv_upsample5 = BasicConv2d(2*channel, 2*channel, 3, padding=1)

        self.conv_concat2 = BasicConv2d(2*channel, 2*channel, 3, padding=1)
        self.conv_concat3 = BasicConv2d(3*channel, 3*channel, 3, padding=1)
        self.conv4 = BasicConv2d(3*channel, 3*channel, 3, padding=1)
        self.conv5 = nn.Conv2d(3*channel, 1, 1)

    def forward(self, x1, x2, x3):
        x1_1 = x1
        x2_1 = self.conv_upsample1(self.upsample(x1)) * x2
        x3_1 = self.conv_upsample2(self.upsample(self.upsample(x1))) \
               *  (self.upsample(x2)) * x3

        x2_2 = torch.cat((x2_1, self.conv_upsample4(self.upsample(x1_1))), 1)
        x2_2 = self.conv_concat2(x2_2)

        x3_2 = torch.cat((x3_1, self.conv_upsample5(self.upsample(x2_2))), 1)
        x3_2 = self.conv_concat3(x3_2)

        x = self.conv4(x3_2)
        x = self.conv5(x)

        return x

compared with high-level features, low-level features demand more computa-
tional resources due to their larger spatial resolutions, but contribute less to
performance. Motivated by this observation, we propose to aggregate high-level
features with a parallel partial decoder component(低特征需要更高的计算资源，性价比低）. More specifically, for an input polyp image I with size h ×w, five levels of features {fi, i = 1, …, 5} with resolution [h/2k−1, w/2k−1] can be extracted from Res2Net-based [12] backbone network. Then, we divide fi features into low-level features {fi, i = 1, 2} and high-level features {fi, i = 3, 4, 5}. We introduce the partial decoder pd(·) [29], a new SOTA decoder component, to aggregate the high-level features with a paralleled
connection. The partial decoder feature is computed by PD = pd(f3, f4, f5), and
we can obtain a global map Sg

Reverse Attention Module(RA)

# ---- reverse attention branch_4 ----
       crop_4 = F.interpolate(ra5_feat, scale_factor=0.25, mode='bilinear')
       x = -1*(torch.sigmoid(crop_4)) + 1
       x = x.expand(-1, 2048, -1, -1).mul(x4)
       x = self.ra4_conv1(x)
       x = F.relu(self.ra4_conv2(x))
       x = F.relu(self.ra4_conv3(x))
       x = F.relu(self.ra4_conv4(x))
       ra4_feat = self.ra4_conv5(x)
       x = ra4_feat + crop_4
       lateral_map_4 = F.interpolate(x, scale_factor=32, mode='bilinear')  # NOTES: Sup-2 (bs, 1, 11, 11) -> (bs, 1, 352, 352)

It can only capture a relatively rough location of the polyp tissues, without structural details (see Fig. 1). To address this issue, we propose a principle strategy to progressively mine discriminative polyp regions through an erasing foreground object manner [27,4]. Instead of aggregating features from all levels like in [4,13,36,33], we propose to adaptively learn the reverse attention in three parallel high-level features. In other words, our architecture can sequentially mine complementary regions and details by erasing the existing estimated polyp regions from high-level side-output features, where the existing estimation is up-sampled from thedeeper layer.()

Learning Process and Implementation Details