We analyse the effectiveness of RMSE, PSNR, SSIM and FOM for evaluating edge
detection algorithms used for automated coastline detection. Typically, the
accuracy of detected coastlines is assessed visually. This can be impractical
on a large scale leading to the need for objective evaluation metrics. Hence,
we conduct an experiment to find reliable metrics. We apply Canny edge
detection to 95 coastline satellite images across 49 testing locations. We vary
the Hysteresis thresholds and compare metric values to a visual analysis of
detected edges. We found that FOM was the most reliable metric for selecting
the best threshold. It could select a better threshold 92.6% of the time and
the best threshold 66.3% of the time. This is compared RMSE, PSNR and SSIM
which could select the best threshold 6.3%, 6.3% and 11.6% of the time
respectively. We provide a reason for these results by reformulating RMSE, PSNR
and SSIM in terms of confusion matrix measures. This suggests these metrics not
only fail for this experiment but are not useful for evaluating edge detection
in general.

我们分析了用于自动海岸线检测的边缘检测算法的有效性，通过评估 RMSE、PSNR、SSIM 和 FOM 等评估指标，发现 FOM 是选择最佳阈值的最可靠指标。相对于 RMSE、PSNR 和 SSIM，在这次实验中，FOM 在 92.6% 的时间内选择到更好的阈值，在 66.3% 的时间内选择到最佳阈值。而 RMSE、PSNR 和 SSIM 的选择率分别仅为 6.3%、6.3% 和 11.6%。通过重新解释 RMSE、PSNR 和 SSIM 的混淆矩阵度量，我们提出了对这些结果的解释，这表明这些评估指标不仅在这次实验中效果不好，而且在一般的边缘检测评估中也不实用。