Abstract

In this study, we designed an automated classification method, inspired by human taxonomic principles, to distinguish visually similar species of pelagic fish in images through the integration of morphological feature analysis with a hierarchical classification technique. By adapting the Keypoint R-CNN model for automated extraction of morphological characteristics, we accurately classified images of anchovies, mackerel, jack mackerel, and sardines, outperforming the results of the direct use of deep learning-based computer vision algorithms. Our method includes taxonomic analysis, exploiting geometric characteristics such as distances and angles between key body parts, segmenting patterned areas, and extracting texture features. Furthermore, we developed hierarchical classification models that employ a dichotomous key based on these key morphological traits to assess specific fish features such as size, shape, mouth orientation, and color patterns, simulating taxonomic classification. We achieved macro-precisions of up to 1.00 for small fish species and 0.98 for larger species, highlighting the pivotal role of keypoint detection combined with hierarchical classification in addressing challenging taxonomic tasks in marine organisms, and providing a scalable and adaptable solution for further applications. © 2013 IEEE.