Quantifying Shark Behavior using Unoccupied Aircraft Systems (UAS) and a Modified Application of a Deep Learning Pose Estimation Framework

Abstract

Understanding the behavioral ecology of marine species is crucial for conservation and management, but has lagged in the study of elasmobranchs due to their elusive nature. Using Unoccupied Aircraft Systems (UAS, or drones) to observe sharks improves detection, minimizes disturbance, reduces costs, and increases efficiency while deep learning, a type of artificial intelligence, streamlines and accelerates data analysis. This study examines the efficacy of fine-scale tracking, quantification, and analysis of near-shore shark behaviors through aerial drone observations and a deep learning pose estimation tool. Surveys were conducted in portions of the Rachel Carson Reserve in North Carolina, a coastal estuarine habitat with a multi-species assemblage of juvenile and adult sharks. The DJI Mavic 3 Classic and the DJI Mavic 3 Enterprise, equipped with a 20-megapixel camera, were used to capture videos at altitudes ranging from 20 to 25 meters in July and August, based on optimal shark and weather conditions. The surveys produced stationary nadir aerial videos of 1-4 sharks at a time with durations ranging from 20 seconds to 2 minutes. Additional videos from the eastern Mediterranean were provided by partners from the University of Haifa using a DJI Phantom 3 equipped with a 12.4 megapixel camera at altitudes ranging from 40 to 60 meters and were clipped to only frames in which the drone is stationary. All videos were re-encoded, downsampled and trimmed to maximize processing efficiency. Behavioral assessments of in-video shark activity were processed with Social LEAP Estimates Animal Poses (SLEAP), a deep learning based framework for multi-animal pose estimation. SLEAP outputs allow for calculating velocities, vectors, tail beat frequency, and other parameters to understand individual shark movement. Further analysis quantifies behavioral synchronicity and covariance of multiple sharks, where data-driven ethograms of movements can be parsed out using k-means clustering and hierarchical clustering can drive comparisons of vectors amongst larger groups of sharks. The parameters able to be analyzed depend on environmental conditions and data collection methods. This study provides a new standardized approach for further characterization and quantification of the behavior of nearshore sharks in isolation and aggregation using drones and video-based behavioral analysis.