Drones and Machine Learning for Marine Animal Behavior Analysis
Abstract
Understanding the behavior of marine animals is critical to effective management especially
as they fall under increasing anthropogenic pressures. Recent advances in two technologies,
drones and machine learning offer versatile, data driven, automatable solutions capable
of effective collection and analysis of large datasets. In this paper I illustrate
how pose estimation as an effective machine learning based solution for analyzing
marine animal behavior. This study investigates pose estimations use on drone imagery
due to its rising prevalence in marine science and prior combination with pose estimation
in our lab. As initial work at our lab has investigated the use of pose estimation
on marine mammal datasets and my goal is to expand on these efforts and build an overview
of both technologies integration for researchers interested working with them. In
the present study I use a collection of shark video taken by myself and other Duke
researchers locally off the Rachel Carson Reserve on the North Carolina coast as demonstration
and to help build a catalog of models and best practices for use of pose estimation
on different taxa.
This paper will provide an overview of drones and pose estimation including Social
LEAP Estimates Animal Poses (SLEAP), a pose estimation framework which has proven
to have good potential in marine science. SLEAP was chosen due to its accessibility,
versatility and tracking algorithm which allows multiple subjects to be tracked and
analyzed at the same time. The latter is a major steppingstone for pose estimation
software as past projects may have been able to identify multiple individuals in one
frame but not be able to keep track of who is who across thousands of frames of video.
Covered topics will include:
1. Technical overview of drones and pose estimation.
2. Data collection
3. Using pose estimation
a. Model types and programming
4. Data export and processing
5. Analysis
6. Conclusions on using pose estimation in marine science and future work.
After data export, a novel solution will also be assessed for compensating for camera
movement, in this case a moving drone, which has proven to be one of the biggest roadblocks
of using SLEAP, which was developed for processing stationary video. This solution
processes data in a way that is plug and play with existing analytical methods and
will be open source.
Type
Master's projectDepartment
Nicholas School of the EnvironmentPermalink
https://hdl.handle.net/10161/27232Citation
Poling, David (2023). Drones and Machine Learning for Marine Animal Behavior Analysis. Master's project, Duke University. Retrieved from https://hdl.handle.net/10161/27232.Collections
More Info
Show full item record
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Rights for Collection: Nicholas School of the Environment
Works are deposited here by their authors, and represent their research and opinions, not that of Duke University. Some materials and descriptions may include offensive content. More info