Back

Quantifying social roles in multi-animal videos using subject-aware deep-learning

Goss, K.; Bueno-Junior, L. S.; Stangis, K.; Ardoin, T.; Carmon, H.; Zhou, J.; Satapathy, R.; Baker, I.; Jones-Tinsley, C. E.; Lim, M. M.; Watson, B. O.; Sueur, C.; Ferrario, C. R.; Murphy, G. G.; Ye, B.; Hu, Y.

2024-07-10 animal behavior and cognition
10.1101/2024.07.07.602350 bioRxiv
Show abstract

Analyzing social behaviors is critical for many fields, including neuroscience, psychology, and ecology. While computational tools have been developed to analyze videos containing animals engaging in limited social interactions under specific experimental conditions, automated identification of the social roles of freely moving individuals in a multi-animal group remains unresolved. Here we describe a deep-learning-based system - named LabGym2 - for identifying and quantifying social roles in multi-animal groups. This system uses a subject-aware approach: it evaluates the behavioral state of every individual in a group of two or more animals while factoring in its social and environmental surroundings. We demonstrate the performance of subject-aware deep-learning in different species and assays, from partner preference in freely-moving insects to primate social interactions in the field. Our subject-aware deep learning approach provides a controllable, interpretable, and efficient framework to enable new experimental paradigms and systematic evaluation of interactive behavior in individuals identified within a group.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

1
Scientific Reports
○ 3612 papers in training set
Top 0.4%
21.5%
Similar papers in this journal
2
Nature Communications
○ 5641 papers in training set
Top 18%
9.6%
Similar papers in this journal
3
iScience
○ 1154 papers in training set
Top 0.7%
7.7%
Similar papers in this journal
4
Nature Methods
○ 385 papers in training set
Top 2%
6.6%
Similar papers in this journal
5
Cell Reports Methods
○ 165 papers in training set
Top 0.2%
5.4%
Similar papers in this journal
50% of probability mass above
6
Scientific Data
○ 209 papers in training set
Top 0.5%
5.1%
7
eLife
● 5828 papers in training set
Top 26%
4.8%
8
Lab Animal
○ 11 papers in training set
Top 0.1%
3.4%
9
PLOS ONE
● 5266 papers in training set
Top 35%
3.4%
10
Journal of Visualized Experiments
● 34 papers in training set
Top 0.2%
3.2%
11
PLOS Biology
● 486 papers in training set
Top 2%
2.7%
12
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 21%
2.4%
13
Methods in Ecology and Evolution
○ 176 papers in training set
Top 0.9%
2.3%
14
Communications Biology
○ 993 papers in training set
Top 12%
1.9%
15
PLOS Computational Biology
● 1863 papers in training set
Top 14%
1.7%
16
Frontiers in Behavioral Neuroscience
○ 49 papers in training set
Top 0.5%
1.7%
17
Nature Human Behaviour
○ 95 papers in training set
Top 1%
1.7%
18
Science Advances
● 1243 papers in training set
Top 24%
1.3%
19
Advanced Intelligent Systems
○ 11 papers in training set
Top 0.2%
1.1%
20
Nature
○ 645 papers in training set
Top 11%
0.8%
21
Nature Machine Intelligence
○ 70 papers in training set
Top 2%
0.8%
22
Communications Psychology
○ 22 papers in training set
Top 0.5%
0.6%
23
Science
● 477 papers in training set
Top 10%
0.6%
24
Nature Aging
○ 60 papers in training set
Top 2%
0.6%
25
Nature Neuroscience
○ 252 papers in training set
Top 6%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.