Back

MacAction: Realistic 3D macaque body animation based on multi-camera markerless motion capture

Martini, L. M.; Bognar, A.; Vogels, R.; Giese, M. A.

2024-04-08 animal behavior and cognition
10.1101/2024.01.29.577734 bioRxiv
Show abstract

Social interaction is crucial for survival in primates. For the study of social vision in monkeys, highly controllable macaque face avatars have recently been developed, while body avatars with realistic motion do not yet exist. Addressing this gap, we developed a pipeline for three-dimensional motion tracking based on synchronized multi-view video recordings, achieving sufficient accuracy for life-like full-body animation. By exploiting data-driven pose estimation models, we track the complete time course of individual actions using a minimal set of hand-labeled keyframes. Our approach tracks single actions more accurately than existing pose estimation pipelines for behavioral tracking of non-human primates, requiring less data and fewer cameras. This efficiency is also confirmed for a state-of-the-art human benchmark dataset. A behavioral experiment with real macaque monkeys demonstrates that animals perceive the generated animations as similar to genuine videos, and establishes an uncanny valley effect for bodies in monkeys.

Matching journals

The top 3 journals account for 50% of the predicted probability mass.

1
Nature Methods
○ 385 papers in training set
Top 0.1%
33.8%
Similar papers in this journal
2
Nature Communications
○ 5641 papers in training set
Top 16%
11.7%
Similar papers in this journal
3
Scientific Reports
○ 3612 papers in training set
Top 5%
8.8%
Similar papers in this journal
50% of probability mass above
4
Scientific Data
○ 209 papers in training set
Top 0.5%
5.4%
Similar papers in this journal
5
eLife
● 5828 papers in training set
Top 29%
4.3%
Similar papers in this journal
6
PLOS Computational Biology
● 1863 papers in training set
Top 9%
4.0%
7
iScience
○ 1154 papers in training set
Top 5%
3.5%
8
PLOS ONE
● 5266 papers in training set
Top 35%
3.4%
9
Science Advances
● 1243 papers in training set
Top 15%
2.4%
10
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 24%
2.1%
11
Communications Biology
○ 993 papers in training set
Top 14%
1.7%
12
Methods in Ecology and Evolution
○ 176 papers in training set
Top 1%
1.7%
13
Science
● 477 papers in training set
Top 6%
1.5%
14
Cell Reports Methods
○ 165 papers in training set
Top 2%
1.4%
15
PLOS Biology
● 486 papers in training set
Top 6%
1.3%
16
Nature
○ 645 papers in training set
Top 9%
1.1%
17
Journal of Neuroscience Methods
○ 122 papers in training set
Top 1%
1.1%
18
Current Biology
○ 665 papers in training set
Top 9%
1.0%
19
Neuron
○ 337 papers in training set
Top 5%
0.9%
20
Proceedings of the Royal Society B: Biological Sciences
● 393 papers in training set
Top 6%
0.8%
21
eneuro
● 439 papers in training set
Top 8%
0.8%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.