KAIST Develops AI That Reads Animal Behavior Like Language (IMAGE)

The Korea Advanced Institute of Science and Technology (KAIST)

KAIST Develops AI That Reads Animal Behavior Like Language

Caption

Overview of the BehaVERT pipeline.
This figure shows how BehaVERT analyzes animal behavior from video. First, skeletal keypoints and behaviors are labeled using a web-based tool. The skeletal coordinates from each video frame are then converted into 768-dimensional “tokens” and entered into a BERT-based transformer model. The model can classify both the behavior in each individual frame and the overall state of the full sequence. The tokens from the final layer are further used for unsupervised clustering and attention analysis, allowing researchers to visualize, along the timeline, which behaviors the model focused on when making its decision.

Credit

KAIST

Usage Restrictions

No

License

Licensed content

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.