Vox
  • 5 months ago
How AI could help us talk to animals
Transcript
00:00Way back in the 80s I noticed that sometimes when an elephant called a member of her family
00:08one individual would answer and everybody else ignored the calling animal and then she would
00:15call again and a different elephant would sort of lift her head up and you know rumble very loudly.
00:24This is Joyce Poole. She's been studying African elephants and their communication for 50 years.
00:30Then I started to think well okay so maybe they have a way of directing a call to a specific
00:36individual but we had no way of detecting that. Decades later she partnered up with Mickey Pardo
00:41who designed a study around her observations. I went out to the field I recorded calls with
00:46careful behavioral observations so we knew who made each call we knew who the call was addressed
00:51to we knew the context of the call. They encoded the acoustic information from the recordings into
00:56a long string of numbers along with the data Mickey collected about the calls. They fed nearly
01:02500 different calls like this into a statistical model and when given the acoustic structure of a
01:08new call the model could predict who the receiver of the call was much better than chance. In other
01:14words evidence suggesting African savannah elephants give each other names. When we posted
01:21it on Facebook somebody wrote back said that the earth just shifted a little bit
01:27and I think that's true. This is just one example of how machine learning is decoding complexities
01:34in animal communication that humans can't detect and now some AI researchers want to take the next
01:39step. Large language models like the ones that power chatbots but built for interspecies
01:45communication. Can we talk a little bit about love? There is still much to be learned about whales.
02:05When researchers study animal communication they usually employ a few methods recording
02:09their vocalizations observing and documenting the behavior and context around those sounds
02:14and sometimes doing a playback to measure the animal's response.
02:18All of these areas are already being impacted by AI. Recordings from the field don't usually sound
02:24like this. They often sound like this. Multiple animals vocalizing on top of one another in a
02:35noisy environment. This is known as the cocktail party problem and it's a common issue in the field
02:40of animal research. But machine learning solved a similar problem in human speech recognition.
02:46AI researchers trained a model called deep karaoke on lots of music tracks where instruments and
02:51vocals were recorded separately. Then also on the fully mixed tracks until it was able to do the
02:57task of separating out instruments and vocals in new music clips. Recently AI researchers have had
03:04some success applying similar algorithms to animal sound recordings which means you can take a clip
03:10of a group of macaque monkeys and single out one discernible call. Researchers could also start
03:19using AI in how they use playbacks in the field. You may have seen AI models that can be trained
03:25on lots of examples of a sound recording and then generate another unique version of it.
03:35Now the last movie woke up. AI researchers are starting to develop similar models for animal recordings.
03:45These are all types of supervised learning. That means that the model gets trained on lots of
03:50examples labeled by humans and in the elephant name study researchers were able to feed a model
03:55their observations which along with the sound data helped them detect something an elephant
04:00calls they couldn't through observation alone. You need to annotate a lot of data. Yossi Yovel
04:05trained a statistical model on 15,000 Egyptian fruit bat vocalizations which was able to identify
04:11the emitter of the call, the context of the call, its behavioral response and who the call was
04:16addressed to. And we annotated them manually or you know I'm already saying this is a restriction
04:21of the study because maybe we're missing something we're humans we're not bats. And that's the problem
04:26with supervised learning models. They are limited by what we humans already know about animal
04:31communication in order to label the training data and we don't know a lot. That's why some AI
04:38researchers say self-supervised models hold the most potential for decoding animal communication.
04:43This is how natural language processing models like CHAT-GBT are trained. Instead of human labeled
04:48examples they are trained on a large amount of unlabeled data and they can sort it according
04:54to patterns and categories it detects all on its own. In the example of CHAT-GBT it learned from
04:59all the books, websites, social media feeds and anything else it could scrape from the internet
05:04and came to its own conclusions about how language works. Every language has a shape that AI discovers.
05:12This is Eze Raskin. He co-founded the Earth Species Project, one of a few organizations that
05:17want to build models like this for animal communication. What he means by language
05:21having a shape is that language models are built out of relationships among words. Words that mean
05:27similar things are placed near each other, words that share a relationship, share a distance and
05:32direction. So man is to king as woman is to queen. So this is the shape of all those relationships
05:38among the English language's 10,000 most common words visualized here by the Earth Species Project.
05:44Flattened out it looks something like this. Something really miraculous happened in 2017
05:49and that was researchers discovered that you could take the shape of any one language
05:57and match it to the shape of any other language and the point which is dog ends up in the same
06:03spot. This idea that similar words can be located in other languages in roughly the same place
06:09is what gives the Earth Species Project hope we could do a version of this for animal communication.
06:14To do a translation without needing any examples, without needing a Rosetta Stone.
06:19This is complicated though because we know that animals don't just communicate with sound
06:23but with other senses too. But Eiza points out that we can learn from the fact that image
06:28generation models like DALI and MIDJOURNEY are built on the same large language model structure
06:32used for text. It turns out behind the scenes it's again these kinds of shapes. There's the
06:39shape that represents sound. The shapes that represents images. Those two shapes get aligned
06:44and now you can translate between images and text. Their expectation is that where non-human
06:52animals communication would line up with ours will tell us even more about what we have in common.
06:58Dolphins look in mirrors and recognize themselves. Elephants too. That's a kind of self-awareness.
07:04One concern with this plan is related to a step in self-supervised learning called
07:08validation. Meaning humans still need to refine these models by grading them on their answers.
07:14How would we do that in a communication so foreign from our own? We also might have too high of
07:19expectations of this overlap or the capacity for having a conversation with a non-human animal in
07:25a shared language and about shared experiences. Hey Kurt, how are you doing dude? So I'm about to
07:30translate that into a meow. We said hi, hi, hi, hi, you know. Next time you want to say how are you?
07:40I do not think that humans should be considered more important than other species but that doesn't
07:45mean that there's no usefulness in distinguishing between language which is this very specific
07:51behavior that at least based on what we currently know seems to be unique to humans and other forms
07:58of communication. In order to build these models the first step is collecting a lot more data on
08:03animal sounds than exists right now. And so I'm actually at the moment building up a database with
08:10all the individual calls so close to 10,000 records in that which is very small actually.
08:18Around the world animal researchers are in the midst of a massive data collection effort
08:22tagging and recording animals with video and sound and spatial data to feed these
08:27data thirsty models. Time will tell whether true interspecies communication will be facilitated by
08:32AI but researchers hope that discoveries along the way will continue to have an impact on our
08:37appreciation and protection of the species we share the planet with. We're not the only ones
08:44on the planet who can communicate, who care about one another, who have thoughts about the past and
08:51about the future. They also have a right to be here and a reason for being here.