Claire Corlett

Fish Food, Fish Tanks, and More
Florian Engert (Harvard) 1: Fish in the matrix: neuronal activity & behavior in virtual environments

Florian Engert (Harvard) 1: Fish in the matrix: neuronal activity & behavior in virtual environments


Hello, everyone. It’s a pleasure to be here and it’s a genuine pleasure to do this, also. My name is Florian Engert. I’m a professor at Harvard Biolabs doing neuroscience. And my main interest is in understanding how the brain generates behavior, and generates everything. It really makes us what we are. The story I’m gonna tell you today is an introduction to circuits, behavior, neuroscience. And it’s also a story of the larval zebrafish as a model system to do circuits in… circuit neuroscience. So, here is the hero of today’s talk. It’s a larval zebrafish. And the real power of the larval zebrafish is it’s small, it’s translucent, and it is a vertebrate. So, because it’s small and it’s translucent, it allows modern optical technology to be applied to the whole animal. And as you can see here, this fish, which is 4 millimeters in length, still has pigment cells up here, and then along the back, and down here as well. But we can use another power of the larval zebrafish, namely that it’s a genetic model system, to generate mutants that are really completely translucent, like this nacre mutant, here. And that it’s really an animal made out of glass. The only pigmented… pigmented thing left is the eye. And there they need pigment… pigments to see. So, what we can do with that animal is we can observe neuronal activity in a living brain. And we can do this at the single-cell resolution throughout the whole brain. And we can do this in an awake and behaving animal. And that’s what I’m gonna tell you about today. Also, this is often a source for confusion… larval zebrafish are not insects, they are card-carrying vertebrates. And as such, the brain structure of a larval zebrafish, up here, is very similar to the brain structure of mammals. This is a mouse brain, down here. And I’ve put colored lines, here, that point to the homologous brain regions. But it’s not just that the brain regions are similar. It’s also that the neuronal cell types, the neurotransmitters that are being used, and the neuromodulators are all basically identical between larval zebrafish and mammals, such as us… such as us. And another thing is that the… well, one thing you should know, of course, the scale. It’s not to scale, here. Larval zebrafish, as I told you already, are tiny. And as such, they are amenable to modern optical technology. So, here is a real image of a larval zebrafish, a live larval zebrafish. It looks green. And it’s supposed to look green because all its neurons are labeled with a GFP variant — a green fluorescent protein variant — in this case, it’s GCaMP5, that allows us to monitor neural activity. Before I go into that, I’ll show you first a brief movie, where we can just see what these neurons inside the brain of these animals look like. So, here we have a two-photon stack, where we… at the moment, we are looking at the top area of the fish’s brain, so the top slice. And when the movie plays we’ll go down, we’ll go deeper and deeper and deeper into the fish’s brain. And as you can probably see, all of the neurons are labeled. Here are the eyes, down here. Now we are going up again. And you’ll see all the neurons up here. There’s the cerebellum. This is the optic tectum, here, all the cell bodies. The black holes in the middle is the nucleus, which is not labeled. And those black things that wiggle through the brain are the blood vessels that also don’t contain any GFP. And the… the other thing we’ll see… now the movie loops… are the tracts, sort of the fiber tracts, that project down into the hindbrain and into the spinal cord that then elicit behavior. And I want to remind you that this is a live fish that has its head fixed in agarose, in Jell-O. And we can now do anything to this animal and observe neural activity while it is interacting with the world. Neural activity will be observable by an increase in fluorescence. This is something we’ll see in the next movie, here. This is again a larval zebrafish. This is the top down view, here. Here is a side view. And here is, if you look at it from… from the front, sort of a coronal section. And the movie is courtesy of a former postdoc and good friend of mine, Misha Ahrens. And what you see here is, now, neural activity throughout the brain in an awake and behaving animal. And that is truly amazing in its potential. So now, we have the ability to record all the neurons online. What you just saw, then, right?, this big flash of activity that just went on. I have no idea what it means but… which is ultimately the next question that I’m gonna get to is, what are we gonna do with this, now? Now that we have this amazing power. Which, really, I think is the most eminent, the most burning question. So, we have the ability to record all the neurons in an awake and behaving brain. What are we gonna do with that? Maybe one question that is still lacking… not the question… an element that’s still lacking here is the connectivity. So, what we are observing here is just the activity of all the neurons. Another thing we would like to know to explain how this activity comes about is the wiring diagram, the circuit diagram that underlies this activity that’s generated by the sensory input, but also spontaneously within the brain. And one modern and very attractive approach to that — it’s called connectomics — is to use electron microscopy to get the circuit. Meaning get all the wires, all the axons, all the cell bodies, all the dendrites that make up this brain. And because the resolution required is so high, this is a truly daunting task at the moment to get the complete wiring diagram at electron microscopical resolution. And here, again, the power of the larval zebrafish, namely its small size, comes into its own. So, because it is so small, we can do sort of a first shot at the connectome. This is shown here. These are all the myelinated axons that go through a larval… larval zebrafish’s brain, here. And the… this is of course done in a fixed, in a dead fish. But the power is that we can do this kind of analysis on the same animal that underwent the behavior and whole-brain imaging before. For this… in… in this case I have to admit it’s not the same animal as in the movie you’ve seen before, but in principle we can do this, now. We can take the same animal where we do the functional analysis, the imaging analysis, and we can do post hoc, get the circuit diagram and overlay it on the functional activity. And as such, get a complete circuit diagram that we know produces the activity. And the activity in the brain produces everything else that animals do when they interact with the world. Here is a beautiful rendering. It’s a rotation of this wiring diagram. And what we are trying to do, now, is establish a pipeline, where we combine this kind of analysis with whole-brain functional imaging and behavioral studies. Right. So, that’s sort of the first part, is, this is the technology we have at our disposal. And I think it is truly awesome. It is also, I think, very advanced compared to what one can do in other vertebrates or even mammals. In a mouse, or in a monkey, or in a human, this is completely impossible. You can do similar things in drosophila, in insects, and you can do it in nematodes, in worms, but for a vertebrate I think this is really the cutting edge technology. And really, the challenge now is, what do you do with this power? What… what are we gonna do? How are we using all of these data that we can collect, all of this information that we can collect, how are we gonna use this to explain what we really want to know? And that’s how the brain works, right? What I’ve shown you so far is not explanations, it’s just maps. And maybe it’s important to point this out. A map is not an explanation. It’s not. And then you ask, so, what is an explanation if it’s not the map itself? But that’s sort of a more harder, more philosophical, question. When do you know that you understand something? Or what… what even is understanding? And that’s complicated. I’ll try to get at that maybe at the end of this talk, and maybe later during the following talks. But the… the challenge, now, really is to take all this and then put it into… into a context that makes sense. And that gets us from information to knowledge to understanding. Yes. The secret to that turns out to be behavior. In my lab, and I think in many of my… many laboratories of my colleagues, what we’ve realized in the past 10 years, really, is that the main challenge is to describe and analyze behavior before you go into the depths of the brain, before you start recording neural activity. It really makes a lot of sense if you first figure out what this animal is trying to accomplish. What is the behavior? What is the goal? What is the ultimate and what is the proximal goal, right? Why is this evolutionarily adaptive, this behavior? And what exactly are the algorithms that the animal is using? So, what I’ll do you… what I’m going to do next is walk you through several aspects of zebrafish behavior. So, that is really the question, now. These larval zebrafish, they are five to six days old, after fertilization. What kind of behaviors can they actually do? So, what do fish do? They swim around and hunt prey. And so here is a movie that shows… on the further side, here, is a movie of a fish swimming around. You can see it’s converging its eyes. It’s sort of turning into a true predator with converged eyes. And then on the right, here, you see a skeletonized version of the movie. And you also see the prey items surrounding the fish, which are paramecia, small unicellular organisms that they pursue, hunt, and eat. Another thing fish do is they avoid dark and seek the light. It’s a behavior called phototaxis. And here is an example, how we can study that in the lab. And what you see here is a tiny larval zebrafish. And we project from the bottom of the dish one part that is bright and the other one that is dark. So, this is the bright side of the dish. This is the dark side. And the trick that we are using here if we follow… with the stimulus, we follow the motion of the fish. So, wherever the fish is, and you’ll see this once the movie plays, it’s field of view is dissected, where, in this case the left side is light and the right side is dark, and the stimulus gets updated. And so you can see how the stimulus, now, this bisected field of view chases the fish around. I’m guessing you can see the fish is always turning left. It’s always turning left into the bright. He never really succeeds because the black dissection chases him. And we can do a detailed behavioral analysis of this. This is just a bunch of left turns into the white side that is… the fish will repeat endlessly. So, this is sort of an easy closed-loop way of doing phototaxis. This idea, or this technology of using closed-loop technology, such that the stimulus actually depends on the behavior of the animal, this will reappear during my talk repeatedly. So, you better get used to that a little bit. Phototaxis. Another behavior that they do extremely robustly is called the opto-motor response. And this means that fish will just follow whole-field motion. So, if there are stripes moving along the bottom of the tank, then the fish will follow along. So, the movie shows, here, a bunch of larval zebrafish that are just following the stripes, here, which are visualized at the bottom of the screen. And as you can see, if the stripes move to the right, the fish follow. If we converge the stripes, again, the fish will accumulate there, again. And now when the stripes go uniformly in one direction, when they move to the right, they follow the motion. You might ask, why do they do that? What’s the point? What is adaptive about this? And what we believe is this is a very robust strategy if you don’t want to get swept away with a moving body of water. So here, in this case, the stripes are moving and the water is still, but you get exactly the same impression if the body… if the water is moving with the bodies and the stripes are standing still. And then this strategy of following what you perceive as the bottom is a very good way of just standing still, holding position in a moving body of water. So, in a way, this is a form of rheotaxis. And this is adaptive for larval zebrafish because, we assume, they don’t want to get swept away by the river into other areas, unknown territories, where there might be bigger fish that might eat them, or simply avoid being dragged into the unknown. Another behavior that they do, like almost all other vertebrates and even insects, is the optokinetic reflex. In this case, it’s a simple behavior where the fish simply follows whole-field motion, again, but now with eye motion, not with body motion. And you can see this, here. This is a larval zebrafish that’s put into a drum that surrounds it, like that… like this. And the drum rotates and you can see that the fish tracking the stripes on the drum with its eyes. So, it’s moving to the right. The eyes and then we’ll cycle… cycle back. Here’s a really cool other behavior. This is escape from potentially threatening predators. In this case, it’s a dragonfly… a dragonfly larva, which is much, much bigger. Here it looks really threatening and this is the tiny larval zebrafish, here. It’s a slow motion capture of an attempt of the dragonfly larvae to catch the larval zebrafish. It’s a movie courtesy of Joe Fetcho, one of the pioneers in larval zebrafish study. So, it’s sped up… or, slowed down dramatically. This is one of the fastest attack motions that we know in biology, this attack of the larval… the dragonfly larva happens within a few milliseconds. But nonetheless the fish can escape. In this case, it is a touch-evoked escape response. Fish will also run away from visual… visually, a pattern of predators. This can be simply a looming stimulus. And here’s an another closed-loop example, where a looming stimulus will show up from the bottom, projected with a video screen, and it’s gonna chase the fish, as so. This is also gonna loop, now, and you can see the fish is running away, swimming away in a directional fashion to escape this looming stimulus. So, this is something that they do robustly. And also, as you can see, something we can study in the lab. Now, I want to get back to hunting, because this is I think one of more sophisticated reflexes that they establish. And this is a movie that Adam Kampff, when he was a grad student in my lab, took of zebrafish. It’s a slightly longer sequence. And then there’s the appropriate music going along with it. What you see here is the paramecia, again, the small unicellular organisms that are swimming through the dish. And a bunch of zebrafish that are chasing them and hunting them. If you look carefully, you sometimes can see how they converge the eyes before they attack. And another thing I’d like to say is that the hardest part about making this movie is getting the fish to swim along with the music. But as you can see, we’ve managed to do that. And the movie ends with another optokinetic reflex where the fish focuses on the paramecia he’s about to attack. Back to the opto-motor reflex. So, all of these behaviors I’ve showed you so far are in freely swimming animals. Ultimately, if we want to apply the power of two-photon microscopy, we have to go to head-fixed animals. And here we see a zebrafish that is head-fixed in agarose. Yeah? Here you see the boundary. And if I play the movie, you’ll see two things. You’ll see the fish trying to do the opto-motor response. So, he’ll try to swim along with the… with the stripes. The other thing you’ll see is this closed-loop such that he’s successful. The very moment he’s swimming, the motion of the stripes is slowing down. So, this is, in a way, a fish that’s being dragged backwards in a moving stream, and he’s now swimming forwards to counteract this backward motion. So, closed-loop. This is, in a way, very close to virtual reality already, where the action of the animal actually controls the stimulus it receives. This is not passive viewing, like television. It’s more like active viewing in a first-person computer game. I’ll get to that in… more details about that in a… in a minute. The adaptive advantage of this opto-motor response — I alluded to that already briefly — is to avoid being dragged into larger parts of the river. There is another element, here, and that is zebrafish evolved in northern India and Pakistan. It’s hot there, so they also need to get out of the sun, but usually it’s, as this delta here shows, rice paddies, swamp lands, that, with the change of the seasons, get drained. And what larval zebrafish, in particular, also need to do is avoid getting drained out of your pond and into the… into the mud. So, this is another argument, sort of a first-level argument for why they should do this behavior, this opto-motor behavior, this… avoid getting dragged by waters. So, the ideal preparation, really, as I said, already is tethered. You’ve seen already the larval zebrafish tethered in agarose. And what I’ll tell you next is how we can turn this, really, into something equivalent to the Matrix. I assume most of you know what the Matrix is. Those in the audience who don’t, I strongly encourage you to watch the movie. So, this is something you’ve seen already. The fish tethered in agarose and he’s behaving, he’s wiggling his tail. And we can plot on the other side, here, we can plot the tail angle while it’s behaving. We can do a little bit better than that. The problem here is that we still get motion artifacts, also the head wiggles a little bit. And we want to get rid of that. So, what we can do, we can paralyze the fish with alpha-bungarotoxin. So, the muscles don’t move anymore. The rest of the brain moves perfectly. And now we can record neural activity from the nerve root, the neurons that run down the spinal cord, from here to here. And there are two electrodes, suction electrodes that just suction onto the skin. One on the left side, one on the right side. And then they record, now, they pick up the neuronal signals that go to the spinal cord and activate the muscles, if the muscles are not paralyzed. This is what this looks like. So, this is the fish simply thinking about wanting to swim, but nothing really happens. But we can record this online and decode the intended swim motion of the fish, right? So, without the fish moving anything, we know exactly what he wants to do. You see the parallel of the behavior, here and here. So, what we can do now is take this… called fictive swims, decode them online, and play them back into video screens that surround the fish, such that the fish can now interact with a virtual environment just by neuronal signals out of his head. That’s really very close to the Matrix, now. And here is an example of that. This is the bottom of the tank, where you see all the… it’s just random doodles. This is also something that Misha Ahrens’ did when he was still in my lab. And here is the larval zebrafish. Remember, the fish is suspended, it’s fixed. The only thing that’s moving is the environment, here. And what we’ll see here, once the movie plays, is the activity in the left and the right channel. And here is what the fish is seeing. If I play the movie, what you’ll see is from the point of view of the fish, his… the motion of the environment. Here are the two behavioral channels that we are recording that get translated into this motion, here. And what you’ll see is the fish is trying to stay on these bright islands — phototaxis, I’ve talked a little bit about this already. So, he’s hanging out on these red islands for a while. But occasionally he’ll take heart and cross one of these dark gaps, and swim over to the next island. The fish is paralyzed. It’s not moving. It’s controlling all of this purely by neural activity. So, here we have a preparation, now, that is perfectly paralyzed, immobile, ideal for doing whole-brain imaging. But nonetheless it is interacting with the world, with a virtual world that we are simulating for it. In the next slide, you’ll see a trajectory of… over 15 minutes. This is what the fish was doing over the course of this… of these 15 minutes. So, they really behave perfectly in these virtual environments. And if you look at traces like that, you can’t really distinguish that from a real fish swimming in a real environment. And this tells us that they adapt perfectly to these new situations. And this really allows us, now, to combine behavioral studies with neurophysiology. With that, I’m coming to an end. The last thing I’d like to say is… and to put another word in for our larval zebrafish. The conventional fear of not just the scientific community but the community is large is that there’s a clear superiority of mammals versus fish, as illustrated by the slide, here. In order to correct this notion that you might have, I want to show you another movie that also illustrates an interaction of a mammal with a fish. It’s again a seal, only the fish is a little bit bigger. So, what we see here is a seal swimming off the coast of South Africa and its interaction with another fish. In this case, it’s a great white shark that was filmed by David Attenborough in one of his spectacular documentaries. I hope this will convince you that it’s not always the mammal that’s superior to the fish. There are certainly cases where this… the roles are reversed. And I’d like to think that larval zebrafish can play a similar role when it comes to science. Lastly, before I finish, I’d like to thank all the people who actually did the work. I was standing mostly on the sidelines and applauding. And here is a photo, two photos of the lab. Mostly now, this is slightly older people who did the work at the time. If you’re interested in all the names, please go to my website. It’s easy to find and look at the names. Other than that, I thank you very much for watching this movie. And I hope you’ve learned a little bit. Thank you very much.

6 comments on “Florian Engert (Harvard) 1: Fish in the matrix: neuronal activity & behavior in virtual environments

  1. Fish in virtual reality! This is so fantastic! 1st year Neuroscience student here. I'm inspired! Going to watch all your videos.

Leave a Reply

Your email address will not be published. Required fields are marked *