# PROJECT TX - Possibly the MOST ADVANCED PHYSICS NPCs in VR! (0R8Q96ukYeo, 1933s)

[Music] It's me, Ninso. Welcome back to another video. Today we are back in Project TX finally. Um, actually I'm not actually in VR today. Uh, which is an unusual one for a VR game video, but there we go. This is how it is. Um, people have been waiting for this one for quite a while, but it's not entirely like the Hold on. Let me just reset my head a little bit. There we go. It's not in the entire video. This is kind of a part one kind of thing. Uh, it's not going to be super long because this is there's so much of the behindthescenes stuff and under the hood stuff that I needed to get done. uh that uh I just wanted it to to be done first before I start building on top of it. And I have started building on top of it, but this video will showcase a little bit more of the the grunt work under the hood. And uh yeah, so let's get into NVCs. What we're looking at in the in the actual scene is not going to be very complicated. It's just going to look very simple and uh not stuff that you're you're used to here. But let's have a look at it very quickly. Uh just quickly running the game. And there we go. Right. Okie dokie. There we go. And uh what we're looking at here is just an NPC following this bowl. Right. So this is this this is just a target bowl. This NPC here is an actual full ragd doll. Now, let's talk a little bit very quickly about how uh how ragdolls are built in in VR physics, right? Um when it comes to VR physics, you'll you'll have seen in plenty plenty of games that characters have and I've mentioned this so many times before, characters have a bit of a flimsiness to them, right? Either they are very very ragd doll accurate and they're able to kind of stand on their own and they waddle about and they kind of very very mildly walk and trip over basically everything including ramps and stairs and everything. Or the alternative being really really cheaply designed rag dolls that are effectively not true uh rag dolls in terms of their joints and everything. They kind of look like ragdolls for the most part. And this is this is very very common, especially in VR. And in reality, they're just using what is normally called an animated ragd doll or something that I think we were calling partial ragd doll in in my previous job. Um, which is where you have if all of your bones kind of pinned in place in the world space. And when you push an arm, for example, it kind of flexes weirdly where the hand stays in pivot position and the arm kind of moves out of way. Everything looks like all of the bones individually are trying to get to their positions, which is very, very awkward. That's not the case with my ragdolls, right? What I'm trying to do here is the the more accurate full physics ragdolls. These are driven by muscle systems which means each and every bone here is actually driving itself. It's uh it's moving itself. This here ball I can move via this uh internal internal ball which I can move and uh this character will literally follow the the ball. Right now, when he walks around, he's actually driving himself using not just the not the feet aren't directly driving his motion, but they are holding him up, right? Which means I can push him around. This is an actual he does walk away from things as well. This is an actual physics character, right? He can be pushed about and you can see here he's actually waddling about and he actually does kind of stumble and everything, right? And this is this is really nice. It's very standard stuff. You've seen this plenty of times before. But of course, he's also able to tip over and get knocked over. Right now, this is very again very standard stuff that you've been seeing on my in my Ragdoll systems, but under the hood, there's actually a lot more going on. Now, when we look at this character here, I'm going to close up everything except for this here. Now, actually, yeah, this will show you a few more things, but you'll notice here there are three points. Uh, three, let me just make sure I'm not covering them up. There we go. There we go. Beautiful. There are actually three sets of IDs under this debug text here. Now, when I move my character, when I force him to move, here we go. Just make him walk. You'll notice that the action state has changed to walk to, and then it's now changed back to idle. Right. What's actually Oh. Oh, I've I've just told him to tip over. Whoops. Sorry, dude. Sorry, dude. I I I was supposed to disable that. I just made him fall over. Right. Okay. Um, so under the hood, what's actually going on, if you think about the physics based uh physics based ragd doll, it ignore the ragd doll. In a normal NPC system, what you want to do is you want to have your character Yeah, I probably should have taken out the console. You want to have your character um basically perform an action and then make a decision as to what the next action you perform. Ordinarily with something that's a much more simple concept, for example, a very a straightforward uh behavior, um if you had an a combat game, right? Your character would spawn, its objective would be to move towards the player, attack the player, right? So a very simple action chain would happen. The character would walk towards the player or walk following the nav mesh. And then when that action completes, it would switch to the combat stance. And when that action completes, it would decide randomly to choose to attack, defend, or there might be some other parameters for defend. And maybe there's just a maneuver. These are very, very simple maneuvers and simple actions. And the action chain itself can kind of decide what to do. But in reality, what if you had a situation where you wanted to use the walk to action, but instead of um instead of switching to a combat stance, you want the character to just stand around. So, for example, if I press one of these buttons, which of course I'm going to have to do physically because they're made for the character to push down. I'll push that button there, and that actually switches him into a different behavior, which is walk around. Right? So now he's in the wonder state. I'm going to hide this pole here. And now he's in his wonder state where he's just wandering around in this in this area. And what he's actually doing under the hood is switching from idle to walk action. But how is it making a decision then? Because the walk to action will then make a decision as to where to go. Right? And what I did is I wanted to separate the decision-m process from the actions themselves. And that's why when you look over here, you see that the character has not only a physical state, which I'll explain in a moment. The action state, which is the actual idle and walk to, and then the goal. The goal is a separate state machine. To explain what a state machine does, it's very very simple thing. Um, a state machine is simply a machine that has it is a finite state machine, right? where you have a finite state uh number of states that uh an object can transi transition to and each state can transition to another state. It's very very easy decision making system. Uh Unity uses it for animations where you can transition from one animation to the next and you have some parameters for the transition as well. Um, so we use a state machine regularly for for handling NPCs in in any regular game, even nonVR, even non-physics, where you just want the character to switch between actions and and there you go, you use a state machine. What I've done is I've duplicated that and I've used a secondary state machine for handling the goal. Right? So now we've got a top level goal machine which effectively decides what the player should be doing and it tells the player when to switch between actions, right? So the goal is doing all of the logic and then switching to the action system, right? And and providing the actions um providing what action the play the character should do. Now, to top on top of that, to top on top of that, that makes perfect sense. To go on top of that as well is the physics handling. When the character gets knocked over, let's uh get this uh ball out here. And you'll see the physical state has switched to fallen. And then he switches to balanced. Now, when he's stumbling, I'll try and make him stumble. You'll notice that it actually said stumble just here for a moment. Right now, it switched to balance. And again, he's able to actually control himself. Those three distinct states run on their own little mini state machine as well. So, what you would normally have on an MPC system is a state machine that handles your actions. Now, I've got three levels of state machine. One that handle handles the physical state of the character, whether they're in control, offbalance, and stumbling, or fallen over. and then a secondary state machine that actually handles the ex the explicit action that they're on, right? Which is just a standard action state machine. And then I've got a state machine above that which is handling what all the decisions are. When should it actually switch between all the actions? And that's the goal, the overall mission of the character, the the the objective of the character. What does the character intend to do? In this case, he intends to wander around. But when I press this button on this side here, wh There we go. And click, you'll notice that the the debug text at the bottom here changed. And now his state, his his goal is in follow. And you'll notice no matter where the ball goes, he will intend to follow. But here's where things get even cooler. Because not only does the character follow the ball, but he can only follow the ball when the ball is in a place that he knows of, right? He didn't see the ball go past him. And now, because he hasn't seen where the ball is, he doesn't know it's there, right? He doesn't know where to go. He doesn't know where the ball is. As soon as the ball comes in front of him, let's uh let's try and get the ball over here. As soon as the ball comes into view, we'll just leave it here. He'll turn his head. And there we go. He noticed it. As soon as the ball came into view, he started walking towards it. Right. But that's not a very simple uh you know, look at and uh uh see, therefore go system. Actually, there's more going on. What I wanted the character to be able to do is actually build on a knowledge base. What's going on here in reality is he sees the ball and that adds a new piece of knowledge to the knowledge system here. He has what I've called a memory time. That's the t the length of time uh before the where the the item has to be not is not seen where he will forget that the object exists. The knowledge base is actually built on a memory, a set of memories that are generated upon seeing the object. So this is another complex system under the hood that's always going on. You'll notice there's a little text here which is called breathing space. That's nothing to do with the object. As he moves very close to the object, he'll actually move away. So he's always he's always having a little bit of a space where where he doesn't want to he doesn't want to be standing too close to it. Right? So even though he'll move towards where the object is, he doesn't want to stand too close to it. So if it moves into his space, he'll move back. Right? Now he'll also do that when he's too close to a wall. And that's what the breathing space value actually is. He's he's detecting walls in front of him. But of course, it's actually a little bit more than just the knowledge base because when he does see the object, he will look at it and he will pathf find to it as well. Right? So now he's actually looking at the object and he's going to path find to it. Now because he has a memory time of 10 seconds, he's able to pathfind to the ball even when he doesn't see the ball. If I was to change that down to.5 seconds, watch what happens now. So now he knows the ball is here. He's seen it. And he's going to walk down this uh thing here, right? But Oh. Oh, he saw the ball. Sorry, I messed up. Oh, he's trying to He's trying to walk up there. Go on. Come on. Come on. Come on. Come up here. Right. Oh, he's forgotten where it is. He does. He's forgotten. Right. There we go. Hello. Here's the ball. Right. So, watch this. The ball is over here. He's going to try and path find to it. He's seen it. He's seen it. But now the ball is not there, right? It's not there. He's still going to path find to it and he That's it. The Where's the ball gone? Right. He was trying to pathf find to where the ball was even though he couldn't see it because he remembered where it was. Right. Uh let's switch that back to 10 seconds because I don't think that was connected properly. But the other thing is that obviously he actually pathf finds to where he remembers the ball being. So if the ball then moves away. Oh, now now he has actually done the 10 sec the memory time of 10 seconds. Come on, walk along. Walk along. You know where the ball is. Go on. Get it. Get it. Get it. Get it. Get it. He's walking to where the ball was. Uh but now he's just walking to where he remembers the ball being, but the ball isn't there. And now you can see that the memory entities the list is now empty because he's forgotten where the ball is right that's that's a 10-second uh space right this is a memory system and he's able to choose between different entities and of course each entity has its own type and it has its own additional data right now it just has a an overall position but when he's actually looking at characters entities have a different set of of values and character entities have where the head is so he'll pathf find to the position of the character, but he'll look at the character's head. And you'll see that his actual look at state is quite nice as well. He's got a lot of flexibility in the way he actually looks at things. And of course, if he gets pushed off, if he gets pushed off, oh, he managed to stabilize. Okay. Anyways, so let's get him to move over to this side, right? And before we do that, we're going to switch on uh here we go. Is it the neck? No. Ragd doll head. I'm going to go to the view cone and switch the view cone on. So here we go. Now we can actually see what he's able to see. And I'm also going to switch on the gizmos, which tells you what the path finding is deciding. Right. So here we go. He's got this blue line that comes out and he has seen the ball. There we go. He's noticed the ball. Now he's going to path find to where the ball is. Right. So there we go. He's actually walking up the ramp. You'll notice that he's able to stablyily walk up the ramp and he's able to walk backwards and walk down again as well. If I move the ball here, he's now trying to go down the ramp there and he's going to go around and actually get to the ball. But let me show you the ball. There we go. The ball's up here. Let him walk up here again and we'll have a look now at some of the actual stability of his physics here. I've been working so much time here. And I know this is going to be a bit of a short video. You're not going to get to see the exact stuff that we've got going on, but my goodness, it's so fun. Let me just push him backwards a little bit. Push him backwards. Oh, he's falling off when he tries to get up. Oh, he did try to get up. Okay, he's got He might struggle. Oh no. Oh. Oh no. Oh dear. Oh dear. Oh my. Get up. And now he's forgotten where the ball is. It was actually more than 10 seconds. If the ball gets just pushes him behind there, he'll just get pushed about. Boom. [Music] Boom. Oh, I love knocking NPCs about. Right. So, here's another thing I wanted to talk about as well. Let's talk a little bit about the physics. The actual physics of the character when he's actually balancing is actually more than just a simple upright check, right? It's actually checking the matching value of how close he is to what he wants to be uh doing, right? So, in terms of his overall tilt, he's trying to match a certain value. And that means that when he's getting up, he's able to still constantly check whether he can fall, whether he needs to fall over again, right? He checks his balance even though he's kind of tilted sideways. So when he gets knocked over, which is this is not the right place to be doing this, but when he gets knocked over, he's still able to actually get pushed back again, right? And we'll see this when Oh, actually, we can just do this. Boink. There we go. And now, let me just pause the game very quickly. I'm gonna move the I'm gonna move the ball over to here. And now there he tried to get up because he tried to get up and he was once he started getting up, he was out of match with the intended position that he was supposed to be. As a result, he fell over again. That means that even though he's in his animation, halfway through his animation of getting up, he's still running all the balance systems. So, you don't get any of that awkward half um where the character gets held by his foot and he completely gets up vertical and then falls over. There's none of that. This character, if he's getting pushed down, even though he's fallen over and he's in a get up animation, he will still not be able to get up because he's halfway through the animation. Watch him there. He tried to get up. A little bit of floatiness, but he just couldn't. And he managed to this time. Knock him over this way. Oh. Oh. Oh. Oh. Oh. Oh dear. You managed. Very nice. Very nice. I'm going to have to lift the ball a little bit. There we go. Knock you over this way. Right. And now he can't get up. Yes, there we go. Anyways, let's have a look at him walking on stairs. That's really nice, right? what I was just showing here. It's most people who haven't played VR games with physics characters won't understand how frustrating it is that they are just able to fully complete a get up animation when you're trying to hold them down or when you're holding them by the leg, right? It's absurd that they're able to just get up anyway, right? Here we go. Let's have a look at how well he handles stairs. You've noticed that he's actually been he's able to stumble around, right? Brilliant. And remember, he's 100% physics at all times. Full ragd doll muscle driven system. And look at him now. He lifts his feet higher and is able to fully walk up the stairs. And of course, he's able to walk backwards as well. He doesn't fall over. He doesn't fall over the stairs unless you push him backwards a little bit. Oh. Oh. Oh. You managed it. Oh. Oh. Oh. Oh. There he goes. Oh, I love it. I flipping love it. Right. You'll notice that when he tried to get up there because he was still on the stairs, he did actually stumble backwards again. Right. So again, let's see this again. Bang. Knock him backwards. And he's in stumble. Oh dear. Oh dear. Oh my. Okay, there we go. He's able to get up. He's stumbling a little bit upwards the stairs. Up the stairs. Right. And you can see here there's some steeper stairs which I'm going to let him walk to. So here we go. Come down here. Right. And again, he's able to handle stairs beautifully. Right. In fact, let's let's get his uh view con. No, you know what? We'll leave the view corn. Oh, shoot. Yeah, I pressed undo. Unity has some real trouble with undos. I don't know why. When you're running physics, Unity does not like undos. Okay, we're not going to do that. We're not going to Oh dear. Oh dear. Come on, dude. Uh, let me switch on your view again. Oh, you know what? You know what? You know, yeah, I I do want to see where the Oh, he's fallen over. Why did you fall over? Oh, it's I think it's cuz I switched the view cone on. He got He got knocked over. That That shouldn't have happened at all, by the way. Just completely nothing to do with anything. Right. There we go. Right. Come here, dude. Move along. Move along. Walk up the You know what? Don't walk up these stairs. What we wanted to see was the steeper stairs. So, here we go. This character is able to, of course, he's he's now meshing as well. So, he's able to pathf find up the stairs. And boom, he's actually lifting his feet higher to get up the stairs. And you'll see this immediately here. There you see how his feet are now lower when he's walking. And of course, the same applies when he's going the other way as well. So, there we go. You follow me. Come down here and follow the ball. And you can see now he's lifting his feet a little higher so that he lowers his body. And of course, he's looking at the ball. If you walk backwards, you can see he walks up the stairs. And of course, if I do this, yeah, he does stumble up the stairs as well. Hold on. Let's Let's see if we can knock him. Oh. Oh, I love making these guys stumble. Okay, there we go. Hey, dude. Guess what's going to happen to you now? Oh. Oh, no. He saw the ball. He saw the ball. Yeah. Oh my. I've I've played with now I Whoa. There we go. All right. Beautiful. Beautiful. We got him to stumble over and fall completely down the stairs. Here we go. Let's Let's get one more of those just for the fun of it. So you can see how nice and beautiful it looks. And we knock him backwards. Hey, come on, dude. Come here. Come here. Come here. And then knock it. Oh dear. Oh my. We're going to need a little bit of a lower ball there. Bang. And then he stumbles. Oh. Oh, he managed. And bang. Oh. Oh, are you managing it? Oh, TI is managing it. Okay, so let's do that one more time. Not too low. Not too low. Um, there we go. Watch how he handles height. Right. So, here we go. I'm going to grab his hip. Um, we can switch off the we can switch off the corn now. We'll just switch off the view cone. He doesn't need to see this thing. Um, we'll make it so that Well, actually, he's still in follow mode. He won't he won't follow anything unless he sees it. But that's fine. Here we go. I've I'm holding on to his hip now. There we go. Beautiful. Right. And of course, Unity doesn't like moving objects anymore uh in in in physics objects. So, that's really annoying. But there we go. Now, he's midair. I'm going to keep throwing him about. Right. So, here's what here's what we have when we rotate him. You can see that he's viewing the world. He's not happy about it. Watch what happens as soon as I release him. Right. He's very aware of velocity and motion. One 2 3. Oh, now he gets up. Okay. Yeah. And he's seen the ball as well. So, there we go. We're going to move you this way again. There we go. Yeah. Unity does not like moving this dude while he is in play mode, but there we go. We're going to lift you right up. We're going to lift you right up. Your arms are going to be a little bit shattered. Don't worry about it. Come on. Come on. We'll rotate you. You'll get your arm back. Don't worry. There we go. Come on. Get your arm back. There we go. Right. Let's watch this happen. So, here we go. We're going to tilt him forward a little bit. And a one. And a two. And a Squidy didly. Two. What? Damn it. I moved the camera too soon. Right. Okay. Here we go. Now, we're just going to lift him as is. We'll lift him up to here. And as soon as we let go, he's very aware of his own velocity. It's beautiful. This dude is the most advanced NPC I have built so far. Right. It is. It is truly unbelievable how far I'm taking this. Right, let's have a look what happens when there's a tire on the floor. Right, because this dude is actually quite able to maneuver around OB. Wow, I do not like the newer Unities that just can't handle physics objects in play mode. Oh dear. Oh dear. Right, so there we go. We've got a tire that is now rotated. Drop it on the floor. Get the ball. Yeah, there we go. Just push him along. Push him along. Come on. Come here. Hey, come here. Oh, well, that was uh that was expected. Right. So, here we go. You follow the ball. Follow the ball. Right. So, here we go. He won't avoid the tire. I did that on purpose. Let's see what happens when he walks over it. He nearly tripped over. Right. So, he's actually able to walk over things like that and sometime and sometimes he will trip over, right? That's it's beautiful. The point being that he's so much more stable, so much more stable than any of the NPCs I've built before. And quite frankly, I'm hoping he's one of the most stable VR physics uh characters you've ever seen. And uh there's so much more to come. Obviously, with the with these characters, the same things that I've shown in the past also apply, and I have tested this in VR. You can grab his leg and lift it up, and he'll tip over. But unlike before, this time, he actually tries to stumble and hold himself before falling over. There's just so much more that I can show on these characters, but right now, this is the best you're going to get. This is all you're going to get right now. There's a lot more to go. And in the next video, I'll actually show you in VR and we'll we'll be seeing more of the combat systems. And you can see here avoidance is a major part of how these characters maneuver, right? Just seeing where that ball is and then, you know, maneuvering to that position. It's so important that these characters are able to avoid obstacles, move towards objects. And of course, if I move this here, right, he'll walk over these physics objects, no problem. Right? So, nothing trips him over just like that. He's able to just maneuver over these small objects, no problem. Although the tire, it was high. He tried to walk over it. He stumbled. He He fell over. Right? It's there's no more of that nonsense where characters are just arbitrarily falling over. Let's press this button again. Click. Right. Right. And now he should be in wonder state. And you'll notice even though he's wondering, he's still able to look at the entities that he sees. So he's looking at that ball. And then of course once he forgets the existence of the ball, he's just going to walk around and go into idle mode, right? And now he's just going to do what I call a habit, which is look, oh, he tripped over. He's he's doing what I call a habit. And a habit is something that I can add on to the existing character control and movement and animation. And one of those habits is him looking around, right? Where he's just randomly choosing where he wants to look. And you can see I added these little pupils just to make it easier for you to see exactly how the character is looking around. So here we go. If I press this, not that button, that that button turns me into a strong character, but I'm not in the scene right now. So there we go. So, we press the button. He's now going to walk towards the ball. And let's have a look at his eyes here. Right. So, you can see here the character looks around. He's able to actually look at the ball there as it moves around. And of course, he moves to avoid it a a little bit as well. Right. It's absolutely beautiful. He's able to actually see the object properly. And there's just a lot of flow to everything. It's It's absolutely beautiful. Anyways, I've got so much more to show you soon. Very, very soon. Sorry that this video is a little bit small. It's not showing you too much of everything, but uh there's there's a lot more to come. And I just wanted to get this out there, talk about the MPCs as they are, the knowledge base, the the the memory system, and the way the physics is able to handle itself and balance and everything like that. right now. Next video you'll see me pushing these characters around and possibly even some combat. We're going to see some much more advanced character control and character systems than you would normally see where it's a very simple idea of character stands there sees sees player and immediately in basic combat with three punch animations or you know like you would see in some of the sword fighting games where they're already in combat. They have very very simple combat moves. They are very very light. You can lift them off the ground and then they're just kind of just literally like a rag doll toy a doll. It's all very flimsy and I don't want any of that in this game. You try and push a character over, you'll see that it doesn't it it doesn't play out so easily. Right. Anyways, thanks very much for watching. I know it was a bit of a long and muddly and wasted way timewasting video. It's a bit a bit rambly, but uh it was a necessary one. Thanks very much for watching. I will see you in the next one. Uh, goodbye. [Music]