time in DC, and I'm going to talk a little bit about human performance on, or the use of VR for human performance. This is my disclosure. I'm the founder and CEO of PrecisionOS, which is a surgical software simulation platform. So my objectives for this talk are going to be the following. I'm going to talk a little bit about the past, I'm going to talk about the present, I'm going to talk about the present future. Now we all know this famous person here who describes see one, do one, teach one. When you dig a little bit deeper in Halstead's approach to education, this is sort of the model that he embodied. Residents worked about 110 hours per week, 24-7. They were discouraged from marrying, and he had a very authoritative approach to his education, which was a dominant teacher with a docile resident. And his progress for the residents was based on relationships. He usually took in eight residents per year. One graduated as a professor, and seven were left poorly trained. Who really changed the model of Halstead was this gentleman here, Dr. Churchill, who is actually from Mass General, who actually reported as Halstead's approach as being anti-intellectual, and he needed more flexibility, more master surgeons with less authoritarian approach, and the goal was proficiency in his mind. Now when we look at the skill model acquisition triangle here, you start off as a novice when you're resident, and you hope to graduate or finish as a master surgeon when you finish your career. The problem with that approach, of course, is we have variable outcomes throughout this, and the other big challenge on the left is that we have no ability or data to measure an individual's performance. So why is it important to be an innovator as a master surgeon? The reason is because that's where people start to think about how to solve problems differently when you become that level of expertise. So this is a video that I pulled, which shows a current state, or the previous state of long-term high jump. We can probably turn the volume down a little bit as well, please. If you can turn the volume down just a little bit, that'd be great. So some of you may never have seen that form of high jump, but that was how it was traditionally done for decades, and the problem with that approach was everyone peaked at around seven feet, just over seven feet of high jump, until this gentleman came on board. His name is Richard Fosbury, and he described this at the 1960 Olympics. When the newspapers first saw Fosbury jump before the games, they said he was like a two-legged camel. Thank you. Very well done. They dismissed him as a curiosity, but this camel went through the start of the competition without knocking the bar off one. So we all know this is to be common right now, and what he found, the reason it's worth, is because the center of his gravity would actually go underneath the bar, and he pushed limits of peak high jump to about eight feet. So when we talk about algorithms of AI and machine learning, there's been great talks given about explaining this. This is a really good book that I recommend for those who are interested, called Prediction Machines, and in this, they dissect down how people like us or anybody make decisions. So when you receive a series of inputs, that input will eventually give you a prediction, and this is where artificial intelligence comes into play, where you will need less inputs over time to create a prediction. What happens when we receive a prediction, however, as surgeons, is we bring in the idea of judgment, and judgment is human intelligence, which cannot be replaced by machines, which actually allows one to build and create a decision tree. So experts will look at a prediction and think of decision trees on how to manage a problem. That then leads to an action, which is in the operating room, you are constantly making decisions that you act upon, which we currently have no way to measure in individuals' performance. That action leads to an outcome, which we have several patient-related outcome measures that we embody and use. That training then reinforces the prediction model to make it smarter, and also increases your judgment as a surgeon. So what is performance? It's the process of carrying out or accomplishing an action, task, or function. But we look at surgery. What is performance in surgery? So it's akin to playing an instrument today, but you can't hear the sound of that instrument for several months, i.e., the outcome. But what if we could quantify your activity today of the decisions you would make actually in the OR and quantify them in a way that's measurable and meaningful? Now, surgical education, we'd all agree, is exceedingly complex and multifactorial. So when we think of arthroscopy skills, and I do 100% shoulder only as my practice, when I think of arthroscopy skills, and I hear this from the residents, is we need to learn the steps of the procedure. But when I dissect through and think about what I understand about arthroscopy, it's instrumentation, when, why, and how, comprehension of normal anatomy, a really deep understanding of it, and interpret abnormal anatomy on pre-op imaging, and then interpret that abnormal anatomy through a scope, and then interpret, if I'm using a C-arm, I don't do hip scope, but some of you do, now you have to interpret the 3D pathology through a scope and decipher it on an intraoperative C-arm, paralyze those two views, and then when do you apply the expert maneuvers that you may have learned from a meeting like this? So, some of you may have heard of this technology, virtual reality. It's a fully digital immersive experience that you can have using this off-the-shelf hardware, as shown here. So I'm not a hip arthroscopist, but I'll show you this video. So this is me going through a hip arthroscopy procedure in a virtual environment. And what I'm trying to understand here is what are the maneuvers that I need to do in the operating room? What does a hip do when I put it under traction, when I put it in adduction or abduction to get into the central compartment? All of this is being tracked as I'm doing it, so all the decisions I'm making are being measured. Now I'm going to try to find that CAM lesion, and I can use the C-arm as often as I want because there's no radiation, and you can see I'm having trouble targeting the CAM lesion with that spinal needle. Then I can pull the bone out of the body and actually understand where I made those mistakes, giving myself real-time feedback. Now I'm going to go here and try to resect the CAM lesion. So this is all virtual. I've got a bird in my right hand, I've got the scope in my left, and I'm going to go ahead and resect that CAM lesion. And what I'm going to do is I'm going to see if I can reinforce that with taking fluoro during the X-ray or during the actual procedure, but then I can pull the bone out of the body and then draw a link between the two-dimensional image and the three-dimensional model. So this study, this was actually looked at, you know, the question that we get asked all the time is, does this work? This is a study done by a now-fellow, Omar Rahman, who looked at, can I take two groups of residents, put half of them in a virtual reality environment, and then half of them on VirtaMed, and then evaluate their performance on a cadaver? What he found was quite interesting. So both groups then went on and did a cadaveric hip arthroscopy, a diagnostic scope. Those videos are recorded, and then those videos are recorded and evaluated by expert hip surgeons that are many of the co-authors here. And what they found was quite interesting is that typically both groups were identical with respect to OSAT scores and ASSET score, both of which are validated outcome scores. And this is both on the VirtaMed and on the VR side. And the time that took them was almost identical. Statistically there was no difference. But the one difference that was notable was the cost. So it was 130 times less expensive compared to using a standard simulator. So when you think about the investment of time and money into your education, this is something to consider. Now, this is a study done out of Queens in Canada, where they looked at novices, meaning completely somebody who's never done an arthroscopy before. And they said, what's the impact if we train them on the image on the left versus doing virtual reality? And what they found was actually quite interesting. So the practice time for the benchtop group was about six hours. And when they measured them against bench lines is where you are as a PGY-2, PGY-3, that group got to about a low level PGY-2. The virtual reality group actually practiced and achieved peak performance at about 70% less time. And they were at just under a PGY-3 level from able to do a diagnostic arthroscopy. So increased skill, more efficient learning and outcomes. And summarized here, the benchtop group advanced to a PGY-2, whereas the VR group advanced to a PGY-2 slash 3, 70% faster. Now, the next question is how do we transfer knowledge, but importantly, how do we acquire it? So we do a lot of this in the operating room, and I'll talk about some other studies related to that. Could viewing position positively or negatively impact your surgical experience? So if you're a resident or a fellow or even another surgeon watching another surgeon, depending on where you stand, could that impact what you see? So this is just an illustration I got from the internet, surgeon in position A who's actually operating, surgeon in position B who may be viewing or assisting. What is the ideal position for someone to watch or assist where you're actually trying to learn? So what we did in this study is we took eight cadavers, and we positioned the cadavers to have a shoulder replacement or actually a glenosphere placed into the cadaver. The resident group actually was able to practice in virtual reality, whereas the expert group watched where the implant was going. After they both provided their numbers with respect to version, tilt, and position on the glenoid, those scapula were skeletonized and put into a CT scanner where we could actually accurately determine the position of the glenoid. And what we found was also quite interesting is that here, this is the actual position of the glenoid. The resident trained group actually was pretty close in the position that they decided that or thought that the version inclination rotation was, whereas the consultant group was actually a little bit quite displaced from what was normal. So being in the position B when you're actually watching a surgery and guiding somebody else or actually just trying to learn can actually be a negative influence to your ultimate learning and your ability to perform. Now this is a video I just took offline, but this is a very consistent message or a consistent video that you would see online where someone's describing how to do, in this case, a bank arm repair. The sutures go through really well. You can see it. There's some wonderful, beautiful knots, and they get an excellent repair done here. But the problem with video is we're not seeing the entire experience. So what that surgeon is doing outside of that patient experience is where the cannula is, how does he have or she have the arm positioned. That knowledge is completely devoid from that video, which is important when you're thinking about how to do that procedure on your own. This study looked at comparing video learning versus doing something in VR, and you can see it's quite a difference with respect to learning time, surgical errors, and then, of course, the cost one over the other when you're practicing on cadavers. What if you could, however, look through the eye of a surgeon? So this is me who's recorded a video and doing exactly what I would do if I was managing this hip fracture. These are exactly the positions I look when I'm actually thinking about this problem. So I put the limb back into the virtual patient. I'm practicing and understanding how that hip deforms. I'm going to go ahead and put that limb under traction to see what that does to the fracture. Now someone can emulate that and say okay, now I understand where an expert would look because that aspect of visualization is extremely important for someone else to learn as opposed to standing beside me or across a table from me. Stress is something we don't talk about, which is actually a significant impact for the psychomotor skill and performance. Now before I play this video, this is Gus Mazzocca, many of you know. There's two things I want you to pay attention to. The time it takes the resident to look at the x-ray and make a decision, and the second is the tone of Gus's voice, which I would say is pretty calm. Front, turn around, that's it. All right, so now you can get a sense of that deformity. Okay, put it back. Okay, take an x-ray. Okay, now look at the x-ray. What happened? What did you do? All right. A little bit correct, so now what do you think you need to do? So the delay in the voice we know is a direct consequence of his inexperience with this procedure. So it's showing you the position you want to be in. So think about, by looking at that x-ray, what you need to do with that right hand to mimic putting in that pin in the correct position. Okay. And really, this is the purposeful practice part of this, which I think is the most valuable. So just to let you know, it took 15 seconds before he made a decision as to where that pin was supposed to go in that environment. So stress, this is a Yerkes-Johnson law bell curve. So we know that stress of a significant amount can be actually optimal, but after a while it actually becomes impairment to your psychomotor performance. But not just your psychomotor performance, it also has an impact on other things. So this study done out of Ireland looked at two groups, supervised VR or unsupervised VR, where a mentor was watching somebody perform a procedure in a virtual environment versus that not. And what they actually looked at was ephemeral nailing in this case. They looked at the time completion, the reduction accuracy, which we can measure to the degree, the guidewire location, both on the greater trochanter as well as on the femoral neck, as well as the cumulative score. And what they found was quite interesting, which was during supervision the heart rate increased across all age and experience groups. They burned more calories and the performance decreased when supervised as opposed to unsupervised. This was consistent across PGYY levels from one through five. Now, it doesn't impact our psychomotor skills alone. It also impacts our visual acuity. This is done out of MGH, where Dr. David Labian-Kirschen looked at baseball players in the MLB and they categorized these into three different types of groups, fight, fright or flow. And they found that the highest strikeouts occur in those people who are stressed about hitting the ball, also referred to as the fright group. And of course, the fight group also had a very low percentage. Whereas the flow group that could function in low tension actually had the highest number of hits. So, how do we encourage or teach a state of flow for someone who's going into practice or early on? So, Michael Jordan obviously does two to six hours of pregame workup, which is completely unreasonable given the time constraints that all of us have, residents, fellows and surgeons. But what if you could change that and could a pregame surgical warmup actually enhance performance? This study out of Ireland looked at that over six months. They collected 150 data points. They looked at a number of procedures you can see on the right side and they had two groups of residents, both senior and junior residents, prime or warmup before the case for 5-10 minutes. And they did things on the left. You can see practice patient positioning, move the C-arm, use tools, and they collected the data points as noted. And what they found was after 5-10 minutes, the no warmup group decreased confidence was noted as the case complexity increased. Whereas the warmup group noticed no difference in confidence, doesn't matter if you started to do it in a carpal tunnel, all the way to a revision case. And when they compared senior and junior residents, confidence was equal across complex cases with the warmup. So this is an example of what I do now because, you know, even after 14 years of practice I think I can get better. So the x-ray on the left is me in VR. The x-ray on the right, or on the left, pardon me, is me in VR. So what I'm understanding here is, you know, I position the arm and I do a lot of arthroscopy is I'm plotting and seeing if I really understand everything I'm doing in the operating room when I put a scope in a shoulder. Can I do something different? Can I plot? Can I plan? What happens when I position the arm in a different way? What do I see through that scope? And of course I've got a monitor there as well that I can use. And so now I'm switching hands and I can manipulate from a 30 to 70 degree scope. What happens if I put the scope in front? So I'm constantly thinking how can I get better? Here I'm doing a combined experience where I move the arm as well as position the anchor just to see what I can do. What if you want to do ACL reconstruction? So I don't do knee surgery. This is one of my colleagues who's trying to learn how to do ACL reconstruction. I asked him to purposely make a mistake because he doesn't do knee surgery. He's more of a shoulder surgeon as well. And I asked him, where do you think the guide pin should go for the femoral tunnel? And what he does, he's manipulating the knee. He's put the scope in. He's going to go ahead and resect the residual ACL. And he's going to plan to put in the ACL guide. And he's going to mark it with an awl, you'll see here in just a second. But he goes ahead and resects the stump of the ACL. And he grabs his awl and he's going to mark it, which most of the knee surgeons in here would agree that's probably not a great spot. But then when he positions the guide pin, he's going to really take an egregious error and totally put it in the wrong place. But what he can do now is understand three-dimensionally where that guide pin was and perhaps where it should have been. So what we look at now, we're looking at a deep dive of all the users that have done the hip arthroscopy case. And I wanted to present some provisional data on this. So what we looked at is the metrics shown here, patient positioning and traction, fluoroscopy management and safe portal placement, the diagnostic arthroscopy and camera section. And what we found was there was 4,000 sessions done of a single case, this particular hip arthroscopy case, in over 40 countries. And these are the metrics we measured. You can see here portal placement, X-rays, femoral head contact and step duration. And what we found was the number of X-rays taken over those 4,000 sessions was about 54,000. And again, these are all virtual, so no one was harmed in this study. New users on average take about 10 X-rays per session. And returning users, you actually get better over time. You actually took less X-rays. And they hit the femoral head about 12,000 times over those 4,000 sessions. And those contacts on average were about 3.4. What was the most interesting early finding in this study is people who described themselves as experts versus novice. These are the findings here. So they each took about the same amount of X-rays, but 40 to 36. The attempts to localize the CAM lesion were statistically different, where you would expect. The percentage of the CAM lesion resected, however, was still not 100% for the expert group. And there's variable reasons for that. But now we have another question that we should be asking from these group of experts is, why was the incomplete CAM resection done in your case? So now we're starting to quantify and ask deeper questions on performance as to why people do what they do in the operating room. So going back to the decision anatomy algorithm, those outcomes are obviously very important. And one of the things that this case illustrates is, here's an example of a surgeon in practice, somewhat of my vintage, who tried to pin this slipped epiphysis, and he put the pin in the wrong place. You can see based on the red and green arrow. This was published in the Journal of the American Academy. After two attempts, he had transferred the case to another hospital where the PGY-4 actually practiced in a virtual environment. And what he found was the following. This paper shows that the number of X-rays he took over time decreased. His training time actually reduced, meaning he was able to develop skill and proficiency in his skill in an expedited way. And what he was also able to quantify was his score. So his score went from 7093, which is based on several metrics you would use in the operating room. And then he went ahead and pinned it by himself. What was interesting, though, is actually comparing the index procedure of the original surgery to the resident surgery, he actually used almost eight times less radiation. So just to summarize, skill acquisition is a journey, certainly not an endpoint. Performance and practice in a low-stress environment does equate to execution in a high-stress environment. We have a lot of data on non-technical skill, OIDs, MCQs, et cetera, but we have no data on technical skill. I think this certainly embraces a generational change with new model of education. And is it possible that tomorrow's young surgeon would be vastly experienced more than today's expert? Thank you. Those are some very interesting presentations. I learned a lot. I think because of the time, we should conclude the session and not do the panel portion. But if some of the authors are willing to hang around for a few extra minutes and entertain some questions from the audience, just personally, that would be great. Thank you.

virtual reality

surgical education

skill acquisition

VR training

hip arthroscopy

ACL reconstruction