»» Great papers and congratulate you all. We have a few minutes. I know we're running a little over for some questions from the audience. If someone has some questions for our speakers, looks like we have one. »» Hi. Joe N. HSS. This one's for both Dr. Liu and Dr. Ingbertson. So for your images, I know both of you sort of noted in your limitations about the need for external validation of your models. Is the other limitation necessarily going to be based on like the hardware and software to produce like those high resolution images besides the actual just logistics of aggregating all that data? »» Yeah, I could probably. I could speak a little bit with regards to our data. So the resolutions, surprisingly, the resolution requirement for segmentation is surprisingly not super high. I think because the anatomic areas that we were asking of the model were fairly simple for it to pick up. I think the most difficulty we had seen was probably segmenting that joint surface. But even in like the worst resolved x-rays, I think you can see that sclerosis pretty well with the human eyes when the machine had no issues with picking that up. And we put everything through kind of a pre-processing pipeline to get all the images into the same size and shape and resolution. I think the biggest limitation for us was probably the rotation, like the perspective of the quality of your lateral knee x-ray. We were lucky because we only had 90 images. So we could kind of just manually go through, like Dr. Kemp went through all of those and we kind of tossed out the ones that we didn't think were great laterals or could lend to reasonable measurement even by human. But we didn't have any real like objective measurement in terms of like what degree of rotation. I know in the original German study where they developed the manual measurements, I think they said 20 degrees of mal-rotation in either plane was probably too much. But we didn't really have any good metrics for measuring that. So that's probably where our biggest limitation in terms of imaging comes from. So for the MR images then, for Dr. Inbredson, are you going to need unique hardware, like a unique manufacturer, unique software to get like the actual sequencing to get those images? I mean, I know you're using 2D images from a 3D, but yeah. Yeah. So what we were looking at, our question wasn't necessarily can we create a model to predict supraspinatus pathology anywhere? It's for people that are developing models on say their own institutional data at Stanford or University of Wisconsin. Could this open access database in RAD ImageNet be available for external validation? Is it a good resource to use? So I think it just points to one of the issues with deep learning and machine learning is the need for high quality and large amounts of data. And so I think that is kind of the problem with external validation is all that manpower and hours spent curating those databases. Dr. Carey, you had a question? I have a question for Thomas. The wall study from JBGS, at the end of that logistic regression kind of process, you can actually provide a patient with a probability of healing. With the machine learning algorithm that you were using, do you get a probability at the end or just a yes or no? No, you can tell them with a level of certainty, there's a confidence interval involved, what the accuracy or what the chances of them healing are. Okay, thank you. Great. I think that's all the time we have for questions for this panel. We'll have another panel later. So we're going to move on to our next speaker.

limitations

external validation

image models

deep learning

machine learning