Grinding from front right

I have a very noticeable grinding noise that comes from my front right side. Seemed most likely to be a CV joint as I couldn’t think of what else it could be. The thing is that I couldn’t necessarily change it by turning, driving different speeds, or anything else. In fact the noise comes and goes to a degree.

I jack it up and turn the wheel – the grinding noise isn’t real loud but still present. So, I take it apart and it certainly feels like the inner joint is binding a little but there is no physical damage I can see. The boot isn’t damaged and the grease was packed well. Does that mean it may not be the joint? What else would make this noise?

Oh, somehow I lost a ball bearing putting it back together, too. Does anyone know what size and/or where to get one?

Thanks.