Trying to beat the bearing out and made a rookie mistake. Assuming if I left this unchecked it would foul up the new bearing. I can probably file the bearing side with a dremmel to get it smooth again but wanted to check here to see if I needed to replace the whole thing.