Exfoliation gets rid of ingrown hairs. It's tougher to exfoliate the ingrown hairs in the nose but it's not impossible. A thin, wet and warm washcloth works, but it can take several tries. I like to exfoliate inside the shower when the hot steam has softened my skin and hair. I exfoliate daily in the shower until the ingrown hair is gone.
I also make sure to take a cotton ball dipped in hydrogen peroxide and clean the inside of my nose after exfoliation. If there is any bleeding, which happened once, I stop immediately and take a break from exfoliation for several days.