I'd say a good rule of thumb is to see a doctor if you have fever for more than 48 hours, or if it goes above 103.0 degrees Fahrenheit. That usually means you've got a serious infection working and they are no fun at all.
Sinus infections are miserable, and can turn into bronchitis or pneumonia if left untreated. Several years ago, I worked a low grade sinus infection for weeks, and then it got nasty and turned into bronchitis. I was on antibiotics for a month and could hardly walk up the steps to my apartment. It was awful and could have become pneumonia. Don't let a sinus infection go for too long.