Well the rust was mainly limited to the tucked in edge of the wheel arches and doesn't look to be affecting any of the quarter panel. As minor as it is it certainly has raised some doubts in my mind but I won't back out completely just yet. The dealer wants to do some standard maintenance, put new tires on, and try to fix/prevent the rust as much as they can without replacing metal. I told them I would think about it after they get done with the car, I'll still probably take it to get a PPI and additionally a body shop to get their thoughts.
Really wish I just stumbled upon a mint 73,000 mile 2003 M3 but that would be ridiculous right?
EDIT: It looks like the previous owner neglected it more than abused it which is a shame.