Weird specks all over the place!?

Unfortunately, I didn’t think to take a picture.

I washed my car today and noticed that paint wasn’t smooth in some areas. It was kind of rough. When I looked close at what appeared to be a light spray of dirt, I found my paint speckled with reddish brown something.

It came off a little with clay bar, but would have taken an hour to do a 3 square inch area. So I fired up my DA Orbital Polisher and took some Meguiars cutting compound/pad to it. After serious elbow grease, it came off. But it was ALL OVER THE PLACE. Estimate at about 15 square feet of my car (in weird spots - along the rocker and driver door on one side, rear quarter on the other side).

So I spent 7 hours washing, polishing, and waxing. Oiy.

I often park in the city. The only thing I can think of is that I park somewhere and some grafitti’d near my car (wouldn’t be too outlandish given where I work in SF). Was that it? Could it have been something else? Oppo ideas?