Good 'ol hot soapy water is your best bet in this case. Anything much harsher will most likely remove the primer or at least compromise it in some way. Use a medium-stiff brush to scrub pretty well and it will be fine.
I've had this situation before, and I also used Ajax cleanser. It will however behave like fine sandpaper and scuff the surface of your primer, but it will also get it very clean.
That is how Maaco and other quickie body shops prep a car for paint. They wash it with Ajax and a Scotchbrite pad. It scuffs and cleans all in one step.
"Free advice is worth what you pay for it."™