When constructing the URL, use the parameter d=404. This will cause Gravatar to return a 404 error rather than an image if the user hasn’t set a picture.
If you’re using the .Net control linked to from the gravitar site, you’ll need to modify the IconSet enumeration (and probably pull the code out of the control, so you can get at the status directly).