I had this exact same problem; in order to make sure that the correct answer is not buried in the comments (as it was for me), I want to reiterate @Sprockincat’s comment:
For me at least, it was indeed an issue with IIS Custom errors, and can be solved with:
Response.TrySkipIisCustomErrors = true;
@Sprockincat – you should get credit for this. I’m just making it more visible because it’s such a subtle fix to a problem that is quite difficult to diagnose.