UPDATE:
It appears Jetbrains removed the checkbox I was referring to since I posted this. galcyurio’s answer looks like the correct way to disable this functionality in current builds.
Original Answer:
You can turn this off in Gradle Settings. Preferences -> Build, Execution, Deployment -> Build Tools -> Gradle: uncheck create separate modules per source set.
I had to turn it off as it was causing build failures because Make was only pulling in the main source set. This is likely a bug in Intellij because gradle can still build the projects just fine.