Put your swagger configuration into separate configuration class and annotate it with @Profile annotation -> so that it will be scanned into Spring context only in certain profiles.
Example:
@Configuration
@EnableSwagger2
@Profile("dev")
public class SwaggerConfig {
// your swagger configuration
}
You can than define profile your Spring Boot app is operating in via command line: --spring.profiles.active=dev or via config file: spring.profiles.active=dev.
Read this section of Spring Boot docs for more info about @Profile