Create spark dataframe schema from json schema representation

There are two steps for this: Creating the json from an existing dataframe and creating the schema from the previously saved json string.

Creating the string from an existing dataframe

    val schema = df.schema
    val jsonString = schema.json

create a schema from json

    import org.apache.spark.sql.types.{DataType, StructType}
    val newSchema = DataType.fromJson(jsonString).asInstanceOf[StructType]

Leave a Comment