You can use
contains (this works with an arbitrary sequence):
like (SQL like with SQL simple regular expression whith
_ matching an arbitrary character and
% matching an arbitrary sequence):
rlike (like with Java regular expressions):
depending on your requirements.
RLIKE should work with SQL expressions as well.
- How to define partitioning of DataFrame?
- Renaming column names of a DataFrame in Spark Scala
- Provide schema while reading csv file as a dataframe in Scala Spark
- DataFrame equality in Apache Spark
- How do I detect if a Spark DataFrame has a column
- How do I check for equality using Spark Dataframe without SQL Query?
- Joining Spark dataframes on the key
- Derive multiple columns from a single column in a Spark DataFrame
- Spark Dataframe :How to add a index Column : Aka Distributed Data Index
- How to change a column position in a spark dataframe?
- Aggregating multiple columns with custom function in Spark
- Create new Dataframe with empty/null field values
- Spark dataframe get column value into a string variable
- How to select the first row of each group?
- How can I change column types in Spark SQL’s DataFrame?
- How to sort by column in descending order in Spark SQL?
- Spark – load CSV file as DataFrame?
- How to convert rdd object to dataframe in spark
- How to create an empty DataFrame with a specified schema?
- Extract column values of Dataframe as List in Apache Spark
- How to save DataFrame directly to Hive?
- Get current number of partitions of a DataFrame
- How to write unit tests in Spark 2.0+?
- how to filter out a null value from spark dataframe
- How to use Column.isin with list?
- Querying Spark SQL DataFrame with complex types
- Why does join fail with “java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]”?
- How to avoid duplicate columns after join?
- How to aggregate values into collection after groupBy?
- Spark unionAll multiple dataframes
- Automatically and Elegantly flatten DataFrame in Spark SQL
- Fetching distinct values on a column using Spark DataFrame
- What are the various join types in Spark?
- How to convert Row of a Scala DataFrame into case class most efficiently?
- dataframe: how to groupBy/count then filter on count in Scala
- Append a column to Data Frame in Apache Spark 1.3
- Get the size/length of an array column
- Spark – extracting single value from DataFrame
- Spark extracting values from a Row
- Flattening Rows in Spark
- Filter Spark DataFrame by checking if value is in a list, with other criteria
- Reading DataFrame from partitioned parquet file
- Change nullable property of column in spark dataframe
- How to convert DataFrame to RDD in Scala?
- Encoder error while trying to map dataframe row to updated row
- Spark: Add column to dataframe conditionally
- Create new column with function in Spark Dataframe
- How to define and use a User-Defined Aggregate Function in Spark SQL?
- Select Specific Columns from Spark DataFrame
- “value $ is not a member of StringContext” – Missing Scala plugin?