What is the difference between shlex.split() and re.split()?

shlex.split() is designed to work like the shell’s split mechanism.

This means doing things like respecting quotes, etc.

>>> shlex.split("this is 'my string' that --has=arguments -or=something")
['this', 'is', 'my string', 'that', '--has=arguments', '-or=something']

re.split() will just split on whatever pattern you define.

>>> re.split('\s', "this is 'my string' that --has=arguments -or=something")
['this', 'is', "'my", "string'", 'that', '--has=arguments', '-or=something']

Trying to define your own regex to work like shlex.split is needlessly complicated, if it’s even possible.

To really see the differences between the two, you can always Use the Source, Luke:

>>> re.__file__
'/usr/lib/python3.5/re.py'
>>> shlex.__file__
'/usr/lib/python3.5/shlex.py'

Open these files in your favorite editor and start poking around, you’ll find that they operate quite differently.

Leave a Comment