How to run given function in Bash in parallel?

sem is part of GNU Parallel and is made for this kind of situation.

for i in "${list[@]}"
    for j in "${other[@]}"
        # some processing in here - 20-30 lines of almost pure bash
        sem -j 4 dolong task

If you like the function better GNU Parallel can do the dual for loop in one go:

dowork() { 
  echo "Starting i=$1, j=$2"
  sleep 5
  echo "Done i=$1, j=$2"
export -f dowork

parallel dowork ::: "${list[@]}" ::: "${other[@]}"

Leave a Comment