How to run given function in Bash in parallel?

by

sem is part of GNU Parallel and is made for this kind of situation.

for i in "${list[@]}"
do
    for j in "${other[@]}"
    do
        # some processing in here - 20-30 lines of almost pure bash
        sem -j 4 dolong task
    done
done

If you like the function better GNU Parallel can do the dual for loop in one go:

dowork() { 
  echo "Starting i=$1, j=$2"
  sleep 5
  echo "Done i=$1, j=$2"
}
export -f dowork

parallel dowork ::: "${list[@]}" ::: "${other[@]}"

Related Contents:

  1. How do you run multiple programs in parallel from a bash script?
  2. Running programs in parallel using xargs
  3. Parallel wget in Bash [duplicate]
  4. Is bash a programming language?
  5. Running several scripts in parallel bash script [duplicate]
  6. Is there a way to make bash job control quiet?
  7. How do I set a variable to the output of a command in Bash?
  8. Difference between sh and Bash
  9. How do I prompt a user for confirmation in bash script? [duplicate]
  10. Create timestamp variable in bash script
  11. Echo tab characters in bash script
  12. How do I find all of the symlinks in a directory tree?
  13. Getting ssh to execute a command in the background on target machine
  14. Repeatedly run a shell command until it fails?
  15. How do I redirect output to a variable in shell? [duplicate]
  16. Run ssh and immediately execute command [duplicate]
  17. How to assign the output of a Bash command to a variable? [duplicate]
  18. How can I reference a file for variables using Bash?
  19. What is your single most favorite command-line trick using Bash? [closed]
  20. Checking from shell script if a directory contains files
  21. Removing created temp files in unexpected bash exit
  22. Bash: pass a function as parameter
  23. How to execute the output of a command within the current shell?
  24. Ignoring specific errors in a shell script
  25. What is the Bash file extension?
  26. prevent xargs from quitting on error
  27. backslash at the beginning of a command
  28. sudo cat File doesn’t work, sudo su does [duplicate]
  29. Bash syntax error: “[[: not found” [duplicate]
  30. Creating tar file and naming by current date
  31. Why can’t I use job control in a bash script?
  32. How can I make a progress bar while copying a directory with cp?
  33. Pass args for script when going thru pipe
  34. Cannot connect to the Docker daemon on bash on Ubuntu windows
  35. How can I replace ALL instances of a string in the previous command in Bash? [duplicate]
  36. Unable to set variables in bash script [duplicate]
  37. What is the meaning of set -o pipefail in Bash Script?
  38. Best way to choose a random file from a directory in a shell script
  39. How to use Pigz with Tar
  40. Print rest of the fields in awk
  41. How to create a frequency list of every word in a file?
  42. What does if [ $? -eq 0 ] mean for shell scripts? [duplicate]
  43. What does — do when running an npm command?
  44. Is it necessary to specify traps other than EXIT?
  45. How to remove filename prefix with a Posix shell
  46. How do I include an environment variable inside an alias for bash?
  47. -bash: gcloud: command not found on Mac
  48. Shell Prompt Line Wrapping Issue
  49. Connect to sqlplus in a shell script and run SQL scripts
  50. Why does the equal to operator not work if it is not surrounded by spaces?

Leave a Comment