In the case where you have both exe and the msi the exe is just a loader for the msi. If you have an installation supporting multiple languages then the exe applies a language transform (mst) on the msi before installing.
You can consider the exe as a wrapper around the msi. The msi file may or may not be given separately. The reason why people give the msi file too is to facilitate a group policy installation (in a Windows Active Directory infrastructure) as you can only push down installations of msi files and not exes.