Largely, for the same reason you start at the least significant digit (the right end) when you add—because carries propagate toward the more significant digits. Putting the least significant byte first allows the processor to get started on the add after having read only the first byte of an offset.
After you’ve done enough assembly coding and debugging you may come to the conclusion that it’s not little endian that’s the strange choice—it’s odd that we humans use big endian.