Yes, you will need to transfer the keys. Mac and Linux work the same, storing the keys in ~/.gnupg. The safest way to transfer the files is using scp (part of ssh):
scp -rp ~/.gnupg othermachine:
However, you will need to have ssh working first.
Transferring them with, say, a USB flash drive isn’t such a great idea because your private key will be left behind on that drive even after you’ve deleted the file. Although it’s protected by a passphrase, if someone got hold of a copy of the key file they could mount a long-running brute-force attack on it at their lesiure.
I don’t know about the location of the directory on Windows. The gpg documentation will say, and the contents will almost certainly be the same.
Copying the entire keyring is quick and easy, but sometimes you want to be able to move individual keys between machines without overwriting the entire keyring and losing the keys that are already there. Copying individual keys selectively can be done with gpg --export-secret-key and gpg --import. If you have ssh access to the destination machine you can do this with a pipe and don’t need to store an intermediate key anywhere:
If you’re on the machine that already has the key:
gpg --export-secret-key SOMEKEYID | ssh othermachine gpg --import
If you’re on the machine that needs the key:
ssh othermachine gpg --export-secret-key SOMEKEYID | gpg --import
If gpg isn’t in one of the default places on the remote machine (eg it’s in /opt/local/bin on a Mac) you’ll have to give its full path to ssh, or symlink it into one of the standard places such as /usr/local/bin.
Note that the data that’s transferred is still protected by the passphrase, and the key will have the same passphrase at the destination as it did at the source. If you want to have different passphrases in each place, you’ll need to change the passphrase at the destination, or change it temporarily at the source before exporting it. I needed to share a secret key with a colleague, in order to give him the ability to update a Debian package repo we both manage, but I didn’t want to share my passphrase with him. So I changed the passphrase to something temporary, sent him the exported key (by gpg-encrypted email!), told him the temporary passphrase orally, and asked him to set a new passphrase immediately after importing the key. I then changed the passphrase on my copy of the key back to what it was originally.
Update on 2021-03-03
With GnuPG 2.0, if you’re exporting the key from a remote machine and you don’t have X11 connection forwarding, you may have problems entering a passphrase. For example, gpg says “cannot open ‘/dev/tty’”. If you force pseudo-terminal allocation with ssh -t the key becomes mixed up with the terminal activity such as the passphrase prompt and \rs. One way to work around this is:
ssh othermachine gpg --passphrase-fd 0 --pinentry-mode loopback ...
You will then need to enter the passphrase and press Enter. There will not be a prompt, and echo will not be suppressed (so the passphrase will be visible on your terminal). If you feel strongly about the echoing, you can turn it off temporarily with stty -echo and back on again with stty echo. This can all be rolled into a single command like this:
stty -echo; \
ssh othermachine gpg --passphrase-fd 0 --pinentry-mode loopback \
--export-secret-key SOMEKEYID | gpg --import; \
stty echo
However, it’s rare that you’ll need to do this since by far the most common case is when you’re exporting from the machine you’re sitting at and importing onto a remote machine that you ssh to.