Are you using any other tools while using TortoiseGit?
For instance, this project mentions an interaction with Visual Studio:
If you see errors about being unable to unlink old files when running git commands on Windows, it’s quite likely that you have the chrome solution open in Visual Studio and it’s updating IntelliSense, which will briefly lock files at random.
The workaround is either to disable IntelliSense entirely, or to be sure to close the solution temporarily while performing any git operations which could touch a large number of files.
In other word, does the problem persists if you close as many currently running applications as you can?
Would TortoiseGit allow you to checkout another branch then?