When you access a “variable”, you look up the address, and then fetch the value.
Remember – a pointer IS a variable. So actually, you:
a) look up the address (of the pointer variable),
b) fetch the value (the address stored at that variable)
… and then …
c) fetch the value at the address pointed to.
So yes, accessing via “pointer” (rather than directly) DOES involve (a bit) of extra work and (slightly) longer time.
Exactly the same thing occurs whether or not it’s a pointer variable (C or C++) or a reference variable (C++ only).
But the difference is enormously small.