There are different valid uses for IDisposable. A simple example is holding an open file, which you need to be closed at certain moment, as soon as you don’t need it any more. Of course, you could provide a method Close, but having it in Dispose and using pattern like using (var f = new MyFile(path)) { /*process it*/ } would be more exception-safe.
A more popular example would be holding some other IDisposable resources, which usually means that you need to provide your own Dispose in order to dispose them as well.
In general, as soon as you want to have deterministic destruction of anything, you need to implement IDisposable.
The difference between my opinion and yours is that I implement IDisposable as soon as some resource needs deterministic destruction/freeing, not necessary as soon as possible. Relying on garbage collection is not an option in this case (contrary to your colleague’s claim), because it happens at unpredictable moment of time, and actually may not happen at all!
The fact that any resource is unmanaged under the cover really doesn’t mean anything: the developer should think in terms of “when and how is it right to dispose of this object” rather than “how does it work under the cover”. The underlying implementation may change with the time anyway.
In fact, one of the main differences between C# and C++ is the absence of default deterministic destruction. The IDisposable comes to close the gap: you can order the deterministic destruction (although you cannot ensure the clients are calling it; the same way in C++ you cannot be sure that the clients call delete on the object).
Small addition: what is actually the difference between the deterministic freeing the resources and freeing them as soon as possible? Actually, those are different (though not completely orthogonal) notions.
If the resources are to be freed deterministically, this means that the client code should have a possibility to say “Now, I want this resource freed”. This may be actually not the earliest possible moment when the resource may be freed: the object holding the resource might have got everything it needs from the resource, so potentially it could free the resource already. On the other hand, the object might choose to keep the (usually unmanaged) resource even after the object’s Dispose ran through, cleaning it up only in finalizer (if holding the resource for too long time doesn’t make any problem).
So, for freeing the resource as soon as possible, strictly speaking, Dispose is not necessary: the object may free the resource as soon as it realizes itself that the resource is not needed any more. Dispose however serves as a useful hint that the object itself is not needed any more, so perhaps the resources may be freed at that point if appropriate.
One more necessary addition: it’s not only unmanaged resources that need deterministic deallocation! This seems to be one of key points of the difference in opinions among the answers to this question. One can have purely imaginative construct, which may need to be freed deterministically.
Examples are: a right to access some shared structure (think RW-lock), a huge memory chunk (imagine that you are managing some of the program’s memory manually), a license for using some other program (imagine that you are not allowed to run more than X copies of some program simultaneously), etc. Here the object to be freed is not an unmanaged resource, but a right to do/to use something, which is a purely inner construct to your program logic.
Small addition: here is a small list of neat examples of [ab]using IDisposable: http://www.introtorx.com/Content/v1.0.10621.0/03_LifetimeManagement.html#IDisposable.