In very simple terms :

if you want to delete resources(objects) those are not using you should not worry about that garbage collecter implicitly call finalize() method and remove all such object but if you want to delete object forcefully(The larger object you want to delete after completeing task) than you can explicitly call dispose() method.

Design Pattern : If your classes use unmanaged resources you need to implement both Dispose & Finalize. Dispose() is called by user code that is the code that is using your class.
Finalize/Destructor cannot be called by User code it's called by Garbage Collector

Finalize : Is a destructor called by Garbage Collector when the object goes out of scope. Implement it when you have unmanaged resources in your code and want to make sure that these resources are freed when the Garbage collection happens.

Dispose : Same purpose as finalize to free unmanaged resources. However implement this when you are writing a custom class that will be used by other users. Overriding Dispose() provides a way for the user code to free the unmanaged objects in your custom class.

As an aside here's how the GC works:
The garbage collector keeps track of objects that have Finalize methods using an internal structure called the finalization queue. Each time your application creates an object that has a Finalize method the garbage collector places an entry in the finalization queue that points to that object. The finalization queue contains entries for all the objects in the managed heap that need to have their finalization code called before the garbage collector can reclaim their memory.

Implementing Finalize methods or destructors can have a negative impact on performance and you should avoid using them unnecessarily. Reclaiming the memory used by objects with Finalize methods requires at least two garbage collections. When the garbage collector performs a collection it reclaims the memory for inaccessible objects without finalizers. At this time it cannot collect the inaccessible objects that do have finalizers. Instead it removes the entries for these objects from the finalization queue and places them in a list of objects marked as ready for finalization. Entries in this list point to the objects in the managed heap that are ready to have their finalization code called. The garbage collector calls the Finalize methods for the objects in this list and then removes the entries from the list. A future garbage collection will determine that the finalized objects are truly garbage because they are no longer pointed to by entries in the list of objects marked as ready for finalization. In this future garbage collection the objects' memory is actually reclaimed.