What is the difference and why would I use delete over unsubscribe and vice-versa?
As far as the effect goes (stopping people from getting further emails) they are the same. The idea is just to distinguish between people who have unsubscribed themselves (or asked you to unsubscribe them) and people who you have just removed from your list for some other reason.
You can also delete without putting people onto the suppression list.
That makes sense - Thanks Matthew :)
The other difference is that deleted people are really completely deleted.
So if you add them again at a later date, they will be treated as a new person, and be added.
But unsubscribed people will remain on the client suppression list, by default. To re-subscribe them, you would need to delete them from suppression list first.
Not quite actually. Both unsubscribed and deleting subscribers are moved to an unsubscribed or deleted state in your list (rather than being completely removed) so if you ever wanted to add them back to that exact list you'd have to resubscribe them. However, deleting doesn't add to the suppression list so you can freely add them to a new list without any problems.