You can use versioning to preserve, retrieve and restore every version of every object a Bucket.
It is very useful to recover file in case of accidental deletion or overwrite.
Version is not active by default, once enabled, you can only suspend it ( version already created will be kept, no new ones will be generated).
Since with S3 you pay per storage, and every version is like a new file, you will incur in new costs per every new version.
When you delete an object in a versioning-enabled bucket, all versions remain in the bucket, and Amazon S3 creates a delete marker for the object. To undelete the object, you must delete this delete marker.
If you want to permanently delete the object and all its versions then you need to use CLI/SDK and call DELETE Object specifying versionId.