When is it OK to prune the upload/upgrades directory?

After a successful upgrade from one version of SugarCRM to another, when is it OK to prune the upload/upgrades directory? We are currently carrying around multiple gigabytes of zip files, backups, and temp files from previous upgrade processes. They seem extraneous and unnecessary.