Summary: Intermediate files generated by your workflow may be an unexpected source of storage costs. Fortunately, you now have an easy option to delete these files immediately after your workflow runs. Just select the new “delete intermediate outputs” box on your workflow configuration.
Saving money on your workflow analysis is always a top priority, but you may not realize one unexpected source of your workflow costs: storing intermediate workflow files. Intermediate files have multiple benefits; they allow you to call cache partial results, saving you time and money troubleshooting unsuccessful workflows. But they also require extra storage and often have no use once workflow output has been computed. This is why we want to offer flexibility in your choice of storing them. Terra now provides a new workflow configuration option to delete intermediate files after a successful run, saving you money on Google Cloud storage cost.
The new “Delete intermediate outputs” option is available as a checkbox (highlighted in image below) on your workflow configuration page.
By checking the “Delete intermediate outputs” box, your intermediate workflow files will be deleted after the workflow successfully completes.
As a reminder, the delete intermediate outputs option will preclude call caching on your workflow once it’s successfully completed. However, if the option is selected and the workflow fails part of the way through, the results can still be cached. If you want to keep intermediate files, simply leave the box unchecked. You can always look to manually delete these files later.
For more information about the delete intermediates option, see the following help article.
Coming Soon:
We hope to offer a manual option for deleting intermediate files in a workspace at a later time, allowing you to complete your analyses first and then look to delete the intermediate files when you are ready to share your workspace and to reduce ongoing storage costs.
