Adam Mullen, Software Product Manager in the Data Sciences Platform at the Broad Institute, is responsible for Terra workspaces. In this blog, he announces a technical change that will affect workspaces created before September 27, 2021.
If you own any workspaces created before that date, please read the full blog to avoid any surprises when the changes are implemented. If all your workspaces were created after that date, feel free to stop reading and get back to your science.
Prior to September 2021, any workspaces attached to a given billing project in Terra shared the same underlying Google Project. This had some side effects that limited our ability to build resource management features, so we introduced a “one workspace per project” model to remove those limitations. Since then, every new workspace created in Terra has been attached to a dedicated Google Project uniquely created for that workspace.
Yet, a number of workspaces created before the switch to the new model are still linked to Google Projects that support multiple workspaces. In order to move forward with upcoming infrastructure changes that will enable more granular spend reporting as well as improve the scale and performance of workspace management services, we need to migrate those older workspaces to dedicated per-workspace Google Projects.
An automated migration process to minimize impact
We have developed a process to automate this migration, so you will not have to do anything on your end to migrate your workspaces. The new Google Projects will appear in your Google Cloud console as described in this overview article. Data and metadata stored in your workspaces will not be affected in any way; the Google Bucket ID in your workspace and any associated references will not change.
However, your workspaces will become temporarily unavailable for use while the migration process is taking place. We estimate that most workspace migrations will take less than 5 minutes to complete, although workspaces with a significant amount of data may take longer. (For reference, a migration test on a bucket with 1PB of data and 400,000 files took about 15 minutes in total.)
In addition, any persistent disks currently attached to an older workspace will become unattached during the migration process. You will find them listed on your Cloud Environments page, colored in red, with an action button that you can use to reattach them to a workspace. Note that we always recommend saving your persistent disk data to your workspace’s storage when you are not actively/regularly using those resources.
Rolling migrations to take place starting August 8, 2022
There are over 33,000 workspaces that need to be migrated, so we will be going through them on a rolling basis, starting on August 8, 2022. We will attempt to migrate all workspaces that belong to the same billing project at the same time to minimize disruption. We expect most migrations to be completed by August 26, 2022 (within 3 weeks).
Given the scale of this undertaking, we cannot give you a specific window of time when your workspaces will be migrated. According to our usage records, most older workspaces are accessed very infrequently, so we hope this migration will happen invisibly for most of you. However, if you are actively using a workspace created before September 27, 2021 and you are concerned that this migration timeline might interrupt time-sensitive work, please reach out to the Terra helpdesk and we can work with you to schedule the migration of your workspaces.
Thank you for your patience as we work to improve our services and provide you with new capabilities to support your research.
