Janki Kaneria, Senior Software Product Manager in the Data Sciences Platform at the Broad Institute, is responsible for Terra UI Cloud Integration and App Services. In this blog, she announces an upcoming change in how Terra uses Google Projects to manage workspace resources that will have important implications for most users: starting the week of September 27th, each newly created Terra workspace will be associated with a dedicated Google Project. Related to this change, Billing Account administrators are advised to request a quota increase from Google Cloud before September 27th, as described below. Please read the full blog to avoid any surprises or disruptions to your work when this change comes into effect.
As part of an effort to improve cost reporting functionality in Terra, we are making changes to how workspace resources are managed. Until now, Terra supported creating multiple workspaces under a single Google Project, but we have come to realize that this severely limits our ability to add functionality for tracking and reporting detailed cost information through the Terra user interface. That is why we are moving to a model where each Terra workspace is associated with its own Google Project, created by Terra on your behalf when you create the workspace.
Switching to this “project-per-workspace” model will enable us to add functionality for displaying a breakdown of costs per workspace in the Terra user interface, as well as other cost-related features further down the line. The new model will also enable you to set up budget alerts in the GCP console that map to individual workspaces, for any workspaces you create after the changes go into effect.
In addition to enabling much-requested cost reporting features, this change will have some knock-on effects on the use of interactive cloud environments for running Jupyter Notebooks, RStudio, and Galaxy. Briefly, since these environments are created and managed per Google project, going forward you will have a separate cloud environment — and its dedicated persistent disk — for each workspace, instead of sharing the same environment across multiple workspaces under a given project. There will also be minor changes to the environment variables available in Jupyter Notebooks and in the Bioconductor AnVIL package. These changes will all be documented in detail in the relevant documentation upon release.
Based on our analysis of what the majority of researchers need from Terra, we believe this change will be an improvement for most of you. We understand that in a subset of cases, this new model may cause some inconvenience, for which we sincerely apologize. We expect that the sum of benefits will vastly outweigh the drawbacks, and we remain open to your feedback as we continue to improve the functionality and user experience offered by Terra.
Action item for billing account administrators: Request a project quota increase from Google Cloud
There is one important limitation that comes into play with this change: by default, Google Cloud only allows the creation of 5 projects per billing account. Once we move to the project-per-workspace model, this would mean that a single billing account could only support 5 Terra workspaces, which would be insufficient for most individuals’ needs, let alone groups who share a billing account.
The good news is that you can request a quota increase from Google Cloud to lift this limitation. The quota increase is free and it takes just a minute to fill out the web form following the instructions included below.
It can, however, take several days to have the quota increase approved by Google. We are therefore advising all Billing Account administrators to request a quota increase from Google Cloud before September 27th. Failure to do so will result in limitations on how many workspaces can be created per billing account.
If you do not have administrator privileges on the account you use in your work, please check in with your account administrator to make sure they are aware of this notice and take the recommended action.
How to request the quota increase
Due to Google Cloud’s policies, we are not able to make the quota request on your behalf; however, the procedure is straightforward and the instructions below will ensure your request is successful with minimal effort.
Go to https://support.google.com/code/contact/billing_quota_increase
Fill out the form as follows:
- Name: Your name
- Email addresses that will be used to create projects: Billing Account Owner email address
- How many projects are being requested: 5
- What kind of services will these projects use? Paid services
- Billing account: (Note: this field appears when you select the paid services radio button)
You can find the 18-digit GCP billing account ID here - Any other things we need to be aware of to help us understand the request: Please input your own version of the text below:
I use a platform (Terra at app.terra.bio) that creates Google projects via an API to handle data security and processing. In order to use this platform, I need sufficient quota so that projects created by the platform can be transferred over to my billing account.
Note that the number of projects recommended is the minimum quota. Google Cloud team recommends starting with the minimum quota and then asking for more quota later, as your project needs grow. This is because a new billing account with little to no reputation, asking for an extremely high number of additional projects is a trigger for an abuse investigation. You may need to increase it further later if you have a very large group that creates a lot of workspaces; for example, if you run workshops or classes where you provide billing account access to students.
What will happen if you do not request the quota increase?
Any pre-existing workspaces will remain associated with the project(s) they were created under, and will therefore not be affected by this change. Creating new workspaces will incrementally increase the number of projects that count against your quota, up to the default limit of 5 projects. Once you reach that number, you will no longer be able to create new workspaces (including via cloning) until you delete or disable billing on an existing project. However, you will continue to be able to work in your existing workspaces.
What will happen if I’m using Google Cloud’s free credits program?
Unfortunately, Google Cloud does not consider free credits accounts to be eligible for project quota increases. If you are using free credits, you will be limited to a maximum of 5 projects per free credits account. As noted above, workspaces created before this change will not be affected.
Updates
February 02, 2022: “How to request the quota increase” section revised to reflect new Google Cloud team recommendations.
