Your mission

Your mission: Create an open workspace
that showcases a fully reproducible analysis!

Do you have a pet workflow or favorite notebook? Have you thought about sharing them with the world, but keep pushing it off? Here's your opportunity to get it done, feel good about it AND win a trip to Switzerland in July!

We've all been in the situation of needing to share our tools and methods with a collaborator, or wanting to build on someone else's work for our own research. We've made a lot of progress toward making it easier to share, collaborate, and build on previous computational biology work by adopting best practices like version control, containers and so on. But there can still be a big gap between handing over the code and enabling others in the field to actually run it and reproduce the work. Let's show how much easier it can be with a showcase of working examples in an environment like Terra that integrates code, data and execution.

Through this friendly contest, we want to challenge the Terra community to raise the bar in how we share computational methods with each other. The best workspace wins (we're crowdsourcing evaluations based on overall ease of reproducibility, including clarity and completeness of workspace documentation... and science chops!). By showcasing a variety of reproducible workspaces, we hope to demonstrate the high value benefits to the entire community of Open Science approaches that boost computational reproducibility.

For some background reading on computational reproducibility, see this blog post.

Instructions

  1. If you don't already have a Terra account, follow these instructions to create one and provision it with free credits to cover compute and storage costs for your project.

  2. Clone this template workspace, making sure to give your workspace a name that follows this convention: TOSC19-NAME where NAME can be anything you choose (doesn't have to be capitalized). This is your master workspace.

  3. Populate your master workspace with any data, tools and/or notebooks necessary to perform the analysis you want to showcase. Make sure everything in there is okay to share publicly! You are solely responsible for complying with data privacy and data use restrictions. Your workspace description should include sufficient documentation to reproduce your analysis without any additional help from you.

  4. Do all your testing in fresh clones so your master workspace stays clean of past runs and output files.

  5. Once you're satisfied with your work, share your master workspace with tosc@terra.bio to enter it into the contest.

Our team will review your workspace and email you to ask for your explicit permission to make it fully public and list it on the contest page.

Prizes

The Grand Prize is a roundtrip to Basel, Switzerland to attend the joint ISMB/ECCB* and BOSC** conference, July 21-25. Flights, hotel accommodation, and conference registration are all included in this prize. The winner will be expected to attend the conference and may be asked to give a short oral presentation about the workspace.

*ISMB: International Society for Computational Biology
**BOSC: Bioinformatics Open Source Conference

Additional prizes:

  • The first runner-up will win $5,000 in Google credits.

  • Second runner-up will win $2,500 in Google credits.

Entry submissions will open May 17 and close June 17. Judging will open May 20 and close June 20.

Evaluation

Entries will be judged based on the following criteria:

  1. Ability for anyone to run the full analysis easily

  2. Opportunity to customize relevant parameters of the analysis

  3. Clarity and completeness of the documentation

  4. Coolness of the science (as a tiebreaker)

Judging will be crowdsourced to the Terra community. We will invite all Terra users to try out the workspaces entered into the contest, and each workspace will include a link to a short google form with the scoring rubrics. Contestants are welcome to advertise their workspace(s) as widely as they like. Judges are not obligated to review all workspaces, but we do encourage everyone to review more than one workspace as this will provide some useful perspective for scoring entries.

Rules & FAQs    What kind of analysis can I showcase?  Whatever you want as long as it has a scientific purpose. It can feature anything from a simple data processing step to a multi-stage analysis, and it can involve one or more workflows, notebooks or combination thereof. This contest is meant to allow you to showcase existing work; maybe even give you an excuse to write up that documentation that keeps getting deprioritized. Now you can tell your PI it's worth doing because it could earn you compute credits for the lab!   How do I know what makes a good open workspace / reproducible analysis?   You can browse the   Terra Showcase   to get inspired on how to set up documentation, data, tools and/or notebooks in your own workspace.   Can I base my workspace entry on one of the example workspaces in the Terra showcase, eg one of the GATK workspaces?  You're welcome to reuse components of the example workspaces as part of your own workspace BUT you must make a material contribution to the work -- so no straight-up clones, please. We reserve the right to reject any workspace that we deem insufficiently novel at the time it is submitted for entry, with appropriate comments to give you the opportunity to improve your submission. You will have the right to resubmit your workspace.   What if my data is not shareable?  If the data you work with is private and cannot be shared publicly, you'll need to substitute equivalent public data. Have a look at the public datasets in the   Terra Showcase   and the Data Library. If that doesn't work for you, consider using synthetic data like in this paper reproduction   case study   and     this hackathon project  . Don't hesitate to reach out to the Terra helpdesk for guidance.   Can I submit multiple entries?  Yes! Feel free to submit as many as you want. They will all be scored independently. Note however that only your highest scoring entry will be eligible for a prize; the same person cannot win two or more prizes. This is intended to protect the opportunity of other contestants to win a prize.   Can I be a judge if I have one or more workspaces entered in the contest?  Yes, with a couple of restrictions. We ask that you refrain from grading your own workspace(s), and play fair when you grade other people's workspaces. We reserve the right to disqualify any contestant-judges if we have reason to believe that they systematically downgraded other contestants' workspaces regardless of actual merit or encouraged collaborators to do so on their behalf.   Will I retain ownership of my workspace?  Yes, absolutely -- you will still be the owner of the workspace and all it contains. If you decide to delete it at any point, you are free to do so. Note however that if you delete anything from your workspace before the end of the contest, that entry will be disqualified. This will not affect any other (unmodified) entries you submit.   If I win the Grand Prize but I am not able or willing to travel to Switzerland, can I nominate someone else to go in my place?  Yes, with some restrictions. You are welcome to nominate a colleague or collaborator who contributed to the work showcased in the winning workspace. They will be expected to attend the conference and may be asked to give a short presentation about the workspace.   Winner Selection & Notification   At the conclusion of judging period (June 20), we will select one (1) potential Grand Prize winner and attempt to notify them via the email address provided. The potential winner will be required to respond to the notification within five (5) days of attempted notification. Failure to respond within the stated time period may result in forfeiture of the Grand Prize.   Exclusions   Employees of the Data Sciences Platform at the Broad Institute are  not eligible for any prizes . They are however allowed to participate  pro bono  in the spirit of Open Science. All other Broad Institute employees are eligible to win prizes.

Rules & FAQs

What kind of analysis can I showcase?
Whatever you want as long as it has a scientific purpose. It can feature anything from a simple data processing step to a multi-stage analysis, and it can involve one or more workflows, notebooks or combination thereof. This contest is meant to allow you to showcase existing work; maybe even give you an excuse to write up that documentation that keeps getting deprioritized. Now you can tell your PI it's worth doing because it could earn you compute credits for the lab!

How do I know what makes a good open workspace / reproducible analysis?
You can browse the Terra Showcase to get inspired on how to set up documentation, data, tools and/or notebooks in your own workspace.

Can I base my workspace entry on one of the example workspaces in the Terra showcase, eg one of the GATK workspaces?
You're welcome to reuse components of the example workspaces as part of your own workspace BUT you must make a material contribution to the work -- so no straight-up clones, please. We reserve the right to reject any workspace that we deem insufficiently novel at the time it is submitted for entry, with appropriate comments to give you the opportunity to improve your submission. You will have the right to resubmit your workspace.

What if my data is not shareable?
If the data you work with is private and cannot be shared publicly, you'll need to substitute equivalent public data. Have a look at the public datasets in the Terra Showcase and the Data Library. If that doesn't work for you, consider using synthetic data like in this paper reproduction case study and this hackathon project. Don't hesitate to reach out to the Terra helpdesk for guidance.

Can I submit multiple entries?
Yes! Feel free to submit as many as you want. They will all be scored independently. Note however that only your highest scoring entry will be eligible for a prize; the same person cannot win two or more prizes. This is intended to protect the opportunity of other contestants to win a prize.

Can I be a judge if I have one or more workspaces entered in the contest?
Yes, with a couple of restrictions. We ask that you refrain from grading your own workspace(s), and play fair when you grade other people's workspaces. We reserve the right to disqualify any contestant-judges if we have reason to believe that they systematically downgraded other contestants' workspaces regardless of actual merit or encouraged collaborators to do so on their behalf.

Will I retain ownership of my workspace?
Yes, absolutely -- you will still be the owner of the workspace and all it contains. If you decide to delete it at any point, you are free to do so. Note however that if you delete anything from your workspace before the end of the contest, that entry will be disqualified. This will not affect any other (unmodified) entries you submit.

If I win the Grand Prize but I am not able or willing to travel to Switzerland, can I nominate someone else to go in my place?
Yes, with some restrictions. You are welcome to nominate a colleague or collaborator who contributed to the work showcased in the winning workspace. They will be expected to attend the conference and may be asked to give a short presentation about the workspace.

Winner Selection & Notification

At the conclusion of judging period (June 20), we will select one (1) potential Grand Prize winner and attempt to notify them via the email address provided. The potential winner will be required to respond to the notification within five (5) days of attempted notification. Failure to respond within the stated time period may result in forfeiture of the Grand Prize.

Exclusions

Employees of the Data Sciences Platform at the Broad Institute are not eligible for any prizes. They are however allowed to participate pro bono in the spirit of Open Science. All other Broad Institute employees are eligible to win prizes.