Nextflow on Shaun Chuah

Launching Nextflow Pipelines From the Cloud

Thu, 28 Apr 2022 18:29:04 +0000

Why bother setting up another server to manage your Nextflow pipeline?

Avoid premature termination of the pipeline. When you run a Nextflow pipeline from your local computer, your local computer is managing the tasks and communicating with Azure as jobs are completed. Depending on the complexity of your pipeline, this may be a long time (couple of days!). If your wi-fi router accidentally comes unplug or the connection is broken, the pipeline will terminate prematurely. While you could recover from this using -resume, you can avoid the risks of disconnection by using a cloud server, which is designed to run 24/7, to orchestrate this.

Setting Up Azure With Nextflow

Wed, 18 Aug 2021 18:27:01 +0000

We will cover getting your nextflow pipeline up and running in the cloud using Azure. First thing to note, this process will take some time so sit back, grab a coffee and take your time working through each section.

Useful References

Nextflow documentation here. Nextflow blog post here.

1. Context

Let’s start with some context. I’m working with a small team of immunology researchers who are following their scientific questions which has led into genomics. We don’t have any bioinformatics setup and are starting from nothing - which is probably going to be the case for many newcomers as NGS technology continues to become more accessible (for example Nanopore sequencing).

How to Create Your Bioinformatics Pipeline with Nextflow

Thu, 12 Aug 2021 18:24:45 +0000

Now that you know how to run bioinformatics software in Docker containers, it’s time to connect them up. If you’ve missed the last post the link is here: Getting started with Docker for bioinformatics.

Content Overview

What is a pipeline?
Nextflow vs Snakemake
Using Nextflow and Docker containers to create your pipeline
Summary

What is a pipeline anyway?

The term ‘pipeline’ is thrown around a lot in bioinformatics. In simple terms, it refers to the programs that have to be run in a certain order to complete the analysis. Some of these programs take the outputs of earlier programs and process them in order to achieve a specific objective.