Skip to content

Publish cram #201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
edmundmiller wants to merge 4 commits into
base: dev
Choose a base branch
from
Open

Publish cram #201

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ccae713
build: nf-core modules install samtools/convert
edmundmiller Dec 16, 2024
6f60c17
feat: Publish CRAM
edmundmiller Dec 16, 2024
784e66f
test: bam => cram
edmundmiller Dec 16, 2024
fabe147
style: Run lsp format
edmundmiller Dec 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## dev - [date]

### Added

- [#201](https://github.com/nf-core/nascent/pull/201) - Add CRAM output format support with `--bam` parameter to control output format

### Changed

- [#201](https://github.com/nf-core/nascent/pull/201) - Default output format changed from BAM to CRAM for space efficiency

### Fixed

TODO Remove bedtools bedgraph
Expand Down
35 changes: 33 additions & 2 deletions conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,11 +185,23 @@ process {
]
}

withName: '.*:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_.*' {
withName: '.*:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_SORT' {
ext.prefix = { "${meta.id}.sorted" }
publishDir = [
path: { "${params.outdir}/${params.aligner}" },
mode: params.publish_dir_mode,
pattern: "*.bam",
enabled: params.bam,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
}

withName: '.*:BAM_SORT_STATS_SAMTOOLS:SAMTOOLS_INDEX' {
publishDir = [
path: { "${params.outdir}/${params.aligner}" },
mode: params.publish_dir_mode,
pattern: "*.ba*",
enabled: params.bam,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
}
Expand Down Expand Up @@ -243,6 +255,25 @@ process {
}
}

withName: SAMTOOLS_CONVERT {
publishDir = [
path: { "${params.outdir}/${params.aligner}" },
mode: params.publish_dir_mode,
pattern: "*.{cram,crai}",
enabled: !params.bam
]
}

withName: SAMTOOLS_MERGE {
publishDir = [
path: { "${params.outdir}/samtools" },
mode: params.publish_dir_mode,
pattern: "*.bam",
enabled: params.bam,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename }
]
}

/////////////////////
// Quality Control //
/////////////////////
Expand Down Expand Up @@ -333,7 +364,7 @@ process {
ext.prefix = { "${meta.id}.minus" }
}

withName:DREG_PREP {
withName: DREG_PREP {
ext.prefix = { "${meta.id}.dreg" }
}

Expand Down
25 changes: 25 additions & 0 deletions docs/usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,31 @@ The sample column is essentially a concatenation of the group and replicate colu

By default, the pipeline uses [BWA](https://bio-bwa.sourceforge.net/) (i.e. `--aligner bwa`) to map the raw FastQ reads to the reference genome. Research as to which aligner works best with Nascent Transcript and Transcription Start Site assays is pending.

## Output Format Options

### CRAM vs BAM Files

By default, the pipeline outputs alignment files in CRAM format to save storage space. CRAM files are typically 30-50% smaller than equivalent BAM files while maintaining full data integrity.

**Default behavior (CRAM output):**

```bash
nextflow run nf-core/nascent --input samplesheet.csv --outdir results
```

**To output BAM files instead:**

```bash
nextflow run nf-core/nascent --input samplesheet.csv --outdir results --bam
```

**Key considerations:**

- **CRAM files**: Space-efficient, require reference genome for some downstream tools
- **BAM files**: Larger file size, more widely compatible with downstream tools
- Internal processing always uses BAM format for compatibility
- Only final output format is affected by this parameter

## Reference genome files

The minimum reference genome requirements are a FASTA and GTF file, all other files required to run the pipeline can be generated from these files. However, it is more storage and compute friendly if you are able to re-use reference genome files as efficiently as possible. It is recommended to use the `--save_reference` parameter if you are using the pipeline to build new indices (e.g. those unavailable on [AWS iGenomes](https://nf-co.re/usage/reference_genomes)) so that you can save them somewhere locally. The index building step can be quite a time-consuming process and it permits their reuse for future runs of the pipeline to save disk space. You can then either provide the appropriate reference genome files on the command-line via the appropriate parameters (e.g. `--star_index '/path/to/BWA/index/'`) or via a custom config file.
Expand Down
91 changes: 46 additions & 45 deletions modules.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,74 +7,72 @@
"nf-core": {
"bbmap/pileup": {
"branch": "master",
"git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"bedtools/bamtobed": {
"branch": "master",
"git_sha": "1d1cb7bfef6cf67fbc7faafa6992ad8bdc3045b3",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"bedtools/intersect": {
"branch": "master",
"git_sha": "575e1bc54b083fb15e7dd8b5fcc40bea60e8ce83",
"git_sha": "749d72f162cb1493dca66b454064ed201de2cd95",
"installed_by": ["modules"]
},
"bedtools/merge": {
"branch": "master",
"git_sha": "a5377837fe9013bde89de8689829e83e84086536",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"bedtools/sort": {
"branch": "master",
"git_sha": "571a5feac4c9ce0a8df0bc15b94230e7f3e8db47",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"bowtie2/align": {
"branch": "master",
"git_sha": "e4bad511789f16d0df39ee306b2cd50418365048",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["fastq_align_bowtie2"]
},
"bowtie2/build": {
"branch": "master",
"git_sha": "1fea64f5132a813ec97c1c6d3a74e0aee7142b6d",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"bwa/index": {
"branch": "master",
"git_sha": "e0ff65e1fb313677de09f5f477ae3da30ce19b7b",
"installed_by": ["modules"],
"patch": "modules/nf-core/bwa/index/bwa-index.diff"
"git_sha": "90aef30f432332bdf0ce9f4b9004aa5d5c4960bb",
"installed_by": ["modules"]
},
"bwa/mem": {
"branch": "master",
"git_sha": "e0ff65e1fb313677de09f5f477ae3da30ce19b7b",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["fastq_align_bwa"]
},
"bwamem2/index": {
"branch": "master",
"git_sha": "7081e04c18de9480948d34513a1c1e2d0fa9126d",
"installed_by": ["modules"],
"patch": "modules/nf-core/bwamem2/index/bwamem2-index.diff"
"git_sha": "90aef30f432332bdf0ce9f4b9004aa5d5c4960bb",
"installed_by": ["modules"]
},
"bwamem2/mem": {
"branch": "master",
"git_sha": "3afb95b2e15fc4a2347470255a7ef654f650c8ec",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"cat/cat": {
"branch": "master",
"git_sha": "c60c14b285b89bdd0607e371417dadb80385ad6e",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"cat/fastq": {
"branch": "master",
"git_sha": "1ceaa8ba4d0fd886dbca0e545815d905b7407de7",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"custom/getchromsizes": {
"branch": "master",
"git_sha": "1ceaa8ba4d0fd886dbca0e545815d905b7407de7",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"deeptools/bamcoverage": {
Expand All @@ -84,18 +82,17 @@
},
"dragmap/align": {
"branch": "master",
"git_sha": "dd2757cc22c5de8943fa38ba7cd6f8cc1eb65ac1",
"installed_by": ["modules"],
"patch": "modules/nf-core/dragmap/align/dragmap-align.diff"
"git_sha": "4f5274c3de0c9521f5033893ff61057a74c45ba9",
"installed_by": ["modules"]
},
"dragmap/hashtable": {
"branch": "master",
"git_sha": "ae9e01cb5e77faada314047e78423b22b4f5bbc5",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"fastp": {
"branch": "master",
"git_sha": "1ceaa8ba4d0fd886dbca0e545815d905b7407de7",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"fastqc": {
Expand All @@ -105,7 +102,7 @@
},
"gffread": {
"branch": "master",
"git_sha": "6c996d7fbe0816dcbb68ce587ad5f873313682a1",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"gunzip": {
Expand Down Expand Up @@ -145,9 +142,8 @@
},
"pints/caller": {
"branch": "master",
"git_sha": "ac55541b663c35e9ff50f79d32049ce7492a6ea5",
"installed_by": ["modules"],
"patch": "modules/nf-core/pints/caller/pints-caller.diff"
"git_sha": "8d5737116be7d55f5642ea96cc6d9396389a1811",
"installed_by": ["modules"]
},
"preseq/ccurve": {
"branch": "master",
Expand All @@ -156,49 +152,54 @@
},
"preseq/lcextrap": {
"branch": "master",
"git_sha": "2c6b1144ed58b6184ad58fc4e6b6a90219b4bf4f",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"rseqc/bamstat": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"rseqc/inferexperiment": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"installed_by": ["bam_rseqc", "modules"]
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"rseqc/innerdistance": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"rseqc/junctionannotation": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"rseqc/junctionsaturation": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"rseqc/readdistribution": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"installed_by": ["bam_rseqc", "modules"]
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"rseqc/readduplication": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"installed_by": ["bam_rseqc", "modules"]
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"rseqc/tin": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["bam_rseqc"]
},
"samtools/convert": {
"branch": "master",
"git_sha": "b13f07be4c508d6ff6312d354d09f2493243e208",
"installed_by": ["modules"]
},
"samtools/flagstat": {
"branch": "master",
"git_sha": "2d20463181b1c38981a02e90d3084b5f9fa8d540",
Expand Down Expand Up @@ -232,16 +233,16 @@
"star/align": {
"branch": "master",
"git_sha": "a21faa6a3481af92a343a10926f59c189a2c16c9",
"installed_by": ["fastq_align_star", "modules"]
"installed_by": ["fastq_align_star"]
},
"star/genomegenerate": {
"branch": "master",
"git_sha": "a21faa6a3481af92a343a10926f59c189a2c16c9",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"subread/featurecounts": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"git_sha": "06c8865e36741e05ad32ef70ab3fac127486af48",
"installed_by": ["modules"]
},
"umitools/dedup": {
Expand Down Expand Up @@ -290,12 +291,12 @@
},
"fastq_align_bowtie2": {
"branch": "master",
"git_sha": "0eacd714effe5aac1c1de26593873960b3346cab",
"git_sha": "0fa40bbf85955171ec197334ac70a1197fba600d",
"installed_by": ["subworkflows"]
},
"fastq_align_bwa": {
"branch": "master",
"git_sha": "e0ff65e1fb313677de09f5f477ae3da30ce19b7b",
"git_sha": "0fa40bbf85955171ec197334ac70a1197fba600d",
"installed_by": ["subworkflows"]
},
"fastq_align_hisat2": {
Expand All @@ -305,7 +306,7 @@
},
"fastq_align_star": {
"branch": "master",
"git_sha": "b4919e9a2b4d8b71061e601633db4600a3858fa1",
"git_sha": "6a367612c2c27aa659293c656afa764361223db4",
"installed_by": ["subworkflows"]
},
"homer_groseq": {
Expand Down
8 changes: 8 additions & 0 deletions modules/nf-core/samtools/convert/environment.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading