1.0 release checklist

[nictru]

• Clean up branches (Cleaned up branch structure before release PR #98)
  • Rename current main branch (default branch of the old implementation) to make clear that it is legacy
  • Rename current master branch to main
  • Update nextflow.config to reflect this
• Label all open issues with bug, enhancement etc.
  • Solve all issues labeled as bug (there should not be any known bugs in the 1.0 release)
  • Create milestones, one for things that should be done before the 1.0 release and one for things that can be done later, assign issues accordingly
• Make sure nf-core pipelines lint finishes without errors and warnings
• Make sure the latest-everything CI checks pass (strict syntax) -> @nictru
• Template update -> @nictru
• Final round of internal review
• Initiate release process

[LuisHeinzlmeier]

• The legacy main branch is now called legacy-main.
• The master branch was renamed to main.
• Only the current/new main is now a target branch.
• Default branch was already set correctly in nextflow.config of the dev branch: defaultBranch = 'main'

[LuisHeinzlmeier]

I’ve labeled all issues. There are 4 that I would consider bugs, but I’m unsure whether and how to resolve them before the release. Some comments on the individual bugs:

#97 I tried to figure out why these tools only assign negative, but I wasn’t able to find a solution. Unfortunately, I haven’t received any response from the developers either. I suspect it might be because the test dataset is too small. What would you suggest I do?

#92 Bff has two methods, which gives three possible ways to call it: RAW, CLUSTER, or COMBINED (the default). The pipeline therefore only outputs results for RAW because CLUSTER doesn’t work. I can try to fix this again, but I wasn’t able to make progress on it previously. This could also be considered an enhancement, since RAW works.

#81 Demuxem unfortunately doesn’t work, and other users have reported the same problem. I haven’t received any response from the developers here either. We could also consider removing the deconvolution tool altogether, what do you think?

#57 This is a bug from the legacy pipeline. I suspect it might still exist, but currently I can’t reproduce it (see comments in the issue). Should we just close it?

@nictru and @Zethson, let me know what you think.

[nictru]

My checklist from above was basically a reconstruction of the official checklist from memory, but you can find more detailed information there.

Here are my thoughts on the issues:

• Two genetic tools assign all cells as negative with testing data #97 can be treated as an enhancement of the testing, as the current tests apparently do not cover the positive case - you could try if the behavior changes with different data, and if yes, include such data in the testing
• bff_methods = 'CLUSTER' is not working #92 would be an enhancement, the pipeline currently just only supports one of the 3 ways
• demuxem does not work #81 I guess removing the module is the best option, as there is no benefit in having a dysfunctional method in the pipeline, but you should keep track of the corresponding issue in the upstream repo and add it if the maintainers react
• Souporcell clustering panicked at 'no entry found for key' for some samples #57 Sometimes certain edge cases are super hard to reproduce, then there's not much one can do. Especially in this case the issue was created a long time ago, and the user is probably not interested in this anymore. How you handle this should be based on your estimation on how often this would occur and how problematic it actually is. I can imagine closing the issue with a 'cannot reproduce' and a note to re-open it if someone runs into it, and also a reminder to provide exact reproduction steps. This way, the next time someone runs into it (and reports it) you can handle it properly.

[Zethson]

#97 I tried to figure out why these tools only assign negative, but I wasn’t able to find a solution. Unfortunately, I haven’t received any response from the developers either. I suspect it might be because the test dataset is too small. What would you suggest I do?

It shouldn't be our fault so I think it's okay. A different test dataset would be great to ensure that things work as expected. If you can't find one within max 3 hours, let's not think too hard about it.

#81 Demuxem unfortunately doesn’t work, and other users have reported the same problem. I haven’t received any response from the developers here either. We could also consider removing the deconvolution tool altogether, what do you think?

Could you please try sending them an email as well if you haven't yet?

#57 Sometimes certain edge cases are super hard to reproduce, then there's not much one can do. Especially in this case the issue was created a long time ago, and the user is probably not interested in this anymore. How you handle this should be based on your estimation on how often this would occur and how problematic it actually is. I can imagine closing the issue with a 'cannot reproduce' and a note to re-open it if someone runs into it, and also a reminder to provide exact reproduction steps. This way, the next time someone runs into it (and reports it) you can handle it properly.

Yeah, I think just like you outlined it is perfectly fine.

[LuisHeinzlmeier]

Alright!

#91 I try to test the pipeline with another dataset. If it takes too long, I will skip that.

#81 I wrote a message to the Cumulus Support Team, which seems to maintain demuxem.

[LuisHeinzlmeier]

Regarding the remaining warnings of nf-core pipelines lint:

• <!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->: I would remove this TODO. All tools are already listed in CITATIONS.md, and other pipelines also did not specify the source of their test data, so it does not seem necessary to add it here.
• // TODO nf-core: Specify the paths to your full test data ( on nf-core/test-datasets or directly in : Do we need a test_full.config? If so, should it use a larger or "real-world" dataset?
• For the remaining warnings/TODO nf-core's, I don’t think they are necessary or applicable to us, so I would remove them. I also noticed that other pipelines (e.g. nf-core/rnaseq) leave TODO nf-core comments in their code.

@nictru, could you please review the remaining TODO nf-core comments and let me know if there’s anything we should consider, and whether I should keep or remove them? And I just wanted to remind you that you added a TODO in nextflow.config.

[nictru]

I would remove this TODO. All tools are already listed in CITATIONS.md, and other pipelines also did not specify the source of their test data, so it does not seem necessary to add it here.

What you should put there is a list of the most important packages/tools used in the pipeline, basically a subset of citations.md. If there is no sensible subset to put here, you can remove it

Do we need a test_full.config? If so, should it use a larger or "real-world" dataset?

Yes, this is a part of the CI process which only happens on pull requests from dev to main. There, the pipeline is executed on AWS with large data, to see if it works in the cloud and with larger data. Maybe you can use the data used in the publication?

@nictru, could you please review the remaining TODO nf-core comments

Yes, will do in the evening or in the next days

[nictru]

Actually, having the test_full shouldn't feel like a chore here - as far as I know, we've never tested the pipeline on real-life datasets, and it would be quite bad if we were to do the 1.0 release and then people come along and say it's unusable. Same for the cloud - many nextflow/nf-core users run their pipelines in the cloud, and sometimes there are small issues with compatibility there.
Now is a good time to catch these issues

[LuisHeinzlmeier]

Yes, this is a part of the CI process which only happens on pull requests from dev to main. There, the pipeline is executed on AWS with large data, to see if it works in the cloud and with larger data. Maybe you can use the data used in the publication?

@Zethson, do you still have the mpsMS-dataset somewhere? In the study, you mentioned that you ran the rescue mode of hadge with it. The other datasets didn't really help me at first, as they were apparently only intended for genetic or hashing mode.

[Zethson]

Only @bio-la would know.