ICGC VCF v4.1 compatibility failure

[josrpowe208]

Hello,

I'm reporting this issue both here and with the International Cancer Genome Consortium. Running the SigProfilerTopography algorithm fails on the open access VCFs of the ICGC (Traceback below):

Traceback (most recent call last): File "~/gdc/sigprofiler_topography.py", line 52, in <module> main() File "/gdc/sigprofiler_topography.py", line 38, in main
topography.runAnalyses(genome,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.local/lib/python3.12/site-packages/SigProfilerTopography/Topography.py", line 1789, in runAnalyses
matrices = matGen.SigProfilerMatrixGeneratorFunc(jobname, genome, inputDir, plot=False, seqInfo=seqInfo, chrom_based=chrom_based)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.local/lib/python3.12/site-packages/SigProfilerMatrixGenerator/scripts/SigProfilerMatrixGeneratorFunc.py", line 1441, in SigProfilerMatrixGeneratorFunc
) = matGen.catalogue_generator_single(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.local/lib/python3.12/site-packages/SigProfilerMatrixGenerator/scripts/MutationMatrixGenerator.py", line 593, in catalogue_generator_single
mnv_seq += tsb_ref[chrom_string[l - 1]][1]
~~~~~~~~~~~~^^^^^^^
IndexError: index out of range`

The VCF version is 4.1. vcc-validator from VCFtools has been run with 0 errors.

Let me know if there is anything else that I need to provide.

[burcak-otlu]

Dear @josrpowe208,

Thank you very much for trying SigProfilerTopography and creating this issue. Sorry for the late response.

I had lost my 2FA verification due to my Skype number is no longer available.
With a newly created GitHub account, I can respond to you.

If possible, could you please provide me with the input open-access VCFs of the ICGC or the link so that I can try to replicate this issue on my side? You can send them to my current email address: burcako[at]metu.edu.tr

Best wishes,
Burcak

[josrpowe208]

Hi Burcak,

Thank you. The data repository for the ICGC is found here. But I will still send a small sample of the files that fail to your email address.

I have some additional information as well. This issue seems to be specific to VCFs that were created by samtools:bcftools pipeline. There are other VCFs created by the GATK:Mutect2 pipeline that fail vcftools validator, but this is a known issue that doesn't seem to be high priority for the GATK team, GATK Issue #6931. I've run the fix suggested in that thread and those files seem to work fine.

Best,
Joseph

[burcak-otlu]

Dear @josrpowe208,

Please correct me if I'm wrong. The issue was due to problematic or unvalidated VCF files; with fixed VCF files, you were able to run SPT successfully.

Best wishes,
Burcak

[josrpowe208]

This issue occurs with validated VCFs generated by the bcftools pipeline.

[burcak-otlu]

Dear @josrpowe208,

Thank you again for the additional information, the link to the ICGC data repository, and sample VC files. Your detailed explanation about the issue being specific to validated VCFs from the bcftools pipeline is extremely helpful.

The traceback points to a specific line: mnv_seq += tsb_ref[chrom_string[l - 1]][1].

The error occurs in SigProfilerMatrixGenerator, so the VCFs likely contain some data that this dependency's code isn't expecting, causing it to try to access an index that doesn't exist. This could be related to how the bcftools pipeline formats specific fields in the VCF.

It seems that there is a discrepancy between the VCF file's content and the assumptions made by the SigProfilerMatrixGenerator code.

I will forward this issue to SigProfilerMatrixGenerator.

Best wishes,
Burcak

[mdbarnesUCSD]

Thanks @burcak-otlu and @josrpowe208 , it appears that this issue originates from SigProfilerMatrixGenerator. I will proceed by transferring this issue at this time.