Skip to content

Fix input/output duplication in file_mapper#1408

Draft
lbeckman314 wants to merge 15 commits intodevelopfrom
chore/fix_file_mapper
Draft

Fix input/output duplication in file_mapper#1408
lbeckman314 wants to merge 15 commits intodevelopfrom
chore/fix_file_mapper

Conversation

@lbeckman314
Copy link
Copy Markdown
Contributor

@lbeckman314 lbeckman314 commented May 8, 2026

Important

This PR is built off of PR #1407 and should be merged to it!

TODO: How to do so against forks? Just merge to develop afterwards #1407 is merged?

Overview 🌀

This PR is an initial (un-tested!) pass at resolving the following file_mapper error introduced when merging Commit 5b95fa0:

Error: kubernetes system error (JobCreationFailed):
Failed to create Kubernetes job: Job.batch "<TASK ID>-0" is invalid:
spec.template.spec.containers[0].volumeMounts[2].mountPath: Invalid value: "/data": must be unique
volumeMounts:
  - name: funnel-storage-<TASK ID>
    mountPath: /tmp
    subPath: <TASK ID>/tmp
  - name: funnel-storage-<TASK ID>
    mountPath: /work          <----- Duplicate mountPath ⚠️
    subPath: <TASK ID>/work
  - name: funnel-storage-<TASK ID>
    mountPath: /work          <----- Duplicate mountPath ⚠️
    subPath: <TASK ID>/work
  - name: funnel-storage-<TASK ID>
    mountPath: /local/work/tmp/.../bin
    subPath: <TASK ID>/local/work/tmp/.../bin

volumes:
- name: funnel-storage-<TASK ID>
persistentVolumeClaim:
  claimName: funnel-worker-pvc-<TASK ID>

This duplication is only happening if we have both inputs and outputs (regardless of the number).

Steps to Reproduce

To test it, create any TES task with at least one input and output.

Credit 🤝

This issue was first identified and documented by @nss10 and the CTDS team, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants