[Discussion] Should we create Job for each Replication to download/sync files

[kerthcet]

Right now, we'll download/sync the files in each reconciliation, which leads to a long running process, which may lead to obj dropped (the default ratelimiter queue has 100 bucket size by default) or slow reconciliation, which could be mitigated by increase the concurrency worker number. We have a discussion in slack https://kubernetes.slack.com/archives/C02MRBMN00Z/p1731402269033139.

Maybe we could use a job to download/sync the files, however, this may lead to latency and another round of job scheduling. And essencially agent should be responsible for this, rather than another job.

[kerthcet]

/kind question