You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have alot of small s3 files and the delete operation ins blueshift is the biggest bottleneck right now.
Any suggestions since i dont know implications of using something like pmap offhand?
We have alot of small s3 files and the delete operation ins blueshift is the biggest bottleneck right now.
Any suggestions since i dont know implications of using something like pmap offhand?
(defrecord Cleaner [credentials bucket cleaner-ch]
Lifecycle
(start this
(thread
(loop [](when-let [m %28<!! cleaner-ch%29]
%28doseq [key %28:files m%29]
%28info "Deleting" %28str "s3://" bucket "/" key))
(delete-object credentials bucket key))
(recur))))
this)
(stop this
this))