Eliminate orphan objects #3732

nidhi-soni1104 · 2024-03-21T08:35:01Z

What this PR does / why we need it:

We need a rake task to go over all tables with the tenant_id field and make check whether that tenant still exists, reporting and optionally removing these on the way.

complete description is in JIRA

Which issue(s) this PR fixes

https://issues.redhat.com/browse/THREESCALE-10855

Note: I have commented the deleted line for now but added puts so whoever will run it while reviewing can see the puts

lib/tasks/database_cleanup.rake

nidhi-soni1104 · 2024-04-08T06:02:23Z

@akostadinov Updated

mayorova · 2024-04-10T13:55:11Z

lib/tasks/database_cleanup.rake

    # Tables to exclude from orphaned objects check
-    excluded_tables = ['accounts', 'audits', 'categories', 'category_types', 'cms_templates', 'legal_term_acceptances', 'legal_term_bindings',
-                      'legal_term_versions', 'proxy_logs', 'schema_migrations', 'service_cubert_infos', 'slugs', 'taggings', 'tags']
+    excluded_tables = ['cms_templates']


Why exclude this?

I think this exclusion can be removed

I removed the exclusion, but I don't know if a coincidence or not - the task got stuck on Found orphaned objects in cms_templates: when I tried it in staging...

If it doesn't persist, let's think it's a coincidence. Why should cms_templates be special?

mayorova · 2024-04-10T14:22:36Z

lib/tasks/database_cleanup.rake

+
+      if orphaned_objects.exists?
+        puts "Found orphaned objects in #{model.table_name}:"
+        orphaned_objects.each { |obj| puts "- ID: #{obj.id}, Tenant ID: #{obj.tenant_id}" }


I'd use find_each here instead of each to use batching.

Addressed in ee9e0a8

jlledom · 2024-06-10T12:30:58Z

I rebased this, It was too old.

jlledom

I wrote a couple of comments but this LGTM. I haven't tried to run the script locally though.

It would be good to write a couple of tests. Do we have tests for rake tasks?

lib/tasks/database_cleanup.rake

jlledom · 2024-06-10T12:35:14Z

lib/tasks/database_cleanup.rake

    # Tables to exclude from orphaned objects check
-    excluded_tables = ['accounts', 'audits', 'categories', 'category_types', 'cms_templates', 'legal_term_acceptances', 'legal_term_bindings',
-                      'legal_term_versions', 'proxy_logs', 'schema_migrations', 'service_cubert_infos', 'slugs', 'taggings', 'tags']
+    excluded_tables = ['cms_templates']


I think this exclusion can be removed

lib/tasks/database_cleanup.rake

github-actions · 2024-07-11T03:34:59Z

This PR is stale because it has not received activity for more than 30 days. Remove stale label or comment or this will be closed in 15 days.

github-actions · 2024-08-12T03:35:10Z

This PR is stale because it has not received activity for more than 30 days. Remove stale label or comment or this will be closed in 15 days.

mayorova · 2024-08-16T11:22:15Z

lib/tasks/multitenant/tenants.rake

+    task :cleanup_orphans, [:mode] => :environment do |_task, args|
+      puts 'Checking and removing orphaned objects...'
+
+      destroy = args[:mode] == "destroy"


So, the idea is that by default calling the task would only print the logs.
To actually destroy stuff, one would need to run

bundle exec rake "multitenant:tenants:cleanup_orphans[destroy]"

By the way, the logs are probably too verbose... Or maybe that's just the first time, as we have lots of orphans. Once the big batch is deleted, probably it will be fine.

mayorova · 2024-08-16T11:24:59Z

lib/tasks/multitenant/tenants.rake

+
+      puts "WARNING: the found orphan objects will be destroyed" if destroy
+
+      provider_account_ids = Account.where(provider: true).pluck(:id) + [Account.master.id]


I've added Master's ID here, to avoid deleting master's objects. Specifically, the master itself was being reported as an orphan 😅 (because it does not have provider: true)

Found orphaned objects in accounts: - ID: 1, Tenant ID: 1

And we probably don't want to destroy the master 🙃

You're too goodhearted..

But this could be hundreds of thousands of objects. Can't we filter this out in the query itself?

mayorova · 2024-08-16T11:25:53Z

lib/tasks/multitenant/tenants.rake

+
+          if destroy
+            puts "Destroying orphan #{model.table_name}..."
+            orphaned_objects.in_batches(of: 100).destroy_all


Added batched deletion, but still not sure if it still may have negative effects on the DB...

We might also need some more logging to see whether the operation was succeeded.

we need to revisit how we iterate over many objects

Added batched deletion, but still not sure if it still may have negative effects on the DB...

For sure batching is an improvement, what else could we do? We should be sure to run this after the billing jobs have finished and heads up ops.

Ideally we would use the new Sidekiq 7.3 iteration jobs. But if we just do in a rake task in the foreground, batches would be better.

If we want to run thins in produciton, we most likely need to use this iteration sidekiq job type. Because running haevy deletion in production may overburden the database. While recent hardware is much faster, I'm not sure how good this would be.

lib/tasks/multitenant/tenants.rake

akostadinov · 2024-09-17T16:40:34Z

lib/tasks/multitenant/tenants.rake

+
+      provider_account_ids = Account.where(provider: true).pluck(:id) + [Account.master.id]
+
+      ApplicationRecord.descendants.each do |model|


We can filter the classes further by filtering out children STI classes. Because if we run the query with the base STI class, running with the children will be redundant. SOmething along the lines of:

def base_models all_models = ApplicationRecord.descendants.select(&:arel_table).reject(&:abstract_class?) all_models.select! { _1.attribute_names.include? "tenant_id"} # we only want base STI classes, not the children all_models.select! do |model| base_class = model.base_class # either current model is the base_class or we can't find a base class amongst the discovered models /which would be very weird/ base_class == model || all_models.none? { |potential_parent| potential_parent == base_class } end end

Good idea, thanks!

FYI adding a base_models helper (app/lib/three_scale/models.rb) method in #3958

github-actions · 2025-07-14T03:54:23Z

This PR is stale because it has not received activity for more than 30 days. Remove stale label or comment or this will be closed in 15 days.

nidhi-soni1104 self-assigned this Mar 21, 2024

akostadinov reviewed Mar 21, 2024

View reviewed changes

lib/tasks/database_cleanup.rake Outdated Show resolved Hide resolved

akostadinov reviewed Mar 21, 2024

View reviewed changes

lib/tasks/database_cleanup.rake Outdated Show resolved Hide resolved

mayorova reviewed Apr 10, 2024

View reviewed changes

github-actions bot added the Stale label May 11, 2024

3scale deleted a comment from github-actions bot May 11, 2024

github-actions bot removed the Stale label May 12, 2024

jlledom force-pushed the THREESCALE-10855-eliminate-orphan-objects branch from 89a470f to 8e47c60 Compare June 10, 2024 12:30

jlledom reviewed Jun 10, 2024

View reviewed changes

github-actions bot added Stale and removed Stale labels Jul 11, 2024

github-actions bot added the Stale label Aug 12, 2024

mayorova force-pushed the THREESCALE-10855-eliminate-orphan-objects branch 2 times, most recently from 23b6efa to ee9e0a8 Compare August 16, 2024 11:17

mayorova reviewed Aug 16, 2024

View reviewed changes

github-actions bot removed the Stale label Aug 17, 2024

jlledom previously approved these changes Aug 20, 2024

View reviewed changes

lib/tasks/multitenant/tenants.rake Outdated Show resolved Hide resolved

lib/tasks/multitenant/tenants.rake Outdated Show resolved Hide resolved

mayorova dismissed jlledom’s stale review via 6af9bf9 September 16, 2024 09:25

mayorova force-pushed the THREESCALE-10855-eliminate-orphan-objects branch 2 times, most recently from 4d3c822 to e2c5276 Compare September 16, 2024 09:45

akostadinov reviewed Sep 17, 2024

View reviewed changes

mayorova force-pushed the THREESCALE-10855-eliminate-orphan-objects branch 2 times, most recently from eea080b to a9d31ad Compare September 30, 2024 14:08

github-actions bot removed the Stale label Mar 11, 2025

github-actions bot added the Stale label Apr 12, 2025

akostadinov removed the Stale label Apr 12, 2025

3scale deleted a comment from github-actions bot Apr 12, 2025

github-actions bot added the Stale label May 13, 2025

3scale deleted a comment from github-actions bot May 13, 2025

akostadinov unassigned nidhi-soni1104 May 13, 2025

akostadinov removed the Stale label May 13, 2025

github-actions bot added the Stale label Jun 13, 2025

3scale deleted a comment from github-actions bot Jun 13, 2025

akostadinov removed the Stale label Jun 13, 2025

github-actions bot added the Stale label Jul 14, 2025

github-actions bot closed this Jul 29, 2025

akostadinov reopened this Jul 29, 2025

akostadinov removed the Stale label Jul 29, 2025

github-actions bot added the Stale label Aug 29, 2025

3scale deleted a comment from github-actions bot Aug 29, 2025

akostadinov removed the Stale label Aug 29, 2025

github-actions bot added the Stale label Sep 29, 2025

3scale deleted a comment from github-actions bot Sep 29, 2025

akostadinov removed the Stale label Sep 29, 2025

github-actions bot added the Stale label Oct 30, 2025

akostadinov removed the Stale label Oct 30, 2025

3scale deleted a comment from github-actions bot Oct 30, 2025

github-actions bot added the Stale label Nov 30, 2025

3scale deleted a comment from github-actions bot Dec 1, 2025

akostadinov removed the Stale label Dec 1, 2025

github-actions bot added the Stale label Jan 1, 2026

3scale deleted a comment from github-actions bot Jan 5, 2026


		puts "WARNING: the found orphan objects will be destroyed" if destroy

		provider_account_ids = Account.where(provider: true).pluck(:id) + [Account.master.id]


		provider_account_ids = Account.where(provider: true).pluck(:id) + [Account.master.id]

		ApplicationRecord.descendants.each do \|model\|

Eliminate orphan objects #3732

Are you sure you want to change the base?

Eliminate orphan objects #3732

Uh oh!

Conversation

nidhi-soni1104 commented Mar 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nidhi-soni1104 commented Apr 8, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jlledom commented Jun 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jlledom left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jul 11, 2024

Uh oh!

github-actions bot commented Aug 12, 2024

Uh oh!

mayorova Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mayorova Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akostadinov Aug 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mayorova Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

akostadinov Sep 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 14, 2025

Uh oh!

Reviewers

Assignees

Labels

nidhi-soni1104 commented Mar 21, 2024 •

edited

Loading

jlledom commented Jun 10, 2024 •

edited

Loading

jlledom left a comment •

edited

Loading

mayorova Aug 16, 2024 •

edited

Loading

mayorova Aug 16, 2024 •

edited

Loading

akostadinov Aug 17, 2024 •

edited

Loading

mayorova Aug 16, 2024 •

edited

Loading

akostadinov Sep 17, 2024 •

edited

Loading