Skip to content

Fixed duplicate id error when creating VM work jobs#3871

Merged
yadvr merged 1 commit into
apache:4.13from
shapeblue:left-over-vmjobs
Feb 12, 2020
Merged

Fixed duplicate id error when creating VM work jobs#3871
yadvr merged 1 commit into
apache:4.13from
shapeblue:left-over-vmjobs

Conversation

@Spaceman1984
Copy link
Copy Markdown
Contributor

@Spaceman1984 Spaceman1984 commented Feb 7, 2020 •
edited
Loading

Description

This PR removes related VMWorkjob records when the async_job table is cleaned out.

Fixes: #3829

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

This was tested by taking these steps:

  • Create an instance.
  • Check the async_job and vm_work_job table for matching rows.
  • Set the job.expire.minutes global setting to 1 minute.
  • Wait a minute.
  • Check the async_job and vm_work_job table for matching rows to be removed.

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Feb 7, 2020

@blueorangutan package

@yadvr yadvr added this to the 4.14.0.0 milestone Feb 7, 2020
@yadvr yadvr added the type:bug label Feb 7, 2020
@yadvr
Copy link
Copy Markdown
Member

yadvr commented Feb 7, 2020

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 7, 2020

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@DaanHoogland
Copy link
Copy Markdown
Contributor

DaanHoogland commented Feb 7, 2020

As this is clearly a bug, can you target 4.13 please, @Spaceman1984 ?

DaanHoogland
DaanHoogland approved these changes Feb 7, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@DaanHoogland DaanHoogland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm, @richardlawley can you test this patch?

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

Spaceman1984 commented Feb 7, 2020

Tageting 4.13 @DaanHoogland

@Spaceman1984 Spaceman1984 changed the base branch from master to 4.13 February 7, 2020 10:28
@Spaceman1984
Copy link
Copy Markdown
Contributor Author

Spaceman1984 commented Feb 7, 2020

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 7, 2020

@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 7, 2020

Packaging result: ✖centos6 ✖centos7 ✔debian. JID-783

@DaanHoogland DaanHoogland modified the milestones: 4.14.0.0, 4.13.1.0 Feb 7, 2020
@richardlawley
Copy link
Copy Markdown
Contributor

richardlawley commented Feb 7, 2020

I don't have an easy way to test this, but the fix looks suitable.

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 7, 2020

Packaging result: ✖centos6 ✔centos7 ✔debian. JID-785

@DaanHoogland
Copy link
Copy Markdown
Contributor

DaanHoogland commented Feb 7, 2020

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 7, 2020

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 7, 2020

Trillian test result (tid-927)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28025 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3871-t927-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 76 look OK, 1 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_02_vpc_privategw_static_routes Failure 181.73 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 179.27 test_privategw_acl.py
test_04_rvpc_privategw_static_routes Failure 260.48 test_privategw_acl.py

@weizhouapache weizhouapache mentioned this pull request Feb 7, 2020
Closed
@yadvr
Copy link
Copy Markdown
Member

yadvr commented Feb 12, 2020

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 12, 2020

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

blueorangutan commented Feb 12, 2020

Packaging result: ✖centos6 ✖centos7 ✔debian. JID-822

GabrielBrascher
GabrielBrascher approved these changes Feb 12, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@GabrielBrascher GabrielBrascher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @Spaceman1984, code LGTM (I have not tested it though).

@yadvr yadvr merged commit a443f31 into apache:4.13 Feb 12, 2020
ustcweizhou pushed a commit to ustcweizhou/cloudstack that referenced this pull request Feb 28, 2020
@Spaceman1984 @ustcweizhou
jobs: fixed duplicate id error when creating VM work jobs (apache#3871)
f353dd9 
This PR removes related VMWorkjob records when the async_job table is cleaned out.

Fixes: apache#3829
Slair1 added a commit to ippathways/cloudstack that referenced this pull request Jan 5, 2021
@Slair1
Fixed duplicate id error when creating VM work jobs
6be203d 
AsyncJobManager contains code to expunge async job records after a configurable time (24h default). Some async jobs have corresponding vm_work_job entries, which are created using the generated key of async_job. The expunge code only removes the async_job record, leaving dangling vm_work_job rows.

Because the supported version of mysql recalculates autonumber sequences based on the last current row, this can cause async jobs to be created with IDs referencing existing rows in vm_work_job, which causes a primary key violation if this clashes, showing the following log message:

Unable to schedule async job for command com.cloud.vm.VmWorkStart, unexpected exception.
javax.persistence.EntityExistsException: Entity already exists

apache#3829
apache#3871
nlgordon pushed a commit to ippathways/cloudstack that referenced this pull request Aug 2, 2022
@Slair1 @nlgordon
Fixed duplicate id error when creating VM work jobs
5f0467e 
AsyncJobManager contains code to expunge async job records after a configurable time (24h default). Some async jobs have corresponding vm_work_job entries, which are created using the generated key of async_job. The expunge code only removes the async_job record, leaving dangling vm_work_job rows.

Because the supported version of mysql recalculates autonumber sequences based on the last current row, this can cause async jobs to be created with IDs referencing existing rows in vm_work_job, which causes a primary key violation if this clashes, showing the following log message:

Unable to schedule async job for command com.cloud.vm.VmWorkStart, unexpected exception.
javax.persistence.EntityExistsException: Entity already exists

apache#3829
apache#3871
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yadvr yadvr yadvr approved these changes

@DaanHoogland DaanHoogland DaanHoogland approved these changes

@GabrielBrascher GabrielBrascher GabrielBrascher approved these changes

@andrijapanicsb andrijapanicsb Awaiting requested review from andrijapanicsb

@weizhouapache weizhouapache Awaiting requested review from weizhouapache

@wido wido Awaiting requested review from wido

Assignees

No one assigned

Labels

component:management-server type:bug

Projects

None yet

Milestone

4.13.1.0

Development

Successfully merging this pull request may close these issues.

Unable to schedule async job - dangling vm_work_job rows

7 participants

@Spaceman1984 @yadvr @blueorangutan @DaanHoogland @richardlawley @GabrielBrascher @svenvogel