Skip to content

Backward compatible variant for migration from Scala 2.12 to 2.13#467

Open
vigoo wants to merge 4 commits into
twitter:developfrom
vigoo:backward-compatible-213
Open

Backward compatible variant for migration from Scala 2.12 to 2.13#467
vigoo wants to merge 4 commits into
twitter:developfrom
vigoo:backward-compatible-213

Conversation

@vigoo

@vigoo vigoo commented Aug 31, 2020

Copy link
Copy Markdown

The default implementation in Chill does not maintain binary compatibility between Scala version 2.12 and 2.13. I guess this is an intentional decision, as even a separate test data set is provided for the two versions.

Earlier this year we migrated a live service to Scala 2.13 that was using Chill with Akka Persistence and clustering, so had to keep binary compatibility to do a live upgrade. This PR contains an alternative ScalaKryoInstantiator for Scala 2.13 that preserves compatibility with 2.12 in both directions. The compatibility does not mean binary equivalence in all cases but roundtrip serialization between the two Scala versions is possible.

This was heavily tested manually and runs since months in production but there are no tests included yet. If this is something you would like to have in the core library as it may help others migrations, I'm happy to add them.

@CLAassistant

CLAassistant commented Aug 31, 2020

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov-commenter

codecov-commenter commented Aug 31, 2020

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.69%. Comparing base (5fc80e9) to head (51ded44).
⚠️ Report is 212 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #467   +/-   ##
========================================
  Coverage    91.69%   91.69%           
========================================
  Files           42       42           
  Lines         1396     1396           
  Branches        35       35           
========================================
  Hits          1280     1280           
  Misses         116      116           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@johnynek

Copy link
Copy Markdown
Contributor

Thanks for the PR.

I'm willing to merge this, but I'd like to have some tests. For instance, one idea would be we could check in a file that has been serialized with one of all these items in 0.9.2 with 2.12 or something, and then assert that we deserialize it.

I get very nervous about code that people depend on that has no tests in the repo.

@vigoo

vigoo commented Aug 31, 2020

Copy link
Copy Markdown
Author

I fully agree and will add some tests!

@johnynek johnynek mentioned this pull request Aug 31, 2020

@regadas regadas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Would you mind rebasing as well and formatting with scalafmt? +scalafmtAll

val nameId = classToNameId.get(placeholderType, -1)
if (nameId != -1) {
output.writeVarInt(nameId, true)
return

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎨 could we rewrite this without using return?


override def write(kryo: Kryo, output: Output, `object`: T): Unit = {
var length: Option[Int] = None
for (field <- getFields) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎨 use while?

val start = kryo.readClassAndObject(input)
val step = kryo.readClassAndObject(input)

for (field <- getFields) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎨 use while? yields better perf.

@vigoo

vigoo commented Oct 2, 2020

Copy link
Copy Markdown
Author

Update: I found some issues, will push fix and tests when have time.

@johnynek

Copy link
Copy Markdown
Contributor

take a look at #514 --

that would break, significantly, binary compatibility, and I assume the serialized data compatibility.

you might want to weight in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants