Not able to run (unpack requires a buffer of 4 byte) (no sane player data available in ds: remaster)

[Torbilicious]

Hi!
I am new to python and this topic so please pardon me if this is a simple error on my side.

Problem: Whenever i try to start dsfp i get the error:
File "XXX\dsfp\dsfp\dsfp.py", line 155, in get_blocks_metadata block_size = struct.unpack('I', block)[0] struct.error: unpack requires a buffer of 4 bytes

I have tried all combinations of python 2/3 and 32/64 bit versions. When i run the python code in the Linux subsystem it works.

[tarvitz]

Hi,

Well python code has no safe checks everythere :), so basically it tries to read data following way:

import struct
struct.unpack('I', b'\x00\x00\x00\xc0')[0]  # valid code because of 4 bytes accessing

struct.unpack('I', b'')[0]  # this one is invalid and raises error: unpack requries a bytes object of length 4

This is not python related issue I guess, the most closest guess is broken or malformed darksouls save file. For example darksouls II and III has other way to pack data inside.

[paimonpal]

Hello, Tarvitz. I have the same issue regarding the topic.
Error in code

>>> data = ds.get_stats()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "XXX\dsfp-master\dsfp\dsfp.py", line 209, in get_stats
    self.get_active_slots_amount()
  File "XXX\dsfp-master\dsfp\dsfp.py", line 176, in get_active_slots_amount
    for idx, header in enumerate(self.get_blocks_metadata()):
  File "XXX\dsfp-master\dsfp\dsfp.py", line 155, in get_blocks_metadata
    block_size = struct.unpack('I', self._fo.read(4))[0]
struct.error: unpack requires a buffer of 4 bytes

I'm new to python and need assistance understanding how to fix. I know that my save file is fine but, no matter what i try it doesnt want to work.
I have python 3.7 and pip v18
^ newest of both :)
hopefully this isn't the issue.

[tarvitz]

@paimonpal Ok, send me your save file I'll take a look. Probably remaster could have some offsets of file format changes (but as far as I got from software changed nothing, but I didn't inspected far too long)

In addition there's a 010 editor templates that could bring the light on a dark souls file save file format to understand it and use it any language you use :)

[paimonpal]

DRAKS0005.zip

This is my current save that I have been playing on with no problems. Same one I used in the test and when the error popped up in code.

[tarvitz]

@paimonpal, Got it, well I've checked the file. Basically it used save BND4 container file storage, however data is protected / compressed / encrypted. That's why dsfp does not work.

If there's somebody who can provide information about this I'll adjust this tiny reader as well.

Unfortunately now I'm very far away from trying to guess what staff has been used to pack dark souls remastered edition data like this into save file.

[paimonpal]

@tarvitz yeah it's very strange.. the furthest i got with another github code set was unpacking the data slots within the save file. But unfortunately the files are listed as (file) and are in an unknown state. Tried different techniques but to no avail. These save files are compressed hardcore it looks like. The only things that work are cheatengine currently... maybe could write a set of code to run like cheatengine but a different intereface.. much cleaner and time saving. It might be possible but it would have to run when the game is running as a process.

the only way is to target the game files through the game itself as a sort of buffer.. since its reading the file and has it open through the process. Much like how texmod works with grabbing the files through the game. Maybe this is the best way for games like these. There's always a risk involved such as soft bans and such but I believe someone could come up with a way to do all this.

[tarvitz]

I'll try to take a look upon content, but as far as I am not professional software reverse engineer this could remain unsealed :).
As far as I told bnd4 container format with whole metadata block is plain and written in "unencrypted" format, which is really interesting, and I doubt data is compressed as far as slot size has increased up to 384 bytes and there are a lot of spaces which compression could shrink, so data is encrypted by something.

[tarvitz]

Here's another update:

data stores in memory same as in prepare to die edition (99% accuracy). So basically 4 byte offset before current souls stores player's level.

[paimonpal]

@tarvitz Okay, awesome! i'll be on the lookout if anything arises or maybe you can get it working. I'm completely useless with python right now.

[tarvitz]

As far as I got new info about remaster:

• it's potentially use (99.9%) bcrypt to store data (don't know which algo it uses yet)
• it has anti debug protection, so there's no fast way to get encryption key fast enough
• encryption key is independent, so you can use save files from another "accounts" freely

So if I can get the key (unfortunately I have not much time to make it) or somebody would help with it it would be great.

[Torbilicious]

@tarvitz have you taken a look at what is done over here?

He seems to be able to decrypt the player slots.

[tarvitz]

Need to check if it works, could probably be as far as remaster use static key for a 99% for encryp/decrypt operations. I'll try to play with it this or next weekend, will post updates a little bit later.

@Torbilicious thx for a finding it :)

[tarvitz]

Unfortunately was busy last two weekends :[ (still need time to build unpack solution around python) .However if unpacker works correctly and decrypt data without file structure modification dsfp should work normally.