Tutorial/Screencap

[derekdinardo]

Hi all,

I'd love to start contributing to the database, but for some reason, following the wiki just isn't clicking for me. I was wondering if there is anyone who would be willing to either screenrecording or hop on a quick zoom where they show how to use the tool?

[lfoust]

Hi @derekdinardo - I would be happy to hop on a call with you to help out. Getting a good screen recording or a tutorial is something that I think would be extremely beneficial but I just haven't had the time to make them. Feel free to email me at web@thediscdb.com and we can figure out a time to meet. Thanks!

[TheL1GHTSiDE]

I would greatly appreciate a tutorial as well! I want to contribute but don't know how.

[DanForever]

Here's a rough and ready guide that I think we could use to improve the wiki:

TMDB:

New Release Slug:
This is the name of the subfolder for this "release" of the movie/show. It's also used as part of the URL when checking out this particular release on the website.
You can name it whatever you want, but my preference is "Year-disctype-countrycode" (it might be worth trying to standardise the site to a convention of some kind)

UPC:
This is the number represented by the barcode on the back of the box.
You can also typically find it on the amazon product page under "manufacturer reference"

ASIN:
"Amazon standard identification number" - also on the amazon product page

Release Date:
Probably better thought of as "publication date for this specific release"
YYYY-MM-DD

Front Cover:
URL to an image of the front cover. A quick cheat is to copy the url for the cover image from the amazon product page

Back Cover:
Same as Front cover, but for the other side of the box

New Release Name:
This is the name of the release, so it could be the same as the show/movie, or it could be something like "Summer blockbuster movie - director's cut".
Usually you want to put the name that's on the box here verbatim.

Disc Name:
You can call this whatever you want. Generally speaking if the discs have names on them use that (i.e. "4k" "bonus features"), otherwise you can default to something like "Disc 1"

Disc Slug:
Again, you can call it anything. This is used as part of the URL for viewing this disc on the site, so it needs to be URL friendly (so no spaces or characters forbidden by urls).
Generally speaking, try to keep it in line with the disc name. For tv shows, I recommend the following convention: disctype-SXXDYY where "disctype" is bd/dvd/4k and XX is the season number and YY is the disc number (e.g. bd-s01d01)

"Release Folder?" on Finalize:
This is the absolute path to the release you just imported (including release slug). So it might look something like C:\TheDiscDB\data\series\Sherlock (2010)\2010-bd-uk

[dill0wn]

As someone who's submitted a few releases, I like this and this generally matches what i've been doing.

It would be nice if some of these changes made it into ImportBuddy itself. For instance having ImportBuddy generate the release slug for you since it should be somewhat deterministic or follow a convention

[pwl4909]

I recently went through the website contribution process. I have not used Import Buddy locally. I wanted to document the steps, and my thoughts on improvements on workflow.

As is, the https://thediscdb.com/contribute Contribute a Movie button:

1. Required to authenticate with github.
2. Find Existing title on the site which you are adding a different release, OR the TMBD number
3. Add Release Details: (This step is hinted at on the contribute page)
   ASIN:
   UPC/EAN:
   Release Date:
   Front Image:
   Back Image:
   Release Title:
   Release Slug:
   Local: en-us
   Region Code: 1
4. Add Discs in Release:
   • Place disc in try, Allow hash to be calculated. (This allows others with the same release to confirm the content is the same)
5. Generate makemkv logs and upload:
   • With disc still in tray, Generate txt logs with makenkvcon and upload:
   • (Choose one) (Commands are provided with unique links for your contribution)
   • Power Shell:
     • Invoke-WebRequest -Uri "https://thediscdb.com/api/contribute/lzl/discs/unique123/logs" -Method POST -ContentType "text/plain" -Body ((& 'C:\Program Files (x86)\MakeMKV\makemkvcon64.exe' --minlength=0 --robot info disc:0) | Out-String)
   • Bash:
     • makemkvcon --minlength=0 --robot info disc:0 2>&1 | curl -X POST -H "Content-Type: text/plain" --data-binary @- https://thediscdb.com/api/contribute/lzl/discs/unique123/logs
   • Local .txt generation:
     • Open gitbash in the makemkv install folder: /c/Program Files (x86)/MakeMKV (This one done on windows with gitbash, alternatives exist)
     • ./makemkvcon.exe --minlength=0 --robot info disc:0 2>&1 > ~/Path/To/Filename.txt
6. Identify Disc Items:
   • If release exists on TheDiscSB, No need to contribute, take advantage of the existing entry and the note below to rename your content appropriately. Note: in expert mode makemkv allows filenames to be renamed before downloading. This can be convenient in your workflow.
   • To contribute a new release or title, use makemkv to get contents from disc, then manually verify content with (SN), size, duration, and label properly. On the contribution page. This can be tricky on some titles that try to obfuscate the content, or have multiple of the same movies. In simple cases this is the movie with chapters and the movie without chapters. Labeling this properly now adds the most value for the users of your contributions. So thank you for taking the time to get it right!

Note: TheDiscDB uses the Source Title Number (SN) as filename for content on disc (00001.mpls). This can be added to the makemkv filename output for easier association.
- Prefernceces > General > Expert Mode (Check)
- Preferences > Advanced: (Update the following)
- Output file name template: {NAME1}{-:CMNT1}{-:DT}{title:+DFLT}{_t:N2}{_:SN}

The above is my notes on the web contribution steps. I wanted to provide documentation of the existing steps, and helpful insights I learned from various posts. The below section is my feedback on this process, to add the most value.

1. I would like to decouple the process/workflow into being able to batch process, having multiple stages in progress. Mostly because some steps require a lot of system time, and others require data I can collect without the physical disc in hand. So being able to do all steps locally and report results later is ideal.
2. Allow for steps to be performed locally. I would like to locally calculate the MD5 hash of the disk and provide it to the web contribution. From a security sense, allow a string of the hash to be uploading is a good alternative to allowing access to disc.
3. Document importBuddy better for local use. The install exe is something I am not interested in using. The steps I have described above are not complicated enough to warrant an exe imo. Better documenting the steps to use import buddy without an exe would be preferable.
4. Some integration into the workflow to support Jellyfin or Plex or other naming convention. This is likely the end goal for most users/contributors of TheDiscDB. Allowing a streamlined approach would be nice.
5. There appears to be several sources of information to cobble together a decent workflow. I think contributions will take off if documentation has a single source, and other sources just link to that source. For example a lot of the valuable information for TheDiscDB or makeMKV are scattered across various posts and sites. can we centralize this documentation and workflow information?

@lfoust @DanForever Does anything here resonate? I am willing to collaborate/contribute to documenting and optimizing the workflow.

[lfoust]

@pwl4909 - these are great notes! Thanks for taking the time. I agree that good instructions (and some clear videos) would go a long way to simplifying the whole process.

on the subject of ImportBuddy and the local process - that process is being deprecated and is really only used for those who contributed that way in the past and prefer to continue that. But eventually I want to remove the instructions for that path and push people toward the online contribution system - that is where all the development and improvement is currently going. There have already been enough schema/model changes that are not in the latest ImportBuddy release that I may end up having to remove that option.

As far as producing the disc hash locally - this is certainly possible - but I have optimized for making the contribution process as simple as possible. People don't really like downloading and running random scripts and programs locally - which it seems you agree with. How would you feel safe calculating the hash locally?

On the subject of the decoupled process - I use it this way and I think the online contribution system works great for that. You can start a contribution, import a disc, and then come back later to identify the items or add more discs. I usually have several in progress at a time. I don't currently use multiple drives at a time, but you certainly could even have multiple drives uploading logs to for different discs (or even different releases) at the same time. I would love to hear more your process and what parts don't parallelize with the online system.

I probably have more thoughts - but I would love any contributions in this area. Thanks!

[pwl4909]

@lfoust Thank you for the quick reply!

I think the web is a smart direction for simplifying usability. I even wrote a basic graphql script I need to push for searching and creating a jellyfin strings. its a proof of concept and needs better error handling. So thank you for putting time into the graphql.

I specifically called out ImportBuddy, as the only way I knew of for local. What I personally dont like is running an exe. I would find a simple script using common/public libraries easier to review and verify, and therefore more trust worthy. When scripts get layered and complicated its harder to review. So a simple hash the disc content script would be acceptable.

Also, to build confidence with the community, I think spelling out the steps is helpful. Is it as simple as pass disc contents through MD5 function? Is there a specific order than matters? or specific folder on the disc? If I wanted to, knowing your high level steps should be able to create my own python script to calculate the disk hash. With sufficient test data, or taking an existing DB entry I should be able to verify the my MD5 has script with the same release disc. This is likely not necessary in practice to create my own script. However the point I am making is that with sufficient documentation of the required steps I could do so.

In hindsight, I now see how the web contribution process can be decoupled. However I only see that because I went through the process. I believe a common mistake for new users is to start using makemkv, and since that process takes a long time, will process a good portion of their library without knowing better. Then struggle to rename content appropriately. This is what I did. So by spelling out the steps to perform manually will allow for the bottleneck steps like hash/download to be done independently of contribution. My mistake lead me to have downloaded content that I did not capture the correct logs for, or have the SN associated with the files. So I need to get the disc again to properly contribute, likely manually associate each file with size/duration. A mistake I would like others to avoid by adding to the documentation.

So the only changes this boils down to:

1. Add Web Text Entry for disc hash string
2. Document specific MD5 steps for replication (Think what is needed to implement an independent implementation)
3. Create script to calculate MD5 locally (I am partial to python, but any widely used public library/language will work)
4. Document setup/usage steps for script
5. Document Web Contribution workflow/s
6. Any specific thoughts on Jellyfin or Plex or other folder/filename string support?

Do you have a preferred location for these documentation items?

[lfoust]

@pwl4909 - I get your point about the hash and wanting to understand the logic behind it before running it on your own machine. The difficulty is figuring out where the line is before you start alienating non-technical users. I know the whole MakeMkv community tends towards the technical side - but not everyone can or wants to understand scripts and programming just to contribute. For technical people, they can find the hash algorithm on gihub - but it wouldn't be too hard to spell out in documentation for others to implement in different languages. It would probably be smart to have several implementations already written out.

For where to put these documentation items, I think it would be great to spell it out on the wiki (https://github.com/TheDiscDb/data/wiki) and then also have a version of it on the website. But I am definitely open to your thoughts here. Ultimately, I think the easiest way for people to find them would be a help or instructions link on the website.

For Plex/Jellyfin support, I have some notes somewhere on features I want to add here but I don't have anything too solid. Basically, since file naming is so subjective (I know there are standards and docs, but ultimately everyone does their own version of organization), my plan was to integrate file naming into the site in a way that users can customize how they name and organize files based on metadata. That is why the site currently doesn't really expose any file naming info. But having users specify some sort of pattern for how they name things, and having the site respect that is my plan. Any thoughts on that?

[pwl4909]

@lfoust your comments on documentation have teased out my specific viewpoint. For the MD5, the file you linked is the specific file I tracked down to determine the hash was MD5. Which I am familiar with CRC, which heavily relies on configuration, I am not familiar enough to know if MD5 has a specific configuration. My assumption is, MD5 is just an MD5, and its all based on the data you are passing to it. In either case this more technical detail would ideally be documented in a readme in the repo. This is where technical people expect technical information. To your point, I think instructions on what is important to port to a new script if desirable is the information to document. Specific details on how MD5 works is not of interest. However specific configuration needed, or input assumptions to match your hash output would be helpful in a readme.

I suspect a MVP python script of the MD5 has wouldnt take me long to pull together with confirmation of the basic information. Testing would be done by confirming on a disc that is already uploaded.

I would prefer to have a single source of the documentation and link to that source. The wiki is a place that is likely more acceptable for more technical data. while the website is best to show the simplest case for the user, and then link to the wiki for more options. Again, I am somewhat advocating for a workflow I am more comfortable with regarding access. So I think the value of having the existing process, and the changes I am looking for is a wider net.