Audio Backend by bob12224 · Pull Request #6824 · stashapp/stash

bob12224 · 2026-04-13T03:21:16Z

This is a copy-paste setup for the backend to support AUDIO, similar to how Scene's and Images are handled.

Please see the docs/dev/AUDIO.md for the SCOPE and as a place for general discussion.

There is no new design, it copies form SCENES and then removes what is unrelated
Exception: FrameRate has been changed to SampleRate (FFPROBE already grabs this)

Relates to: #1258

TODOs that require Reviewer input

└─ stash
   ├─ audio.graphql
   │  ├─ line 11: TODO (audio|AudioCaption): need to update IF AudioCaption required
   │  └─ line 29: TODO (audio|AudioCaption): need to update IF AudioCaption required
   ├─ resolver_model_audio.go
   │  └─ line 119: TODO (audio|AudioCaption): need to update IF AudioCaption required
   ├─ audio.go
   │  └─ line 84: TODO (audio): can we return no urls?
   ├─ config.go
   │  └─ line 857: TODO (audio): update this to AudioFileNamingAlgorithm?
   ├─ fingerprint.go
   │  └─ line 67: TODO (audio): should Audio's also use OSHash instead of md5 for default (if so, then will need to update Audios)
   ├─ manager_tasks.go
   │  └─ line 34: TODO (audio): figure out this IF condition
   ├─ repository.go
   │  └─ line 30: TODO (audio): is this only used for stashbox?
   ├─ delete.go
   │  └─ line 42: TODO (future|audio generated files): add paths here
   ├─ migrate_hash.go
   │  └─ line 1: TODO (audio): should this file be deleted since there are no transcodes?
   ├─ scan.go
   │  └─ line 46: TODO (audio): this PR has no generation
   ├─ browser.go
   │  └─ line 91: TODO (audio): do we need to check ProbeAudioCodec for audio containers? (i.e. can `.mp3` contain a codec we need to transcode for?
   ├─ container.go
   │  └─ line 17: TODO (audio): better way to do this, without suffix this clashes with `Mp3 ProbeAudioCodec`
   ├─ caption.go
   │  └─ line 1: TODO (audio): Can this file be deleted if we utilize VideoCaptions?
   ├─ caption_test.go
   │  └─ line 1: TODO (audio): Can this file be deleted if we utilize audioCaptions?
   ├─ model_audio.go
   │  └─ line 235: TODO (audio): don't know if we need this, using VideoCaption for now due to `pkg/models/repository_file.go` and `FileReader` using
   └─ file.go
      └─ line 27: TODO (audio|AudioCaption): need to update IF AudioCaption required

u

…ket. Will utilize for discussion and agreement on MVP

Gykes

I want to preface this by saying I'm not the main dev and, quite frankly, have limited experience with design implementation so take what I say with a grain of salt. If something sounds dumb or wrong it probably is and feel free to call me on it.

Gykes · 2026-04-14T04:57:30Z

+    - duration
+    - audio codec
+    - OPTIONAL (can be added now or later)
+        - channels (mono, stereo, 5.1, 7.1)


Would tthis really be needed? Currently scenes have audio and we don't track this at all.

I would suggest looking at bitrate again, as this is tracked for scenes (the video portion).
And it might be better to add now, instead of planning to add later.

Mostly I was referring to the channels. I think bitrate would be fine.

Gykes · 2026-04-14T04:58:48Z

+    `created_at` datetime not null,
+    `updated_at` datetime not null,
+    `code` text,
+    `artists` text,


If we decide to go with performers this would need to be changed.

Gykes · 2026-04-14T05:00:40Z

+    `updated_at` datetime not null,
+    `code` text,
+    `artists` text,
+    `album` text,


Studio was in the Audio Metadata but album is used here. How and where would we store albums? In the case of audio, studios seem like the best fit.

Gykes · 2026-04-14T05:01:08Z

+    `title` varchar(255),
+    `details` text,
+    `date` date,
+    `rating` tinyint,


Probs better to use rating100 similar to scenes.

the migrations for scenes uses rating (ref: pkg/sqlite/migrations/1_initial.up.sql). I can only find rating100 in the graphql, which I do provide in graphql/schema/types/audio.graphql (line 51)

Gykes · 2026-04-14T05:01:48Z

+    foreign key(`audio_id`) references `audios`(`id`) on delete cascade,
+    PRIMARY KEY("group_id", `audio_id`)
+);
+CREATE INDEX `index_movies_audios_on_movie_id` on "groups_audios" ("group_id");


renamed to index_group_audios_on_group_id

rezreal · 2026-04-16T22:16:54Z

tbh there is no reason why audios should not just be scenes and why there would be a dedicated section for them.

Lots of errors to fix and TODO notes

- Removed funscripts, they are for interactive - updated the scanner to correctly create `audio_files` row - Adding Audio to `paths` - Updated sqlite to add AudioFile Need to update mutations next

New instructions since newest go version does not work well with v1 of linters

- removed todos - tested mutations - updated the autotagger - added missing sort options - open question generated from this

bob12224 · 2026-04-29T02:29:52Z

@Gykes I can see nothing else required for this PR. In your court. Please see my TODO notes that require your input

Gykes · 2026-04-29T03:16:33Z

Will give it a gander tomorrow probably. I don't have the brain power right now to look over a 13k addition lol

bob12224 · 2026-05-03T19:41:29Z

Will give it a gander tomorrow probably. I don't have the brain power right now to look over a 13k addition lol

Any luck, haven't seen any PR comments yet

Gykes · 2026-05-03T19:54:54Z

Will give it a gander tomorrow probably. I don't have the brain power right now to look over a 13k addition lol

Any luck, haven't seen any PR comments yet

I have various thoughts lined up. Been thinking on stuff the last few days passively. I will start putting my thoughts in now so you can think on it as well.

Gykes

These are just initial thoughts that I remembered off the top of my head. Adding testing would probably be a good idea. Review process will be slow on this. Sorry but prepping to move cross country and time is becoming more and more limited.

I went over some of your TODO questions in the review. Also I think, if we are doing transcoding in a separate PR, then we should probably rip out the commented trans coding support sections. Currently all it is doing is increasing LOC count.

Gykes · 2026-05-03T19:56:44Z

+--
+
+-- TODO(audio): think of better name for this, too close to `audios_files`
+CREATE TABLE `audio_files` (


audio_file_info? Not married to this was just the first thing that popped into my head. Happy to hear recs.

files_audio (would need to migrate video_files -> files_video to match naming convention)

audio_only_files

audiofiles (not a fan)

Gykes · 2026-05-03T19:59:02Z

+	if instance.Config.IsCreateImageClipsFromVideos() && stash != nil && stash.ExcludeVideo {
+		// TODO(audio): figure out this IF condition
+		return isImage(pathname) || isVideo(pathname)
+	}


Does this even need to exist?

I am confused by this as well and would value input here

Gykes · 2026-05-03T20:03:37Z

+var (
+	directAudioEndpointType = endpointType{
+		label:     "Direct stream",
+		mimeType:  ffmpeg.MimeMp3Audio,


If we hardcode this for all direct streams wont that also serve all other formats? Flac,m4a, etc. This could break audio playback. We could resolve this at the container instead

This is copied from internal/manager/scene.go, I don't see any benefit making audio different from scenes unless it requires it to meet feature-set.

Copied to make it easier to compare:

directEndpointType = endpointType{ label: "Direct stream", mimeType: ffmpeg.MimeMp4Video, extension: "", }

Gykes · 2026-05-03T20:04:51Z

+		endpoints = append(endpoints, makeStreamEndpoint(directAudioEndpointType))
+	}
+
+	// TODO(audio): can we return no urls?


Seems like the scheme declares a non null array. You would need to return it as an empty slice. Something like []*AudioStreamEndpoint{}

Gykes · 2026-05-03T20:06:07Z

+  title: String
+  code: String
+  details: String
+  url: String @deprecated(reason: "Use urls")


Could probably remove this from the audio side. It was never implemented on here so don't see the need for it to be deprecated but there's not issue with having it, just extra LOC.

EDIT: This could apply to the other deprecated fields as well

Gykes · 2026-05-03T20:11:41Z

@@ -0,0 +1,39 @@
+// TODO(audio): should this file be deleted since there are no transcodes?


If we are doing transcodes at a later date then yea, should probably remove this to keep scope and LOC down.

File should continue existing, it is used to find duplicate files with the same hash and merge them into the same audio

Gykes · 2026-05-03T20:25:32Z

+func (a *AudioFile) getStreamIndex(fileType string, probeJSON FFProbeJSON) int {
+	ret := -1
+	for i, stream := range probeJSON.Streams {
+		// skip cover art/thumbnails
+		if stream.CodecType == fileType && stream.Disposition.AttachedPic == 0 {
+			// prefer default stream
+			if stream.Disposition.Default == 1 {
+				return i
+			}
+
+			// backwards compatible behaviour - fallback to first matching stream
+			if ret == -1 {
+				ret = i
+			}
+		}
+	}


This is a copy of the video version at roughly line 284. Could break this out to a helper to reduce LOC changes.

I can do this, but I think that might make things more confusing. Since this is a tradition copy-paste+rename PR, making new changes would require more thinking than just accepting that what works for Scenes will work for Audio.

Perhaps we can defer this change to when the feature is more accepted and new audio specific features are added (transcodes/waveform sprites/ui/etc)?

Gykes · 2026-05-08T02:50:07Z

I see you making review comments. I am currently beginning my move and wont be setup back up fully till around the 16th. I still have my laptop and MAY get to them if I'm bored or have time to kill during transit. I haven't forgotten but just ping me after the 15th if you think I have

bob12224 and others added 3 commits March 5, 2026 21:31

Merge pull request #1 from stashapp/develop

1314938

u

Merge branch 'stashapp:develop' into develop

5f26e48

Rough copy-paste setup for backend + TODO list and scope for this tic…

31b69c1

…ket. Will utilize for discussion and agreement on MVP

bob12224 mentioned this pull request Apr 13, 2026

[Feature] Support audio files/stories #1258

Open

bob12224 commented Apr 13, 2026

View reviewed changes

Comment thread docs/dev/AUDIO.md

Gykes reviewed Apr 14, 2026

View reviewed changes

bob12224 added 2 commits April 22, 2026 21:43

Able to Generate

bb76aff

Lots of errors to fix and TODO notes

Simple GraphQL works

23c4134

bob12224 commented Apr 25, 2026

View reviewed changes

Comment thread .golangci.yml

bob12224 added 6 commits April 26, 2026 20:21

Direct Streams working

169bebe

- Removed funscripts, they are for interactive - updated the scanner to correctly create `audio_files` row - Adding Audio to `paths` - Updated sqlite to add AudioFile Need to update mutations next

Removing unused functions

a0e3ab2

Linting

01cf70a

Cleaning up comments

c3922f6

clean up

361e514

Linting

b06200a

New instructions since newest go version does not work well with v1 of linters

bob12224 commented Apr 27, 2026

View reviewed changes

Comment thread docs/DEVELOPMENT.md Outdated

Adding reverse lookups group/performer

bfab3e0

- removed todos - tested mutations - updated the autotagger - added missing sort options - open question generated from this

bob12224 marked this pull request as ready for review April 28, 2026 00:51

bob12224 added 6 commits April 27, 2026 17:52

Merge branch 'stashapp:develop' into audio

cedcdfe

Fixing Audio

e9ff880

Undo to some changes

0068eb3

Merge branch 'stashapp:develop' into develop

5f5eab2

Merge remote-tracking branch 'origin/develop' into audio

1a6d23a

Fixing linter upgrade

e673003

Gykes requested changes May 3, 2026

View reviewed changes

Merge branch 'stashapp:develop' into audio

01fc6f0

PR fixes

9d0648b

		@@ -0,0 +1,39 @@
		// TODO(audio): should this file be deleted since there are no transcodes?

Uh oh!

Conversation

bob12224 commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Gykes left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bob12224 Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rezreal commented Apr 16, 2026

Uh oh!

Uh oh!

Uh oh!

bob12224 commented Apr 29, 2026

Uh oh!

Gykes commented Apr 29, 2026

Uh oh!

bob12224 commented May 3, 2026

Uh oh!

Gykes commented May 3, 2026

Uh oh!

Gykes left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Gykes May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Gykes commented May 8, 2026

Uh oh!

Reviewers

bob12224 commented Apr 13, 2026 •

edited

Loading

bob12224 Apr 16, 2026 •

edited

Loading

Gykes May 3, 2026 •

edited

Loading