Skip to content

feat: add support to realtime.messages binary payloads#1869

Open
edgurgel wants to merge 5 commits into
mainfrom
feat/binary-payload-realtime-messages
Open

feat: add support to realtime.messages binary payloads#1869
edgurgel wants to merge 5 commits into
mainfrom
feat/binary-payload-realtime-messages

Conversation

@edgurgel
Copy link
Copy Markdown
Member

What kind of change does this PR introduce?

  • Add binary_payload column to realtime.messages with a constraint that only apply to new rows that checks that we can't have binary_payload AND payload NOT NULL at the same time. They can both be NULL though because we already allowed payload to be NULL so I didn't want to break anythiung. We could be more strict and ensure that one of them exclusively must be NULL.
  • Add realtime.send accepting bytea overloading the existing function.

@blacksmith-sh

This comment has been minimized.

@coveralls
Copy link
Copy Markdown

coveralls commented May 15, 2026

Coverage Status

Coverage is 91.265%feat/binary-payload-realtime-messages into main. No base build found for main.

use Ecto.Migration

def change do
execute("ALTER TABLE realtime.messages ADD COLUMN IF NOT EXISTS binary_payload bytea")
Copy link
Copy Markdown
Contributor

@leandrocp leandrocp May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we handle big binaries? I can simulate a OOM crash with this payload:

payload = :binary.copy(<<0>>, 900 * 1024 * 1024)

Postgrex.query!(
  db,
  "SELECT realtime.send($1::bytea, 'oom', 'adios', true)",
  [payload]
)

Running in a container with ~1.2GB free mem

Copy link
Copy Markdown
Member Author

@edgurgel edgurgel May 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The process would reach the max heap size and crash before OOMing (The container must have a bit more memory than the max heap size we set)

+hmax <%= div(1_000_000_000, :erlang.system_info(:wordsize)) %>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep I was just concerned about exposing a possible scenario where one could try to send a really large bin for whatever reason.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah makes sense! We do have this protection to not let any single process kill the system. The assumption is that we have enough free memory to not hit the limit quickly 👍

@edgurgel edgurgel force-pushed the feat/binary-payload-realtime-messages branch from 1c999c5 to 2d0995c Compare May 19, 2026 00:57
@edgurgel edgurgel force-pushed the feat/binary-payload-realtime-messages branch from 2d0995c to 7f6b856 Compare May 19, 2026 03:24
@edgurgel edgurgel requested a review from leandrocp May 19, 2026 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants