Skip to content

refactor(backend): Implement unified Message collection to fix 16MB limit (#25)#123

Open
abhinavkdeval08-design wants to merge 3 commits into
0rigin-c0de:mainfrom
abhinavkdeval08-design:refactor-chat-schema
Open

refactor(backend): Implement unified Message collection to fix 16MB limit (#25)#123
abhinavkdeval08-design wants to merge 3 commits into
0rigin-c0de:mainfrom
abhinavkdeval08-design:refactor-chat-schema

Conversation

@abhinavkdeval08-design
Copy link
Copy Markdown

Description

This PR addresses issue #25 by refactoring the chat storage schema.
The previous implementation stored messages as embedded arrays within thread/chat documents, which could lead to MongoDB's 16MB document size limit bottleneck over time...

Changes Made:

  • Created a new, unified Message.js model to store all DMs and Server messages independently.
  • Updated controllers to query this new collection.
  • Introduced Cursor Pagination limit to fetch 50 messages at a time instead of loading the entire array.

Fixes #25


Hi @0rigin-c0de, I have implemented the schema refactor as discussed. Please review the code whenever you have time, and kindly add the gssoc:approved and level:advanced type:feature type:performance labels to this PR so the GSSoC tracker can catch it. Thank you!

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 20, 2026

@abhinavkdeval08-design is attempting to deploy a commit to the Sunil Kumar's projects Team on Vercel.

A member of the Team first needs to authorize it.

…o refactor-chat-schema

# Conflicts:
#	server/src/models/Message.js
#	server/src/routes/chat.js
@abhinavkdeval08-design
Copy link
Copy Markdown
Author

Hi @0rigin-c0de,

I’ve submitted the PR for the schema refactor as discussed. This implementation moves messages out of embedded arrays into a unified collection, which permanently resolves the MongoDB 16MB document size bottleneck (#25).

Key highlights of this PR:

Scalability: Messages are now stored independently, preventing unbounded array growth.

Performance: Implemented cursor-based pagination (50 messages per fetch), which significantly reduces the initial payload size and improves load times.

Cleanup: The code has been refactored for better query maintainability.

Since this is an Advanced level task under GSSoC'26, could you please review the changes when you have a moment? Also, if the implementation aligns with your expectations, kindly add the relevant labels (gssoc:approved, level:advanced, type:feature, type:performance) so it can be tracked correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor chat storage schema to avoid unbounded nested message arrays

1 participant