MID-11009: Fix report import to use streaming processing and add max file size limit#480
Open
wadahiro wants to merge 1 commit intoEvolveum:support-4.10from
Open
MID-11009: Fix report import to use streaming processing and add max file size limit#480wadahiro wants to merge 1 commit intoEvolveum:support-4.10from
wadahiro wants to merge 1 commit intoEvolveum:support-4.10from
Conversation
…file size limit - Change CSV parsing from loading all records into memory to streaming processing to prevent OutOfMemoryError with large import files - Add max file size configuration for report import to prevent DoS attacks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://support.evolveum.com/notifications/details/11009/activity
This PR fixes a memory issue when importing large CSV files and adds a DoS protection mechanism.
Changes
Streaming CSV Processing
Changed CSV import from loading all records into memory (List) to streaming processing. This prevents OutOfMemoryError when importing large CSV files.
Max File Size Limit for Report Import
Added configurable maximum file size for report imports to prevent DoS attacks via extremely large file uploads.
Configuration note:
MidPoint already has a default 100MB upload limit at the servlet level
(spring.servlet.multipart.max-file-size in application.yml).
The new
maxReportImportFileSizesetting allows administrators to apply a more restrictive limit specifically for report imports if needed.-Dmidpoint.webApplication.maxReportImportFileSize=<bytes>-Dmidpoint.webApplication.maxReportImportFileSize=5242880limits report imports to 5MBImage