Skip to content

MID-11009: Fix report import to use streaming processing and add max file size limit#480

Open
wadahiro wants to merge 1 commit intoEvolveum:support-4.10from
openstandia:fix-11009
Open

MID-11009: Fix report import to use streaming processing and add max file size limit#480
wadahiro wants to merge 1 commit intoEvolveum:support-4.10from
openstandia:fix-11009

Conversation

@wadahiro
Copy link
Contributor

https://support.evolveum.com/notifications/details/11009/activity

This PR fixes a memory issue when importing large CSV files and adds a DoS protection mechanism.

Changes

Streaming CSV Processing

Changed CSV import from loading all records into memory (List) to streaming processing. This prevents OutOfMemoryError when importing large CSV files.

  • ImportController: Added streaming methods (initializeCsvParser(), getNextVariablesMap(), close())
  • ClassicReportImportActivityRun: Updated to use streaming iteration instead of list-based processing

Max File Size Limit for Report Import

Added configurable maximum file size for report imports to prevent DoS attacks via extremely large file uploads.

Configuration note:

MidPoint already has a default 100MB upload limit at the servlet level
(spring.servlet.multipart.max-file-size in application.yml).

The new maxReportImportFileSize setting allows administrators to apply a more restrictive limit specifically for report imports if needed.

  • Default value: -1 (unlimited at the application level, falls back to the 100MB servlet limit)
  • Configuration: Set via JVM system property -Dmidpoint.webApplication.maxReportImportFileSize=<bytes>
  • Example: -Dmidpoint.webApplication.maxReportImportFileSize=5242880 limits report imports to 5MB

Image

image

…file size limit

- Change CSV parsing from loading all records into memory to streaming processing
  to prevent OutOfMemoryError with large import files
- Add max file size configuration for report import to prevent DoS attacks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments