Skip to content

Set field_size_limit to max long int size on csv plugin #352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
humrochagf wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Set field_size_limit to max long int size on csv plugin #352

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions rows/plugins/plugin_csv.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

from io import BytesIO

import ctypes
import six
import unicodecsv

Expand All @@ -32,9 +33,8 @@
sniffer = unicodecsv.Sniffer()
# Some CSV files have more than 128kB of data in a cell, so we force this value
# to be greater (16MB).
# TODO: check if it impacts in memory usage.
# TODO: may add option to change it by passing a parameter to import/export.
unicodecsv.field_size_limit(16777216)
unicodecsv.field_size_limit(int(ctypes.c_ulong(-1).value) // 2)


def fix_dialect(dialect):
Expand Down