gridpath_run_data_toolkit no longer works with open_data_toolkit_settings_sample.csv

[HaleyRoss]

The gridpath_run_data_toolkit command calls run_data_toolkit.py which now requires "script_true_false_arg" and "reverse_default_behavior" columns in the settings.csv, however open_data_toolkit_settings_example.csv only contains "script", "setting", and "value" columns.

Calling the gridpath_run_data_toolkit now results in the following error:

gridpath_run_data_toolkit --single_step create_database --database /Users/hross2/Documents/GitHub/gridpath_data/raw_data 
--db_schema ./raw_data_db_schema.sql --omit_data --settings_csv /Users/hross2/Documents/GitHub/gridpath/data_toolkit/
open_data_toolkit_settings_sample.csv
Traceback (most recent call last):
  File "/Users/hross2/anaconda3/envs/grid_path/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3805, in get_loc
    return self._engine.get_loc(casted_key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc
  File "index.pyx", line 196, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7081, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7089, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'script_true_false_arg'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/hross2/anaconda3/envs/grid_path/bin/gridpath_run_data_toolkit", line 33, in <module>
    sys.exit(load_entry_point('GridPath', 'console_scripts', 'gridpath_run_data_toolkit')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hross2/Documents/GitHub/gridpath/data_toolkit/run_data_toolkit.py", line 180, in main
    row["script_true_false_arg"],
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hross2/anaconda3/envs/grid_path/lib/python3.11/site-packages/pandas/core/series.py", line 1121, in __getitem__
    return self._get_value(key)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/hross2/anaconda3/envs/grid_path/lib/python3.11/site-packages/pandas/core/series.py", line 1237, in _get_value
    loc = self.index.get_loc(label)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hross2/anaconda3/envs/grid_path/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3812, in get_loc
    raise KeyError(key) from err
KeyError: 'script_true_false_arg'

[anamileva]

@HaleyRoss, thanks for bringing this up. The 'open_data_toolkit_settings_example.csv' file should have been removed. Please see here for example settings with the correct format: https://github.com/blue-marble/gridpath/blob/develop/tests/test_data/data_toolkit_open_data_settings.csv.

[HaleyRoss]

That ran, but in addition to removing the file I would also recommend updating the default settings.csv location on line 105 of run_data_toolkit.py. Also, in the docs in section 6.2.1 it's not specified that a settings.csv file would even be needed. I just tried it based on the error that was appearing when I tried that line of code.

[HaleyRoss]

Also, is there anywhere that describes options/effects of changes to the settings.csv?