Run Demo 1 using xl2times and GAMS' GDX->SQL converter

[siddharth-krishna]

No description provided.

[siddharth-krishna]

This is the DB file produced, in case it helps testing: (zipped)
DemoS_001-all.db3.zip

[olejandro]

Interesting! Which TIMES version did you use? There were breaking changes (with respect to functioning of the prototype) after v4.6.7. The error occurs because an ALL_REG column is expected in the ALL_REG table, but the name of the column is uni.

[siddharth-krishna]

I was using the same version we use in xl2times, 4.7.8 or commit b488fb0.

However, I just tried now with TIMES version 4.6.6 and 4.6.7 and both give me the same error. Would it be easier to manually change the columns / update the prototype code to expect the new column names?

[olejandro]

Strange, let me try...

[olejandro]

@siddharth-krishna #13 fixes this issue.

[olejandro]

Looks like #14 is what causing the next issue...

[siddharth-krishna]

@olejandro I tried running it again, and now we get a new error, any idea what could be the issue?

ERROR: LoadError: SQLite.SQLiteException("no such column: ALL_REG_0")
Stacktrace:
  [1] sqliteerror(args::SQLite.DB)
    @ SQLite [~/.julia/packages/SQLite/UqCGE/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/SQLite/UqCGE/src/)SQLite.jl:34
  [2] macro expansion
    @ [~/.julia/packages/SQLite/UqCGE/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/SQLite/UqCGE/src/)base.jl:10 [inlined]
  [3] prepare_stmt_wrapper
    @ [~/.julia/packages/SQLite/UqCGE/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/SQLite/UqCGE/src/)SQLite.jl:110 [inlined]
  [4] SQLite.Stmt(db::SQLite.DB, sql::String; register::Bool)
    @ SQLite [~/.julia/packages/SQLite/UqCGE/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/SQLite/UqCGE/src/)SQLite.jl:147
  [5] Stmt
    @ [~/.julia/packages/SQLite/UqCGE/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/SQLite/UqCGE/src/)SQLite.jl:146 [inlined]
  [6] prepare
    @ [~/.julia/packages/SQLite/UqCGE/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/SQLite/UqCGE/src/)SQLite.jl:181 [inlined]
  [7] execute
    @ [~/.julia/packages/DBInterface/nQcsk/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/DBInterface/nQcsk/src/)DBInterface.jl:130 [inlined]
  [8] execute
    @ [~/.julia/packages/DBInterface/nQcsk/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/~/.julia/packages/DBInterface/nQcsk/src/)DBInterface.jl:152 [inlined]
  [9] (::TIMES.var"#1#2"{SQLite.DB})(::Pair{String, String})
    @ TIMES [./](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/)none:0
 [10] iterate(g::Base.Generator{Dict{String, String}, TIMES.var"#1#2"{SQLite.DB}}, s::Int64)
    @ Base [./](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/)generator.jl:48
 [11] Dict{String, DataFrames.DataFrame}(kv::Base.Generator{Dict{String, String}, TIMES.var"#1#2"{SQLite.DB}})
    @ Base [./](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/)dict.jl:95
 [12] read_data
    @ [/scratch/htc/skrishna/TIMES.jl/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/src/)load_data.jl:98 [inlined]
 [13] create_model(file_path::String)
    @ TIMES [/scratch/htc/skrishna/TIMES.jl/src/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/src/)TIMES.jl:29
 [14] macro expansion
    @ [./](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/)timing.jl:581 [inlined]
 [15] top-level scope
    @ [/scratch/htc/skrishna/TIMES.jl/](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/)demo.jl:320
in expression starting at [/scratch/htc/skrishna/TIMES.jl/demo.jl:5](https://vscode-remote+ssh-002dremote-002bz1.vscode-resource.vscode-cdn.net/scratch/htc/skrishna/TIMES.jl/demo.jl:5)

[olejandro]

Interesting! @siddharth-krishna I saw that the TIMES source code was updated to the latest version. Could you pin it to v4.6.7 as in the pyomo prototype? Hopefully this will solve the error!

[siddharth-krishna]

Thanks, Olex, I ran it again now with TIMES v4.6.7. I realized that although I thought I was running the code on the Demo 1 .db3 file, there was a line in TIMES.jl that was still trying to read the old PROTO.db3 file:
5741ab5#diff-ad0a98ddbf90e109a686e0269448bd146552966e7108c51be5b4021041202cdfR44

I fixed that, and then did an SQL schema dump of the Demo 1 DB to check if there are other table/column mismatches:
db_schema_full.txt

I tried fixing the COEF tables to look for ALLYEAR instead of YEAR, but we also have the following 3 missing tables:

[MISSING TABLE] RS_PRETS: RS_PRETS
[MISSING TABLE] RTP_IPRI: RTP_IPRI
[MISSING TABLE] RP_PTRAN: RPFF_GGS

Trying to comment these rows from data_info leads to errors later on in TIMES.jl/src/compute_indexes.jl:69, so it looks like these tables are required. Is the issue the TIMES version or perhaps an unsupported feature in xl2times?

[olejandro]

Thanks @siddharth-krishna ! Let me try to reproduce the error using WSL. Maybe there are some differences related to that... The reference to proto.db3 in the precompiling section is correct...

[olejandro]

@siddharth-krishna here is a working db3 file:
DemoS_001-all.zip
Still checking for the reasons behind the differences...

[olejandro]

Just verified that the three tables are present in the db3 file I shared and contain data...