Decision-Driven Regularization: A Blended Model for Learning and Optimization

This archive is distributed in association with the INFORMS Journal on Computing under the MIT License.

The code and data in this repository are used in the research reported in the paper Decision-Driven Regularization: A Blended Model for Learning and Optimization by Gar Goei Loke, Qinshen Tang, Yangge Xiao, and Xun Zhang.

Cite

To cite the contents of this repository, please cite both the paper and this repo, using their respective DOIs.

https://doi.org/10.1287/ijoc.2024.0930

https://doi.org/10.1287/ijoc.2024.0930.cd

Below is the BibTeX for citing this snapshot of the repository.

@misc{CacheTest,
  author =    {Gar Goei Loke, Qinshen Tang, Yangge Xiao, and Xun Zhang},
  publisher = {INFORMS Journal on Computing},
  title =     {{Decision-Driven Regularization: A Blended Model for Learning and Optimization}},
  year =      {2026},
  doi =       {10.1287/ijoc.2024.0930.cd},
  url =       {https://github.com/INFORMSJoC/2024.0930},
  note =      {Available for download at https://github.com/INFORMSJoC/2024.0930},
}  

Description

The goal of this software is to solve contextual optimization problems using decision-driven regularization. Given observed features, the decision-maker seeks optimal decisions that minimize a context-dependent cost function. This setting arises in many business applications, including on-demand delivery, retail operations, portfolio optimization, and inventory management.

This paper studies contextual optimization from the perspective of integrated learning and optimization. We show that directly optimizing decisions without sufficient control of prediction accuracy may lead to overfitting and poor decision performance compared with simple separate learning-and-optimization approaches. To address this issue, we propose a bi-objective framework, termed decision-driven regularization, that balances prediction accuracy and cost minimization. The framework also handles ambiguity in the cost function through a surrogate objective controlled by a new hyperparameter. We further show that robust optimization and regret minimization provide alternative but closely related formulations, and that our model generalizes existing approaches such as SPO+. Numerical experiments demonstrate that the proposed method outperforms benchmarks including OLS, Random Forest, XGBoost, SPO+, Perturbation Gradient, and Learning and Rank.

Requirements

The code is implemented in Python 3.9.6. The required packages are listed in requirements.txt.

Code Structure

2024.0930/
├── README.md                       # This file
├── AUTHORS                         # Author contact information
├── LICENSE                         # MIT License
├── DDR_Reproduce/                  # Main code
├── Shortest_Path/                  # Main code
├── Data/                           # Generated data files
│   ├── Figure_B1                   # Data used for Figure B1
│   ├── Shortest_Path               # Data used for all other figures and tables.
│       ├── Baseline_SPO_Data_Generation     
│       ├── Quadratic_Term_SPO_Data_Generation
│       ├── Various_Settings_SPO_Data_Generation

Because some data files in the Baseline_SPO_Data_Generation/ folder exceed GitHub's file size limit, please download these data files via this link. All other data files and results are included in this repository.

Reproducing the Results

Figure 1

• Run DDR_illustration_1.ipynb to generate Figure 1.

Shortest Path

Baseline Setting-No misspecification

• Run Baseline_Setting.ipynb to implement experiments, and then run Baseline_Results.ipynb to create Table 1,2, D2 and Figure 2,3,4,5.

The misspecified setting

• Run Experiment_Model_Quadratic.ipynb to implement experiments and to create Figure 6 and 7.

Robustness of Our Model Across Various Settings

• Please run the following notebooks step by step.

  0. Run Various_setting_Reproduce_SPO.ipynb to generate parameters.
  1. Impact of sample size: Run Various_Setting_Reproduce_Data_Size_SPO.ipynb to create upperleft subfigures in Figure D2 and D3.
  2. Impact of number of feature: Run Various_Setting_Reproduce_Num_Feature_SPO.ipynb to create upperright subfigures in Figure D2 and D3.
  3. Impact of range of error term: Run Various_Setting_Reproduce_Noise_Level_SPO.ipynb to create loweright subfigures in Figure D2 and D3.
  4. Impact of model misspecification: Run Various_Setting_Reproduce_Mis_SPO.ipynb to create lowerleft subfigures in Figure D2 and D3.

Table B.1

• Run Angle-DDR_2025_0731.ipynb to generate Table B1.

Figure B.1

• Run Figure_B1.ipynb to generate Figure B1.