Skip to content

fix: restore advisor's quarto book and properly merge feature-cm#96

Merged
xuyiqing merged 38 commits into
masterfrom
dev
Mar 26, 2026
Merged

fix: restore advisor's quarto book and properly merge feature-cm#96
xuyiqing merged 38 commits into
masterfrom
dev

Conversation

@TianzhuQin

Copy link
Copy Markdown
Collaborator

Summary

  • Restored the advisor's quarto book chapter structure (03-ife-mc, 04-cfe, 06-plots, 07-gsynth, 08-panel, 09-sens, bb-updates) that was incorrectly overwritten during the feature-cm merge
  • Properly merged feature-cm's causal moderation functionality (cm parameter in fect/plot, fect_iden(), loess.fit, pretreatment diagnostics) into the advisor's R code via 3-way merge
  • Rewrote 05-hte.Rmd to cover both basic HTE visualization and the new causal moderation framework
  • Restored all support files: rscript/, .Rprofile, vignettes.Rproj, _quarto.yml

What was wrong

The previous merge of feature-cm into dev (via claude/merge-cleanup-dev-o2BQ6) incorrectly resolved conflicts by reverting the advisor's quarto book restructuring from the cfe branch, and failed to bring over feature-cm's R code changes to default.R, plot.R, fe.R, and boot.R.

Test plan

  • Verify quarto book renders with correct chapter ordering
  • Verify fect(..., cm = TRUE) runs without error
  • Verify plot(..., type = "hte", cm = TRUE) produces correct output
  • Verify fect_iden() over-identification test works
  • R CMD check passes

claude and others added 30 commits March 23, 2026 20:02
…re-cm

The previous merge of feature-cm into dev incorrectly resolved conflicts,
overwriting the advisor's quarto book restructuring from the cfe branch.

Restored:
- Chapter structure: 03-ife-mc, 04-cfe, 05-hte, 06-plots, 07-gsynth,
  08-panel, 09-sens, bb-updates (was incorrectly renumbered/deleted)
- All rscript/ companion files
- Support files: .Rprofile, vignettes.Rproj, .gitignore
- _quarto.yml with correct chapter ordering
- All other vignettes to advisor's versions

Properly merged feature-cm additions:
- R/default.R: cm parameter for causal moderation
- R/fe.R: dual imputation (g0/g1) when cm=TRUE
- R/plot.R: cm plotting, loess.fit, pretreatment, covariate.value params
- R/boot.R, R/cfe.R: cm parameter pass-through
- R/fect_nevertreated.R: cm/II.cm parameter signature
- R/fect_iden.R, R/polynomial.R: already present (kept)
- DESCRIPTION: added splines dependency
- 05-hte.Rmd: rewritten with working feature-cm examples

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
fect.RData only contains legacy datasets (simdata, simgsynth, hh2019,
gs2020, turnout). The new datasets (sim_base, sim_gsynth, sim_linear,
sim_trend, sim_region) exist as separate .rda files but were not loaded
by data(fect). After rm(list = ls()), these datasets became unavailable
causing "object not found" errors during quarto render.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…W7xO

fix: add explicit data() calls for new datasets in vignettes
- Add cm parameter to fect.Rd usage and arguments sections
- Add missing plot.fect parameters: loess.fit, covariate.value,
  covariate.value.range, relative.time, pretreatment, num.pretreatment, cm
- Add globalVariables declaration for fit, group, y_hat, y_hat_lower,
  y_hat_upper to suppress R CMD check NOTEs

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…W7xO

fix: sync documentation with code for cm parameter and plot.fect
data(fect) / data(sim_base) only works when the installed package
contains the data files. For local quarto render from the source tree
(where the installed package may be outdated), load .rda/.RData files
directly from ../data/. Falls back to data() for R CMD check where
the package is freshly installed.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…W7xO

fix: load data from source tree for local quarto render
Local quarto render uses the installed fect package, which may be
outdated and lack new features (e.g., cm parameter). Added _common.R
that calls devtools::load_all() when rendering from the source tree,
ensuring all R functions and lazy-loaded datasets come from the latest
source code. Falls back to library(fect) for R CMD check.

Also reverted data loading to clean data() calls (load_all handles
everything) and fixed fect.Rd line width exceeding 90 chars.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

fix: use devtools::load_all() for local quarto rendering
…a() calls

fect.RData was redundant (all objects exist as individual .rda files) and
caused CI WARNING about duplicate data objects. Replace data(fect) with
specific dataset loads (e.g., data(simdata), data(hh2019)) across all
vignettes and rscript files. Also fix fect_boot crash when keep.sims=TRUE
by adding missing eff/D/I/boot.id fields to the boot0 early-return list.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

fix: remove fect.RData and replace all data(fect) with individual dat…
The previous fix only patched the degenerate D/I early-return boot0.
The try-error fallback boot0 in one.nonpara (and parametric bootstrap
functions) was also missing these fields, causing dimension mismatch
when keep.sims=TRUE and any bootstrap iteration fails with try-error.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

fix: add eff/D/I/boot.id to ALL boot0 fallback lists in fect_boot
…tions

When boot.rm removes failed iterations, the 2D matrices (att.boot etc.)
were trimmed but the 3D arrays (eff.boot, D.boot, I.boot) used by
keep.sims were not. This caused subscript out of bounds in effect()
which uses length(att.avg.boot) as nboots but indexes into the
untrimmed 3D arrays.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

fix: trim eff.boot/D.boot/I.boot when removing failed bootstrap itera…
length(att.avg.boot) may differ from the number of bootstrap slices
in the 3D eff.boot/D.boot/I.boot arrays after failed-boot removal.
Use the actual array dimension to prevent subscript out of bounds.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…ls::load_all()

_common.R already loads fect via devtools::load_all() from source tree.
Calling library(fect) afterwards could re-load the installed (outdated)
package version, causing boot.R fixes to not take effect. Also fix
effect() to use dim(eff.boot)[3] for nboots to match actual array size.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

Claude/resolve merge conflicts x w7x o
…smatched

Add defensive checks for D.boot and I.boot 3D arrays in effect().
If they are missing or have fewer slices than eff.boot, fall back to
using the original D.dat and I.dat matrices. This prevents the
"subscript out of bounds" error regardless of which fect_boot version
produced the object.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
If devtools::load_all() fails, fall back to library(fect) with a
diagnostic message instead of silently crashing the vignette render.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

Claude/resolve merge conflicts x w7x o
The if(keep.sims) conditional in boot0 lists inside one.nonpara added
keep.sims as a free variable that may not survive trim_closure_env
serialization for parallel workers. Always create NA matrices instead
(harmless when keep.sims=FALSE since they're not stored).

Also use parallel=FALSE and nboots=200 for the keep.sims=TRUE call
in 06-plots.Rmd to avoid parallel serialization issues entirely.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

fix: remove keep.sims conditional from boot0 inside one.nonpara
The out_no_reversals fect() call also used parallel=TRUE with
keep.sims=TRUE, hitting the same parallel serialization issue.
Switch to parallel=FALSE, nboots=200 to match the out.hh fix.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

fix: use parallel=FALSE for second keep.sims=TRUE call in 06-plots.Rmd
When rendering locally with devtools::load_all(), parallel workers
load the installed (old) fect package via .packages=c("fect",...),
causing version mismatches between source R functions and installed
internal helpers. This makes all parallel bootstrap/jackknife
iterations fail. Use parallel=FALSE for all vignette rendering.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

fix: set parallel=FALSE in all vignette fect() calls
polars is not available on CRAN and may not be installed. Wrap the
library() calls in a requireNamespace check and gate the two didm
chunks with eval=has_polars so the vignette renders even without polars.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

Make polars/DIDmultiplegtDYN conditional in 08-panel.Rmd
The MultisessionFuture was getting interrupted during vignette rendering
due to resource constraints. Set parallel=FALSE and nboots=50 to avoid
the issue while still demonstrating the bootstrap SE functionality.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
TianzhuQin and others added 8 commits March 24, 2026 10:03
…xW7xO

Fix wrapper_boot chunk: disable parallel and reduce nboots
The bootstrap covariance computation (cov(t(att.boot))) can fail and
return NA, causing fect_sens to error with "incorrect number of
dimensions" when indexing att.vcov as a matrix.

- Switch vignette fect() call to vartype='jackknife' which reliably
  produces a vcov matrix via the jackknifed() function
- Add validation in fect_sens.R to give an informative error when
  att.vcov is not a valid matrix

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
The bootstrap path in boot.R can produce att.vcov = NA (scalar) when
cov() fails in tryCatch, causing fect_sens to error with "incorrect
number of dimensions".

- In the vignette: compute att.vcov from att.boot if it's not a matrix,
  before calling fect_sens. Also set cache=FALSE to avoid stale cache.
- In fect_sens.R: add fallback to compute vcov from att.boot when
  att.vcov is not a valid matrix.

https://claude.ai/code/session_014ctjR27HYMWhCzsP5jesLW
…xW7xO

Claude/resolve merge conflicts x w7x o
1. Fix duplicate #sec-hte anchor in 06-plots.Rmd that caused "Chapter
   Effect Heterogeneity" instead of "Chapter 5" in cross-references.
   Renamed to #sec-plots-hte so @sec-hte uniquely resolves to 05-hte.Rmd.

2. Add warning = FALSE to all fect() chunks in 04-cfe.Rmd to suppress
   CFE convergence warnings in rendered output.

3. Add plot.margin to HTE, calendar, and box plot themes in plot.R to
   give more space above the title in rendered figures.

https://claude.ai/code/session_017KSYJi57fu5CmzGwTjw4Tj
…a5ul

Fix three Quarto book rendering issues
- Fix dif3_tol reset bug in CFE optimizer (src/cfe_sub.cpp): missing
  reset caused non-convergence when Q.type="linear" with zero trends
- Add early-exit on overall fit convergence (3 consecutive iterations)
- Remove redundant ggplot2 size=0.3 from geom_rect in plot.R
- Suppress HonestDiDFEct internal warnings in fect_sens.R
- Fix data(fect,...) -> data(simdata,...) in test-plot-refactor.R
- Delete test-gsynth-parametric-group-att-align.R (tested anti-pattern)
- Add skip_on_cran() to 4 test files (keep only basic + plot for CRAN)
- Update vignettes: parallel=TRUE cores=16, nboots corrections
- Re-purl all rscript files from Rmd

Tests: 0 FAIL, 0 WARN, 586 PASS
Quarto: 13/13 chapters rendered

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@xuyiqing xuyiqing merged commit 104aec8 into master Mar 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants