Detailed Bug Report and Implementation Observations
While building upon this work for our own research, we identified several potential bugs and implementation discrepancies. We are sharing these findings in the spirit of open science and hope they are useful. We have grouped them by severity:
Critical
Emitter: Padding mask is inverted in self-attention
Location: emitter_model.py:207, root cause in emitter_dataloader.py → normalize_padding_origin_goal()
Description: The padding mask produced by normalize_padding_origin_goal() appears to follow the convention True = agent present, False = padded, whereas PyTorch’s nn.MultiheadAttention expects the opposite convention (True = ignore / padded, False = attend).
If this interpretation is correct, this would result in attention being computed primarily over padded positions, while valid agent positions are masked out.
Suggested fix: Negate the mask before passing it to the attention layer.
Emitter: EmitterPipeline coordinates heuristic sampling
Location: emitter_pipeline.py:206/207
Description: The origin and goal coordinates in the EmitterPipeline do not appear to be generated by the diffusion model. Instead, they are sampled heuristically from the appearance (population) density map. As a result, the diffusion model seems to only influence a subset of variables (e.g., agent type, frame of origin, and preferred speed), while the spatial trajectory endpoints are determined outside of the diffusion process. This appears inconsistent with the expectation that the diffusion model is responsible for generating full spawn conditions. If this interpretation is correct, it suggests that a significant part of the trajectory generation is governed by heuristic sampling rather than the learned model. Also a bit unclear as the leapfrog mechanism is not described in the paper, which makes it difficult to fully understand it.
Simulator: Neighbor Information Leackage
Location: utils/trajectory.py:268ff
Description: The function builds a dictionary mapping each target trajectory window to its neighboring agents' history windows. It does this by extracting a single frame from each window, then merging on the frame column to find temporally co-occurring windows.
For traj (20-frame target windows), it correctly extracts frame index nth (= history_length - 1 = 9), which is the observation endpoint — the last frame the model is allowed to see.
But for hist (10-frame neighbor windows), it uses .first() instead of .nth(nth), extracting frame index 0 — the start of the window:
# Current (buggy):
traj_frame = traj.groupby('meta_id').nth(nth).reset_index() # frame at index 9 (observation endpoint)
hist_frame = hist.groupby('meta_id').first().reset_index() # frame at index 0 (window start) ← WRONGSince the merge is on frame, this matches a target window whose observation endpoint is at frame F with neighbor windows whose first frame is F. The neighbor's 10-frame window therefore covers [F, F+9] — which is 9 frames into the future relative to the target's observation period [F-9, F].
In effect, during training the model sees where neighbors will be in the future, not where they were in the past. At inference time, only true neighbor history is used (built from traj_hist_meter in inference_model.py). This creates a train/inference mismatch: the model learns to rely on future neighbor information that is unavailable at test time.
Potential Fix: Use .nth(nth) for both DataFrames so they are matched on the same temporal anchor (the observation endpoint).
Medium
Simulator: Missing neighbor padding mask
Location: simulator_model.py:127ff
Description: The simulator pads missing neighbor slots with zeros (up to max_neighbors), but never tells the social transformer which slots are padded. Therefore the social_transformer (which wraps nn.TransformerEncoder) attends equally to real neighbors and zero-padded slots.
Potential Fix:Detect padding slots by checking for all-zero rows, then pass a proper padding_mask:
# Build padding mask: True where a neighbor slot is all-zero (absent)
neighbor_is_padding = (neighbor.abs().sum(dim=2) == 0) # (B, N)
ego_valid = torch.zeros(batch_size, 1, dtype=torch.bool, device=neighbor.device)
neighbor_padding_mask = torch.cat([ego_valid, neighbor_is_padding], dim=1) # (B, N+1)
for neighbor_encoder in self.neighbor_encoders:
neighbor_embedding = neighbor_encoder(neighbor_embedding, padding_mask=neighbor_padding_mask)Minor
Emitter Pre: Population probability normalized differently during training vs. inference
Locations: Training: emitter_pre_dataloader.py:59 — normalization via max
Inference: inference_model.py:148 — normalization via sum
Description: The population probability map is normalized using max during training but sum (i.e., to a probability distribution) during inference. This introduces a distributional mismatch between training and inference inputs, which may affect prediction quality.
Potential Fix: Make it consistent with either max or sum at both positions.
Faulty EDIN config
Location: model/CrowdES_edin.yaml
Description: The EDIN model config points to the HOTEL dataset config instead of EDIN.
Potential Fix: Replace
dataset_config: ./configs/dataset/hotel.yaml
with
dataset_config: ./configs/dataset/edin.yaml
Design / Logic Observations
These are not necessarily bugs, but represent design choices that we could not fully reconcile with the paper description.
- Hardcoded scale factor of 1.7 for predicted crowd. No explanation for this value is provided in the paper or code comments. If this was empirically tuned, documenting it would improve reproducibility.
emitter_pipeline.py:204: pred_crowd[:, :, 1::num_classes] *= 1.7
Closing
If we have misunderstood any part of the implementation, we would greatly appreciate clarification. We are happy to provide additional details or minimal examples if helpful.
This is by no means intended to diminish your work — we appreciate that you made the code publicly available, which made this analysis possible.
Detailed Bug Report and Implementation Observations
While building upon this work for our own research, we identified several potential bugs and implementation discrepancies. We are sharing these findings in the spirit of open science and hope they are useful. We have grouped them by severity:
Critical
Emitter: Padding mask is inverted in self-attention
Location: emitter_model.py:207, root cause in emitter_dataloader.py → normalize_padding_origin_goal()
Description: The padding mask produced by
normalize_padding_origin_goal()appears to follow the convention True = agent present, False = padded, whereas PyTorch’snn.MultiheadAttentionexpects the opposite convention (True = ignore / padded, False = attend).If this interpretation is correct, this would result in attention being computed primarily over padded positions, while valid agent positions are masked out.
Suggested fix: Negate the mask before passing it to the attention layer.
Emitter: EmitterPipeline coordinates heuristic sampling
Location: emitter_pipeline.py:206/207
Description: The origin and goal coordinates in the EmitterPipeline do not appear to be generated by the diffusion model. Instead, they are sampled heuristically from the appearance (population) density map. As a result, the diffusion model seems to only influence a subset of variables (e.g., agent type, frame of origin, and preferred speed), while the spatial trajectory endpoints are determined outside of the diffusion process. This appears inconsistent with the expectation that the diffusion model is responsible for generating full spawn conditions. If this interpretation is correct, it suggests that a significant part of the trajectory generation is governed by heuristic sampling rather than the learned model. Also a bit unclear as the leapfrog mechanism is not described in the paper, which makes it difficult to fully understand it.
Simulator: Neighbor Information Leackage
Location: utils/trajectory.py:268ff
Description: The function builds a dictionary mapping each target trajectory window to its neighboring agents' history windows. It does this by extracting a single frame from each window, then merging on the
framecolumn to find temporally co-occurring windows.For
traj(20-frame target windows), it correctly extracts frame indexnth(=history_length - 1= 9), which is the observation endpoint — the last frame the model is allowed to see.But for
hist(10-frame neighbor windows), it uses.first()instead of.nth(nth), extracting frame index 0 — the start of the window:Since the merge is on
frame, this matches a target window whose observation endpoint is at frame F with neighbor windows whose first frame is F. The neighbor's 10-frame window therefore covers[F, F+9]— which is 9 frames into the future relative to the target's observation period[F-9, F].In effect, during training the model sees where neighbors will be in the future, not where they were in the past. At inference time, only true neighbor history is used (built from
traj_hist_meterininference_model.py). This creates a train/inference mismatch: the model learns to rely on future neighbor information that is unavailable at test time.Potential Fix: Use .nth(nth) for both DataFrames so they are matched on the same temporal anchor (the observation endpoint).
Medium
Simulator: Missing neighbor padding mask
Location: simulator_model.py:127ff
Description: The simulator pads missing neighbor slots with zeros (up to
max_neighbors), but never tells the social transformer which slots are padded. Therefore thesocial_transformer(which wrapsnn.TransformerEncoder) attends equally to real neighbors and zero-padded slots.Potential Fix:Detect padding slots by checking for all-zero rows, then pass a proper
padding_mask:Minor
Emitter Pre: Population probability normalized differently during training vs. inference
Locations: Training: emitter_pre_dataloader.py:59 — normalization via max
Inference: inference_model.py:148 — normalization via sum
Description: The population probability map is normalized using max during training but sum (i.e., to a probability distribution) during inference. This introduces a distributional mismatch between training and inference inputs, which may affect prediction quality.
Potential Fix: Make it consistent with either max or sum at both positions.
Faulty EDIN config
Location: model/CrowdES_edin.yaml
Description: The EDIN model config points to the HOTEL dataset config instead of EDIN.
Potential Fix: Replace
with
Design / Logic Observations
These are not necessarily bugs, but represent design choices that we could not fully reconcile with the paper description.
Closing
If we have misunderstood any part of the implementation, we would greatly appreciate clarification. We are happy to provide additional details or minimal examples if helpful.
This is by no means intended to diminish your work — we appreciate that you made the code publicly available, which made this analysis possible.