chore/sc-41640/reducing-tokens-of-disambiguator#3262
Open
nsantacruz wants to merge 49 commits intomasterfrom
Open
chore/sc-41640/reducing-tokens-of-disambiguator#3262nsantacruz wants to merge 49 commits intomasterfrom
nsantacruz wants to merge 49 commits intomasterfrom
Conversation
… for ambiguous entries
…ed reference handling
…d build section-level base refs _get_commentary_base_ref previously matched nodes by internal key (e.g. 'OrachChaim'), causing misses when the key diverges from the English display title (e.g. Prisha → Tur). It also did not handle the case where the base index has an extra SchemaNode wrapper around its content leaf. Changes: - Match nodes by primary English title instead of schema key - After a successful match, append non-default node titles to base_title to form a valid complex ref (e.g. 'Tur, Orach Chayim') - Navigate through any intermediate SchemaNodes to the content leaf, then zip with the leaf's addressTypes[:-1] to produce a section-level (not segment-level) base ref - Return early from the complex/complex branch so the generic section loop only runs for simple bases Add disambiguator_helpers_test.py covering None/empty inputs, non-commentary, multiple base titles, book-level citing refs, both-simple Torah and Talmud cases, both-complex with matching and mismatching titles, complex/simple XOR cases, and missing base_text_titles.
High-score Dicta matches (score >= 8) previously bypassed LLM confirmation, causing false positives when the citation was a vague book/folio reference with no direct quote. Key changes: - Add _verify_high_score: a dedicated LLM check for high-score candidates with a focused prompt warning about book-title citations, bare folio references, and incidental topical similarity - Add VERIFICATION_EXAMPLES: 6 few-shot examples (4 NO, 2 YES) covering book titles, folio refs with/without inline quotes, paraphrases, and whole-book references; placed in SystemMessage with cache_control - Add _get_candidate_text_for_confirmation: pads short candidates with neighboring segments for better confirmation context - Update _llm_form_prior to flag whole-book/chapter citations early - Move examples into SystemMessage with explicit cache_control in both _llm_confirm_candidate and _verify_high_score Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Skip non-segment spans that already have `llm_resolved_ref_non_segment` set, and treat ambiguous losers (llm_ambiguous_option_valid=False) as resolved so they are not re-grouped and re-dispatched. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
False positives occurred when Dicta matched a high-score quote elsewhere in the document body rather than adjacent to the actual citation marker. Now only bypass LLM verification when the matched phrase is within 60 characters of the citation in the windowed text, measuring from the nearest edges of the phrase and citation intervals. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ref_but_rejected When _confirm_candidate returns NO (in both the Dicta and fallback search paths), save the rejected candidate ref on the span under llm_resolved_ref_but_rejected. Adds rejected_ref to NonSegmentResolutionResult and threads it through the apply functions so it is written even when no resolution is ultimately found. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Text to best-matching window - _dicta_phrase_distance: replace abs-distance formula with proper interval logic so any overlap between the resolution phrase and the citation returns 0 (previously a phrase containing the citation could return a large positive distance) - resolution_phrase: when Dicta provides both baseMatchedText and compMatchedText, use Levenshtein sliding-window search (normalized to strip nikkud/cantillation) to find the sub-span of baseMatchedText closest to compMatchedText, then map positions back to the original string so the result is always a true substring Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the disabled `if False` gate with a rule derived from 1,600-row statistical analysis: - section_level=True AND score>=8 AND dist<=10: 99.3% precision (2 misses/280) - score>=15 AND dist<=5 (any section level): 100% precision (0 misses/101) Previous rule (score>=6 / score>=8 with no distance gate) had ~4.3% miss rate and included a risky non-section-level branch at only ~88% precision. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
📊 Code Quality Score: 27/100
Scored by GitVelocity · How are scores calculated? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A brief description of the PR
Code Changes
The following changes were made to the files below
Notes
Any additional notes go here