Skip to content

Recursive-rel node filter is silently ignored in SHORTEST mode (wrong results) #680

Description

@jfno

Summary

A node filter on a recursive relationship, for example
-[e:Knows* SHORTEST 1..3 (r, n | WHERE ...)]-, is not applied when the recursive rel
uses SHORTEST mode. The same filter works in plain (non-SHORTEST) mode. There is no
error or warning, so queries return wrong results that look plausible.

  • ladybug 0.18.1 (pip, macOS arm64): reproduces
  • kuzu 0.11.3 (final upstream release): reproduces. Upstream is archived, hence filing here.

Minimal reproduction

Three persons in a chain. The middle one fails the predicate, so with a per-hop filter
nothing should be reachable from person 1:

(1, date=100) --KNOWS-- (2, date=50) --KNOWS-- (3, date=200)      filter: n.date >= 100
import ladybug

db = ladybug.Database(":memory:")
con = ladybug.Connection(db)
con.execute("CREATE NODE TABLE Person(id INT64 PRIMARY KEY, date INT64)")
con.execute("CREATE REL TABLE Knows(FROM Person TO Person)")
con.execute("CREATE (:Person {id: 1, date: 100})")
con.execute("CREATE (:Person {id: 2, date: 50})")
con.execute("CREATE (:Person {id: 3, date: 200})")
con.execute("MATCH (a:Person {id: 1}), (b:Person {id: 2}) CREATE (a)-[:Knows]->(b)")
con.execute("MATCH (a:Person {id: 2}), (b:Person {id: 3}) CREATE (a)-[:Knows]->(b)")

def rows(q):
    res = con.execute(q)
    out = []
    while res.has_next():
        out.append(res.get_next())
    return out

print(rows("""
    MATCH (p:Person {id: 1})-[e:Knows* SHORTEST 1..3 (r, n | WHERE n.date >= 100)]-(x:Person)
    WHERE x.id <> 1 AND x.date >= 100
    RETURN x.id ORDER BY x.id
"""))
print(rows("""
    MATCH (p:Person {id: 1})-[e:Knows*1..3 (r, n | WHERE n.date >= 100)]-(x:Person)
    WHERE x.id <> 1 AND x.date >= 100
    RETURN x.id ORDER BY x.id
"""))

Output on 0.18.1:

[[3]]   <- SHORTEST mode: person 3 returned
[]      <- plain mode: correct (empty)

Person 3 passes the endpoint predicate but is only reachable through person 2, which
fails the per-hop filter. So [] is the correct answer in both modes, as plain mode
confirms. In SHORTEST mode the traversal walks through person 2 as if the filter were
not there. Only the endpoint WHERE is applied.

Larger-scale confirmation

Found while benchmarking LDBC SNB SF1 (person/knows graph, about 9.9k persons and 180k
edges). A query counting persons reachable within 5 hops through persons passing a
creationDate cutoff returns 7778 under SHORTEST with the per-hop filter. That is
bit-identical to the answer for the unfiltered traversal. The correct count is 7697,
confirmed by three independent implementations (a SQL recursive query in DuckDB, a
level-wise BFS unrolled in Cypher on ladybug itself, and a hand-written BFS).

Two side observations from that setup, in case they help locate the issue:

  • The filtered SHORTEST variant runs about 1.6x slower than the same query without the
    filter, while returning identical unfiltered results. So the filter expression seems
    to be evaluated somewhere, but its result does not constrain the traversal.
  • Plain-mode recursion honors the filter but enumerates trails, which explodes
    combinatorially at depth 4 and beyond on this graph, so it is not a practical
    workaround. SHORTEST is exactly the mode where this filter matters.

Happy to provide the SF1 setup or any additional detail.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions