Enable TypeForm support by default#11412
Conversation
...rather than requiring the --enableExperimentalFeatures bit to be set
...by the current interence context or because EvalFlags.TypeFormArg is set.
Without this optimization, the call5.py test fails where it defines a
NamedTuple "X", [("a", int), ...]. Attempting to interpret the string "a"
as a forward reference looks up the name `a` (bound by a later for-loop),
and the resulting flow analysis recursively re-enters the Z(...) constructor,
causing a "Class definition for Z depends on itself" error.
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
Nit: Misspelling in commit message: "current interence context" -> "current inference context"
|
Diff from mypy_primer, showing the effect of this PR on open source code: sympy (https://github.com/sympy/sympy)
+ .../projects/sympy/sympy/polys/euclidtools.py:1974:24 - error: Argument of type "dmp[Er@dmp_cancel] | Unknown | dup[Unknown]" cannot be assigned to parameter "f" of type "dmp[Er@dmp_mul_ground]" in function "dmp_mul_ground"
+ Type "dmp[Er@dmp_cancel] | Unknown | dup[Unknown]" is not assignable to type "dmp[Er@dmp_mul_ground]"
+ "builtins.list" is not assignable to "builtins.list"
+ Type parameter "_T@list" is invariant, but "dmp" is not the same as "dmp"
+ Consider switching from "list" to "Sequence" which is covariant (reportArgumentType)
+ .../projects/sympy/sympy/polys/euclidtools.py:1974:34 - error: Argument of type "Domain[Er@dmp_cancel] | Ring[Unknown]" cannot be assigned to parameter "K" of type "Domain[Er@dmp_mul_ground]" in function "dmp_mul_ground"
+ Type "Domain[Er@dmp_cancel] | Ring[Unknown]" is not assignable to type "Domain[Er@dmp_mul_ground]"
+ "Domain[Er@dmp_cancel]" is not assignable to "Domain[Er@dmp_mul_ground]"
+ Type parameter "Er@Domain" is invariant, but "Er@dmp_cancel" is not the same as "Er@dmp_mul_ground" (reportArgumentType)
- .../projects/sympy/sympy/polys/euclidtools.py:1976:12 - error: Type "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel]]" is not assignable to return type "tuple[dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]] | tuple[Er@dmp_cancel, Er@dmp_cancel, dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
+ .../projects/sympy/sympy/polys/euclidtools.py:1976:12 - error: Type "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel | Unknown]]" is not assignable to return type "tuple[dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]] | tuple[Er@dmp_cancel, Er@dmp_cancel, dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
- Type "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel]]" is not assignable to type "tuple[dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]] | tuple[Er@dmp_cancel, Er@dmp_cancel, dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
+ Type "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel | Unknown]]" is not assignable to type "tuple[dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]] | tuple[Er@dmp_cancel, Er@dmp_cancel, dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
- "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel]]" is not assignable to "tuple[Er@dmp_cancel, Er@dmp_cancel, dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
+ "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel | Unknown]]" is not assignable to "tuple[Er@dmp_cancel, Er@dmp_cancel, dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
- "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel]]" is not assignable to "tuple[dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
+ "tuple[dmp[Er@dmp_cancel | Unknown | RingElement*], dmp[Er@dmp_cancel | Unknown]]" is not assignable to "tuple[dmp[Er@dmp_cancel], dmp[Er@dmp_cancel]]"
- .../projects/sympy/sympy/solvers/bivariate.py:135:15 - error: Operator "-" not supported for "None" (reportOptionalOperand)
- .../projects/sympy/sympy/solvers/bivariate.py:139:17 - error: Operator "-" not supported for type "Basic | Unknown" (reportOperatorIssue)
- .../projects/sympy/sympy/solvers/bivariate.py:144:23 - error: Operator "-" not supported for "None" (reportOptionalOperand)
- .../projects/sympy/sympy/solvers/deutils.py:234:14 - error: Operator "not in" not supported for types "str" and "Unknown | int"
- Operator "not in" not supported for types "str" and "int" (reportOperatorIssue)
- .../projects/sympy/sympy/solvers/diophantine/diophantine.py:176:44 - error: Cannot access attribute "expand" for class "Basic"
- .../projects/sympy/sympy/solvers/diophantine/diophantine.py:423:38 - error: Argument of type "Unknown | Expr | Literal[0]" cannot be assigned to parameter "expr" of type "Expr" in function "make_args"
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:423:38 - error: Argument of type "int | Expr" cannot be assigned to parameter "expr" of type "Expr" in function "make_args"
- Type "Unknown | Expr | Literal[0]" is not assignable to type "Expr"
+ Type "int | Expr" is not assignable to type "Expr"
- "Literal[0]" is not assignable to "Expr" (reportArgumentType)
+ "int" is not assignable to "Expr" (reportArgumentType)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:504:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:505:19 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:506:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:569:42 - error: Operator "*" not supported for types "int" and "Unknown | Basic"
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:569:50 - error: Operator "*" not supported for types "Expr" and "Unknown | Basic"
+ Operator "*" not supported for types "Expr" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:725:42 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:735:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:736:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:737:19 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:814:47 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:841:26 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:842:22 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:36 - error: Operator "*" not supported for types "int" and "Unknown | Basic"
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:42 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:51 - error: Operator "*" not supported for types "int" and "Unknown | Basic"
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:57 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:843:66 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:845:53 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:854:18 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:856:22 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:865:22 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:865:36 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:867:27 - error: Operator "**" not supported for types "Unknown | Basic" and "Literal[2]"
+ Operator "**" not supported for types "Basic" and "Literal[2]" (reportOperatorIssue)
+ .../projects/sympy/sympy/solvers/diophantine/diophantine.py:868:27 - error: Operator "*" not supported for types "Unknown | Basic" and "Unknown | Basic"
+ Operator "*" not supported for types "Basic" and "Basic" (reportOperatorIssue)
... (truncated 1245 lines) ...
|
|
There's a prettier error too:
Checking formatting... You should be able to run |
|
Squish change to -> Do not try to interpret a string as a TypeForm unless one is expected
Squish to -> Do not try to interpret a string as a TypeForm unless one is expected
Squish to -> Enable TypeForm support by default
I previously reported a performance issue with TypeForm support: #11008. This PR mostly fixes that large slowdown, but it seems like Pyright is still a little slower with TypeForm support enabled for that testcase. |
|
Analysis of mypy_primer output:
Analysis of new errors in diophantine.pyErrors at diophantine.py:175-183 are in the class DiophantineEquationType:
"""..."""
name: str
def __init__(self, equation, free_symbols=None): # LINE 175
self.equation = _sympify(equation).expand(force=True)
if free_symbols is not None:
self.free_symbols = free_symbols
else:
self.free_symbols = list(self.equation.free_symbols)
self.free_symbols.sort(key=default_sort_key) # LINE 182
@overload
def sympify(a: int, *, strict: bool = False) -> Integer: ... # type: ignore
@overload
def sympify(a: float, *, strict: bool = False) -> Float: ...
@overload
def sympify(a: Expr | complex, *, strict: bool = False) -> Expr: ...
@overload
def sympify(a: Tbasic, *, strict: bool = False) -> Tbasic: ...
@overload
def sympify(a: Any, *, strict: bool = False) -> Basic: ...
def sympify(a, locals=None, convert_xor=True, strict=False, rational=False,
evaluate=None): ...
For reference, the revealed types within diophantine.py:175-183 changed in the following ways: Analysis of new errors in euclidtools.pyAffected code: def dmp_cancel(...) -> tuple[dmp[Er], dmp[Er]] | tuple[Er, Er, dmp[Er], dmp[Er]]:
...
p = dmp_mul_ground(p, cp, u, K) # LINE 1973
q = dmp_mul_ground(q, cq, u, K) # LINE 1974
return p, q # LINE 1976One new error at line 1974 ( The two new errors at line 1974 are word-for-word identical to the two errors already at line 1973 (with q/cq substituted for p/cp). On main branch (Baseline) vs. this feature branch (TypeForm), the inputs to this section of code are identical: So p and q have identical types going in, yet:
Probable cause: A constraint-solver asymmetry in baseline.The only difference between the two calls is the second argument's type. cp is a 3-way union that includes RingElement* (the Er TypeVar's bound); cq is a 2-way union without it. Baseline pyright treats these very differently when solving for Er@dmp_mul_ground:
TypeForm enabling makes the solver flag both cases uniformly. The q value at TypeForm enabling makes the solver flag both cases uniformly. Thus the "new" error at line 1974 is not a regression. ✅ Summary of next steps
|
|
Holy moly. diophantine.py is complex. I think I'm going to disregard the remaining error deltas in sympy. sympy was already producing very many errors under pyright before this branch and now its producing very many but different errors after this branch. Not worth the investigation time: sympy was already broken at the start. I've just pushed up commits integrating the outstanding feedback. NextSo the next item (and possibly last) item I want to investigate is any performance regressions that this branch might introduce. I'll look at the psf/black codebase as issue #11008 did. |
|
GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for packages/pyright-internal/src/analyzer/typeEvaluator.ts:L5574. Copilot generated: Suggested fix: function expectedTypeWantsTypeForm(expectedType: Type): boolean {
return someSubtypes(expectedType, (subtype) =>
isClassInstance(subtype) && ClassType.isBuiltIn(subtype, 'TypeForm')
);
}🔍 Structurally confirmed: [verified] |
| @@ -1743,11 +1744,15 @@ export function createTypeEvaluator( | |||
| }; | |||
There was a problem hiding this comment.
Copilot generated:
The wantsTypeForm guard is well-designed — it prevents expensive string-as-TypeForm interpretation for plain string literals. However, [unverified] — it's unclear whether inferenceContext propagates deeply enough through container literal evaluation. For example, in d: dict[str, TypeForm] = {"key": "int"}, does the TypeForm expected type reach the string "int" via inference context? A targeted test case for container scenarios (list[TypeForm], dict[str, TypeForm]) would confirm or refute this.
[verified]
|
GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for packages/pyright-internal/src/analyzer/typeEvaluator.ts:L7659. Copilot generated:
[verified] |
|
GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for packages/pyright-internal/src/analyzer/typeEvaluator.ts:L990. Copilot generated: All 7 TypeForm test files (typeForm1–7) still explicitly set [verified] |
...rather than requiring the --enableExperimentalFeatures bit to be set
CONTEXT:
OPEN QUESTIONS: