Skip to content

fix(calculator): block sympify string-eval escape without breaking safe constructors#525

Open
yonib05 wants to merge 5 commits into
strands-agents:mainfrom
yonib05:fix/calculator-sympify-string-eval
Open

fix(calculator): block sympify string-eval escape without breaking safe constructors#525
yonib05 wants to merge 5 commits into
strands-agents:mainfrom
yonib05:fix/calculator-sympify-string-eval

Conversation

@yonib05

@yonib05 yonib05 commented Jun 30, 2026

Copy link
Copy Markdown
Member

Summary

The calculator's expression evaluator validates each expression against an AST allowlist before evaluation. Some functions exposed in the safe locals (for example N, simplify, and solve) accept string arguments and re-parse them through SymPy's sympify, which evaluates that string in a namespace separate from the restricted one used for the top-level expression. As a result a string-literal argument to one of those functions is not subject to the same restrictions as the surrounding expression, which allows code execution (for example simplify("__import__('os').system('...')")).

This change rejects string literals during AST validation, except when they appear as positional arguments to the symbol/number constructors that parse their string as a plain name or numeric literal rather than through sympify: Symbol, symbols, Rational, Integer, and Float. These constructors do not re-evaluate their argument, so they cannot be used to escape the sandbox, and keeping them allowed avoids breaking the documented Symbol('x') / Rational('1/3') syntax.

Changes

  • Reject string-literal ast.Constant nodes in _validate_expression_ast, allowing them only as positional arguments to the safe string constructors (Symbol, symbols, Rational, Integer, Float).
  • Add regression tests: string-literal arguments to the sympify-backed functions are rejected and do not execute code; the safe string constructors remain accepted and a malicious string passed to Symbol() becomes a symbol name rather than executing.
  • Update the module and tool docstrings to describe the positional exception.

Backward compatibility

Not a breaking change. String-argument constructors that were previously valid and safe (Symbol('x'), symbols('x y z'), Rational('1/3'), Integer('5'), Float('3.14')) continue to work. Only string literals in positions that reach sympify (for example N("..."), simplify("..."), solve("...")) or in other non-constructor positions are rejected.

Testing

  • pytest tests/test_calculator.py tests/test_calculator_precision.py (81 passed)
  • ruff format --check and ruff check on the changed files (clean)

…ator

Several functions exposed in the calculator's safe locals (for example N,
simplify, and solve) re-parse string arguments through SymPy's sympify,
which evaluates them in a namespace separate from the restricted one used
for the top-level expression. Mathematical expressions operate on numbers,
symbols, and containers and never need string values, so reject string
literals during AST validation.

Adds regression tests covering string-literal arguments to sympify-backed
functions and plain string literals in other positions.
…g system-of-equations test

Explain that rejecting string-literal arguments (a hardening measure against
SymPy re-parsing a string arg through a builtins-exposed sympify path) also
disallows string-argument constructors such as Symbol('x') and Rational('1/3');
callers should pass bare symbols and numbers. Update the module docstring, tool
docstring, and add a comment at the rejection site.

Replace the mocked test_calculator_tool_with_system_of_equations, which mocked
parse_expression and falsely implied the quoted string-list syntax works, with
unmocked tests: one exercising the supported comma-separated system syntax end
to end, and one asserting the string-list form now returns a validation error.
Permit string literals only as positional arguments to Symbol, symbols,
Rational, Integer, and Float, which parse them as names or numeric
literals rather than through sympify. String literals elsewhere remain
rejected, keeping the sympify-backed re-parse escape (N, simplify, solve)
blocked while restoring the documented constructor syntax.
@yonib05 yonib05 changed the title fix(calculator): reject string-argument calls in the expression evaluator fix(calculator): block sympify string-eval escape without breaking safe constructors Jul 2, 2026
Extend the literal check to bytes so the validator matches its stated
intent (only positional string args to Symbol/symbols/Rational/Integer/
Float are allowed). Add tests covering keyword-position string args and
bytes literals, both of which stay rejected.
@yonib05

yonib05 commented Jul 2, 2026

Copy link
Copy Markdown
Member Author

/strands-ts review

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Review — LGTM ✅

I reviewed the change, ran the suite, and did hands-on security probing. This is a well-scoped, correct hardening of the AST allowlist.

Verification

  • pytest tests/test_calculator.py tests/test_calculator_precision.py85 passed
  • ruff format --check + ruff check on changed files → clean

Security probing (manual)

  • Rejected: N("..."), simplify("..."), solve("..."), factor("..."), expand("..."), integrate("...", x), nested Symbol(simplify("...")), string in keyword position (Symbol('x', foo="...")), and bytes literals everywhere.
  • Allowed (backward compatible): Symbol('x'), symbols('x y z'), Rational('1/3'), Integer('5'), Float('3.14'), and normal math.
  • Confirmed no code execution: a malicious string passed to Symbol() becomes a symbol name and never runs (verified with a filesystem marker that stays absent).

Why the constructor exception is safe

I checked the concern that a maliciously-named symbol could later be re-parsed: sympify does not re-parse a Symbol's name (N(Symbol('x + y')) stays a single symbol, not x + y). So smuggling a string in via Symbol('…') and passing it to a sympify-backed function does not create a bypass. The allowlisted constructors (Symbol, symbols, Rational, Integer, Float) only parse names/numeric literals, not arbitrary expressions.

Notes (non-blocking)

  • The two ast.walk(tree) passes are fine — clarity over micro-optimization, and expression ASTs are tiny.
  • The behavior change to the quoted string-list system syntax ("['x + y - 10', ...]") is well-justified: it was never functional and is now cleanly rejected with a clear error. Good call adding an explicit regression test for it.

Test coverage is thorough (attack vectors, no-exec side-effect checks, positive math, positional-vs-keyword, bytes). No changes requested from me.

The marker-file tests embedded tmp_path into the evaluated expression
string. On Windows the path (C:\Users\...) contains backslash escapes
that make ast.parse raise a unicodeescape SyntaxError, so the tests
failed on Windows for the wrong reason. Use fixed payloads and mock
os.getpid to prove no execution regardless of platform.
@yonib05 yonib05 deployed to auto-approve July 2, 2026 12:46 — with GitHub Actions Active
@yonib05 yonib05 marked this pull request as ready for review July 2, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant