Skip to content

fix: don't strip title when it is a name, not a schema keyword - #12219

Open
LHMQ878 wants to merge 4 commits into
deepset-ai:mainfrom
LHMQ878:fix/tool-schema-title-name-keyed-and-data-keywords
Open

fix: don't strip title when it is a name, not a schema keyword#12219
LHMQ878 wants to merge 4 commits into
deepset-ai:mainfrom
LHMQ878:fix/tool-schema-title-name-keyed-and-data-keywords

Conversation

@LHMQ878

@LHMQ878 LHMQ878 commented Aug 2, 2026

Copy link
Copy Markdown

Related Issues

No open issue — this is the remaining half of the bug class fixed by #12037. That PR stopped _remove_title_from_schema from misreading the keys of a properties mapping as schema keywords; the same reasoning applies to eight more keywords, and nothing in #12037 or its discussion mentions them, so this looks like an unnoticed gap rather than a deferred one.

Proposed Changes:

_remove_title_from_schema walks a Pydantic-generated schema deleting every title key. title is a prose keyword, but the string title is also a legal definition name, a legal patternProperties regex, a legal property name, and a legal key inside a default/const/enum value. The walker can't tell those apart, so it corrupts the schema in two distinct ways:

1. Name-keyed maps — the key gets deleted. $defs, definitions, patternProperties, dependentSchemas, dependentRequired are all keyed by user-chosen names, not by schema keywords.

$defs is the one that's reachable without trying: Pydantic keys $defs by class name, so a nested model named title produces a definition named title, which is then deleted while the $ref pointing at it survives. Through the public API:

class title(BaseModel):        # a nested model named `title`
    text: str

class Report(BaseModel):
    heading: title
    body: str

def make_report(report: Report) -> str:
    """Create a report."""

tool = create_tool_from_function(make_report)

before → after this PR:

$defs keys:     ['Report']                                  ->  ['Report', 'title']
dangling $ref:  #/$defs/Report/properties/heading -> #/$defs/title   ->  (none)
jsonschema validate of a tool call:
  _WrappedReferencingError: PointerToNowhere: '/$defs/title' does not exist
                                                            ->  passes

Any consumer that resolves $refs — schema validation, an OpenAI-style strict-schema conversion, a provider-side schema check — sees a schema that points at nothing. For the other four keywords the effect is a silently dropped validation rule rather than a dangling pointer.

2. Instance-data keywords — the value gets edited. The values of default, const and enum are instance data, not subschemas. A title key inside one of them belongs to the value:

def render(options: Annotated[dict, "rendering options"] = {"title": "Untitled", "width": 80}) -> str: ...
emitted default:  {"width": 80}   ->  {"title": "Untitled", "width": 80}

The model is now told the default is a dict without a title, so it can't reason about the real default and a caller relying on the schema's default writes the wrong value. No error is raised anywhere.

The fix generalises #12037's properties special case into two module-level frozensets: name-keyed maps are recursed into by value only (keys kept verbatim), and instance-data keywords are skipped entirely. Everything else is unchanged, so keywords whose value is a genuine subschema (items, propertyNames, additionalProperties, anyOf, …) keep losing their title exactly as before.

Both public entry points benefit, since ComponentTool (component_tool.py:375) calls the same helper as create_tool_from_function / @tool.

How did you test it?

  • 7 new unit tests in test/tools/test_from_function.py: one per affected group ($defs incl. a real jsonschema.Draft202012Validator round-trip, the draft-07 definitions spelling, instance data, pattern/dependent keys), plus two through the public API (create_tool_from_function with a nested model named title, and a default carrying a title key), plus one guarding the opposite directionitems/propertyNames/additionalProperties must still lose their title.
  • Reverting only haystack/tools/from_function.py and keeping the new tests: 6 fail, 25 pass. The opposite-direction guard passes either way, as it should. With the fix: 31 passed.
  • pytest test/tools/ -m "not integration"292 passed, 12 deselected, no change in the pass set.
  • ruff check / ruff format --check clean; mypy haystack/tools/from_function.py reports nothing in the touched file.
  • Release note added under releasenotes/notes/.

Notes for the reviewer

  • The two lists are deliberately explicit rather than "recurse into values for anything that looks like a map" — that heuristic would break properties-shaped keywords that genuinely are keyed by keywords, and being explicit keeps the failure mode a missing entry rather than a silently kept title.
  • dependentRequired's values are arrays of strings, not schemas, so it only needs its keys protected; it is grouped with the name-keyed maps because the isinstance(value, dict) guard plus the value-recursion is a no-op for it.
  • test/tools/test_from_function.py defines a lowercase class title(BaseModel) at module level with # noqa: N801. The lowercase name is the point — it's what makes Pydantic emit a $defs entry named title.
  • Prose stripping is unchanged in every case where title really is a keyword, including the title keyword inside the $defs["title"] definition, which is still removed.

Checklist

deepset-ai#12037 stopped `_remove_title_from_schema` from misreading the keys of a
`properties` mapping as schema keywords. The same reasoning applies to five
more keywords whose values are keyed by user-chosen names, and to three whose
values are instance data rather than subschemas:

- `$defs`, `definitions`: an entry named `title` was deleted, leaving every
  `$ref` that pointed at it dangling. Pydantic keys `$defs` by class name, so
  a nested model named `title` reaches this path through the public API and
  the resulting schema fails validation with `PointerToNowhere`.
- `patternProperties`, `dependentSchemas`, `dependentRequired`: a rule keyed
  on the property name / regex `title` was silently dropped.
- `default`, `const`, `enum`: a `title` key inside one of these is part of the
  *value*. A default of `{"title": "Untitled", "width": 80}` was emitted as
  `{"width": 80}`, changing the tool's contract.

Keywords whose value is a genuine subschema (`items`, `propertyNames`,
`additionalProperties`, `anyOf`, ...) keep losing their `title` as before;
a test guards that direction too.
@LHMQ878
LHMQ878 requested a review from a team as a code owner August 2, 2026 18:33
@LHMQ878
LHMQ878 requested review from davidsbatista and removed request for a team August 2, 2026 18:33
@vercel

vercel Bot commented Aug 2, 2026

Copy link
Copy Markdown

@LHMQ878 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Aug 2, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @LHMQ878, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot
HaystackBot removed the request for review from davidsbatista August 2, 2026 19:45
@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Aug 2, 2026
@HaystackBot
HaystackBot marked this pull request as draft August 2, 2026 19:45
@HaystackBot
HaystackBot marked this pull request as ready for review August 4, 2026 02:00
@HaystackBot

Copy link
Copy Markdown
Contributor

Thanks for signing the CLA, @LHMQ878! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

@HaystackBot HaystackBot removed the cla-pending PR is in draft until the contributor signs the CLA label Aug 4, 2026

@ebarkhordar ebarkhordar left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One keyword looks like it belongs in _DATA_SCHEMA_KEYWORDS next to default/const/enum: examples. Its value is a list of instance values, Pydantic emits it straight from Field(examples=...), and the walker's list branch recurses into each example dict, so a title key that is part of the data gets deleted.

Measured on this branch (451ce7b), through the public API:

class Cfg(BaseModel):
    opts: dict = Field(
        default={"title": "Untitled", "width": 80},
        examples=[{"title": "Untitled", "width": 80}, {"title": "Draft", "width": 40}],
    )

def configure(cfg: Cfg) -> str:
    """Configure the renderer."""

tool = create_tool_from_function(configure)
tool.parameters["$defs"]["Cfg"]["properties"]["opts"]
default:  {'title': 'Untitled', 'width': 80}    kept, as this PR intends
examples: [{'width': 80}, {'width': 40}]        'title' dropped from both

Adding "examples" to the frozenset restores it. pytest test/tools/ -m "not integration" is 292 passed both with and without the addition (python:3.12-slim, runtime deps from pyproject).

Not blocking, and whether it belongs here or in a follow-up is your call. The rest reads right to me, and the $defs case is the one that genuinely bites: a dangling $ref breaks every consumer that resolves refs, while the other four lose a validation rule quietly.

@LHMQ878

LHMQ878 commented Aug 4, 2026

Copy link
Copy Markdown
Author

Good catch. Added examples to _DATA_SCHEMA_KEYWORDS in 9e7a2e7 and extended the existing instance-data regression test so a itle key inside an example is preserved while the schema-level itle is still removed. Focused ruff check and format check pass. I could not run the Hatch test command here because Hatch is not installed in this environment.

@github-actions github-actions Bot added the type:documentation Improvements on the docs label Aug 4, 2026
@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
haystack-docs Ready Ready Preview Aug 4, 2026 2:53pm

Request Review

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/tools
  from_function.py
Project Total  

This report was generated by python-coverage-comment-action

@davidsbatista

Copy link
Copy Markdown
Contributor

@sjrl, this seems OK to me, resuming a previous fix, and generalising to more keywords - but I think it's good if you have an extra look

@davidsbatista
davidsbatista requested a review from sjrl August 4, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants