add Zoi.Ecto: changeset bridge for Zoi parse errors#162
Conversation
|
@phcurado Let's please not merge this PR until we have decided whether some gaps I want to tell you about will be addressed first, or will be left for future PRs. I pinged you on ElixirForum to let me know where do you want them discussed. Point being: current Ecto compatibility is not 100%; there are some gaps. If Zoi fills them then we can get to 100%. |
|
sorry taking a bit of time but I addressed the problems you mentioned. I think I implemented all suggestions now, it's on the main branch if u want to check it out (not released yet). Only point I didnt work was the coerce because we already have helpers for that. Zoi.map(%{
address: Zoi.map(%{zip: Zoi.string()})
})
|> Zoi.Schema.traverse(&Zoi.coerce/1)
|> Zoi.coerce()should already coerce everything, from basic types to more complex structures |
Adds `Zoi.Ecto.changeset/2` and `errors_to_changeset/1` to convert Zoi parse errors into Ecto-native changesets. Error opts match Ecto's `validation:`/`kind:` format with template-style messages. Nested maps produce nested changesets, arrays produce padded lists. Ecto is an optional dependency.
`changeset/2` now uses `Zoi.Context` directly so that successfully parsed flat fields and nested maps are preserved in `.data` on failure. Array items still get empty changesets at non-errored positions because `Context.parsed` drops errored array items without preserving their index. Use `&fun/1` capture for `Zoi.refine` in test schemas.
enum values are now a native list (phcurado#167), no more string splitting. size validation uses opts[:type] (phcurado#170) instead of template matching. valid array items carry parsed data from Zoi.Context (phcurado#166) instead of empty changesets. updated known limitations to reflect resolved items.
Ecto uses type: :list for array length errors, Zoi uses type: :array. remap to match Ecto's convention so consumers checking opts[:type] get the expected value.
full cross-reference of all Ecto validation types vs Zoi error codes, with exact opts produced. separate table for Ecto validations that have no Zoi equivalent and why.
|
Awesome work @phcurado, thank you. I have rebased and me & Claude did the following:
RE: the coerce, got it, it's not bothering me. Also added a Markdown table with what exists and does not exist in Zoi that Ecto supports so the users of the library have a clear picture. |
two variants sharing an email field with different constraints prove that discriminator in error opts identifies which variant produced the error. demonstrates the value of phcurado's phcurado#168.
|
Also added a test for |
|
hi @dimitarvp thanks btw for helping so much here, I know it takes a lot of time and effort so I appreciate it! I looked at the PR and seems in a decent shape, do you mind if I do some changes and play around with your PR (I can branch out your PR if you dont want me to change)? I want to see if I can simplify/improve somethings. I know you mentioned you used AI and I think I can improve this part specifically since I know the lib pretty well. I think Claude exaggerated in some guards (I might be mistaken I need to properly test it out) but I think I can improve it. Btw regarding the core logic, what do you think in general, do you think it will help you and the problems you face? Or do you think you might need more features (such as embeds), etc? Brainstorming here so I can get some ideas for future support with ecto and if I finally make the decision to add here or we create another lib |
|
Hey @phcurado. Of course, feel free to massage this -- you know your codebase best. I have used Claude indeed, though I am quite rigorous and do architecture and design much more because as we know LLMs are very happy to take shortcuts or outright skip work. So please, play with it and modify the PR. And yes this PR absolutely will help me in my work already as it is. No need for embedded schemas just yet. By itself it's already super valuable and if it was officially in I could start integrating it tomorrow. HOWEVER. I do think more integration -- like the embedded schemas -- can be added with time, which indeed makes it potentially a better idea to spin off all this into a The logistics of what goes into which project don't trouble me but you as the author should take all this into account. I am fine using the code of this PR either way: as part of core Zoi or as a separate library. My role was to explain the need for this so you know what problems are people like me solving with Zoi, plus give you a code draft. From then on it's up to you. If you allow me one vague idea: you could look into how does Ash integrate with Ecto, especially forms. Might give you an idea or two, I think. So TL;DR of course, feel free to modify this until you are satisfied with it. My apologies for the many coding lines added -- I worked with Claude for hours to make sure the test coverage is good and that's really like 75% of all code added here, as you can see. I'll monitor here periodically out of curiosity. Let me know if that helps or if you think anything else deserves discussion! |
|
I might be wrong, could not improve much the current implementation, it's consistent and ecto way of working is what makes it more complex to convert the After analyzing I'm willing to take this to a separated package, I think it will be a better direction. Let me know if you would like to be the owner of the package. If not I can take the lead and I will do it, although will be highly inspired on all the good work you have done already. |
|
Yes, I prefer authoring it and maintaining it. If you're okay with it, I'd like to give you maintainer access as well. |
|
that would be perfect, happy to contribute aswell |
🤖⚠️ ROBOT DISCLAIMER 🤖⚠️
This entire PR, and the texts, are heavily influenced and coded by Claude Code -- with a lot of input and iteration from me. I have a pretty good idea what extra functionality I need from Zoi so I can daily-drive it in my work and recommend it to other people and companies, and I have made sure to try and get the maximum Ecto compatibility as the utmost priority.
Adds
Zoi.Ecto.changeset/2anderrors_to_changeset/1to convert Zoi parse errors into Ecto-native changesets. Error opts match Ecto'svalidation:/kind:format with template-style messages. Nested maps produce nested changesets, arrays produce padded lists. Ecto is an optional dependency so if the using project does not have it in itsmix.exsthen this code will not be compiled into it at all.What does this add to Zoi?
A lightweight bridge so that Zoi parse errors can be consumed by anything in the Phoenix/Ecto ecosystem that expects
%Ecto.Changeset{}-- Absinthe error rendering, LiveView forms,traverse_errors/2, etc.Two public functions:
Zoi.Ecto.changeset(schema, input)-- parses with Zoi viaZoi.Context, returns a changeset (valid or invalid). On failure, successfully parsed flat fields and nested maps are preserved in.data.Zoi.Ecto.errors_to_changeset(errors)-- converts[%Zoi.Error{}]into a changesetEcto error format mapping
validation:kind::required:required:invalid_type:cast:invalid_format:format:invalid_enum_value:inclusionenum: [list]:invalid_length:length:iscount::greater_than:numberor:length:greater_thanor:minnumber:for numeric:greater_than_or_equal_to:numberor:length:greater_than_or_equal_toor:minnumber:for numeric:less_than:numberor:length:less_thanor:maxnumber:for numeric:less_than_or_equal_to:numberor:length:less_than_or_equal_toor:maxnumber:for numeric:unrecognized_key:unknown_field:custom(fromrefine):customSize codes (
:numbervs:length) are disambiguated via Zoi's error message templates -- string constraints end with"character(s)", array constraints end with"item(s)". Length errors use Ecto-native:min/:max/:isforkind:. Numeric errors includenumber: target_valuematching Ecto's convention.Error path routing
[]→:base(root-levelrefineerrors)[:field]→ flat changeset error[:address, :zip]→ nested changeset inchanges[:address][:items, 0, :name]→ list of changesets inchanges[:items]Tests
E-commerce order scenario: nested address, array of line items with min/max, discriminated union for payment method, cross-field validation via
refine. Covers every mapped code, every error path pattern, edge cases, Ecto compatibility, and partial parse data preservation.Known limitations
These may be raised as separate issues and/or addressed in follow-up PRs:
Zoi.Context.parseddrops errored array items without preserving their original index -- valid items shift position, making alignment impossible.enum:key is produced by splitting Zoi's comma-joinedvaluesstring. Values containing", "will split incorrectly.type:key -- Ecto includestype: :string/:list, Zoi's error opts don't carry target type info."character(s)"/"item(s)").