ocaml-utf8/lib/ at main · gazagnaire.org/ocaml-utf8 · Tangled

gazagnaire.org / ocaml-utf8

0

UTF-8 validation with malformed-offset reporting

0

ocaml-utf8 / lib

3 files

at main

Thomas Gazagnaire utf8: new nox-utf8 package homes the cloned UTF-8 validators 6w ago

utf8: new nox-utf8 package homes the cloned UTF-8 validators dupfind flagged the same Uutf-driven validate_utf8 loop in csv (twice: check_utf8 and validate_utf8_field), sexp (twice) and toml. Neither uutf nor the stdlib offers the function everyone re-implements: String.is_valid_utf_8 is a bare bool, and uutf only has decoders you drive yourself. nox-utf8's Utf8.validate returns Ok () or Error i with the byte offset of the first malformed sequence, taking the stdlib fast path on valid input and uutf's positioned fold only on the error path; vectors from RFC 3629 / Unicode table 3-7 plus fuzz agreement with the stdlib check pin it down. The five call sites collapse to Utf8.validate, each codec keeping its own exception; csv and sexp drop their direct uutf dependency.

1 month ago

utf8: new nox-utf8 package homes the cloned UTF-8 validators dupfind flagged the same Uutf-driven validate_utf8 loop in csv (twice: check_utf8 and validate_utf8_field), sexp (twice) and toml. Neither uutf nor the stdlib offers the function everyone re-implements: String.is_valid_utf_8 is a bare bool, and uutf only has decoders you drive yourself. nox-utf8's Utf8.validate returns Ok () or Error i with the byte offset of the first malformed sequence, taking the stdlib fast path on valid input and uutf's positioned fold only on the error path; vectors from RFC 3629 / Unicode table 3-7 plus fuzz agreement with the stdlib check pin it down. The five call sites collapse to Utf8.validate, each codec keeping its own exception; csv and sexp drop their direct uutf dependency.

1 month ago

utf8: new nox-utf8 package homes the cloned UTF-8 validators dupfind flagged the same Uutf-driven validate_utf8 loop in csv (twice: check_utf8 and validate_utf8_field), sexp (twice) and toml. Neither uutf nor the stdlib offers the function everyone re-implements: String.is_valid_utf_8 is a bare bool, and uutf only has decoders you drive yourself. nox-utf8's Utf8.validate returns Ok () or Error i with the byte offset of the first malformed sequence, taking the stdlib fast path on valid input and uutf's positioned fold only on the error path; vectors from RFC 3629 / Unicode table 3-7 plus fuzz agreement with the stdlib check pin it down. The five call sites collapse to Utf8.validate, each codec keeping its own exception; csv and sexp drop their direct uutf dependency.

1 month ago