Skip to content

Commit 52b7c6c

Browse files
committed
Fix yaml and json loaders
This is mostly an issue on windows, but technically it affects every OS. I know about this issue so there's no excuse, although I guess this means there's no user on windows, or at least of the yaml loader on windows. The (well known, at least by me) problem where is that `open` in text mode (the default) will retrieve the encoding it uses via `locale.getencoding()` which is pretty much always the wrong thing to do but it is what it is[^1]. On most unices this is innocuous because they generally have utf-8 set as the locale encoding, but on windows it's generally going to fuck you up because it likely has a stupid ANSI locale set (only if you're really lucky has someone chanced setting 65001), leading to any non-ascii content potentially breaking file reading as the codepage will either not be able to decode it or will misread it. Both JSON and PyYAML are perfectly happy reading binary files, in which case they'll apply UTF-8 decoding and go on their merry way, so change the code to that. [^1]: it's finally being fixed in 3.15 by PEP 686 https://docs.python.org/3.15/whatsnew/3.15.html#whatsnew315-utf8-default
1 parent 7b1b32d commit 52b7c6c

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

src/ua_parser/loaders.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -196,7 +196,7 @@ def load_json(f: PathOrFile, loader: DataLoader = load_data) -> Matchers:
196196
197197
"""
198198
if isinstance(f, (str, os.PathLike)):
199-
with open(f) as fp:
199+
with open(f, "rb") as fp:
200200
regexes = json.load(fp)
201201
else:
202202
regexes = json.load(f)
@@ -224,7 +224,7 @@ def load_yaml(path: PathOrFile, loader: DataLoader = load_data) -> Matchers:
224224
instead.
225225
"""
226226
if isinstance(path, (str, os.PathLike)):
227-
with open(path) as fp:
227+
with open(path, "rb") as fp:
228228
regexes = load(fp, Loader=SafeLoader) # type: ignore
229229
else:
230230
regexes = load(path, Loader=SafeLoader) # type: ignore

0 commit comments

Comments
 (0)