Commit 52b7c6c
committed
Fix yaml and json loaders
This is mostly an issue on windows, but technically it affects
every OS. I know about this issue so there's no excuse, although I
guess this means there's no user on windows, or at least of the yaml
loader on windows.
The (well known, at least by me) problem where is that `open` in text
mode (the default) will retrieve the encoding it uses via
`locale.getencoding()` which is pretty much always the wrong thing to
do but it is what it is[^1].
On most unices this is innocuous because they generally have utf-8 set
as the locale encoding, but on windows it's generally going to fuck
you up because it likely has a stupid ANSI locale set (only if you're
really lucky has someone chanced setting 65001), leading to any
non-ascii content potentially breaking file reading as the codepage
will either not be able to decode it or will misread it.
Both JSON and PyYAML are perfectly happy reading binary files, in
which case they'll apply UTF-8 decoding and go on their merry
way, so change the code to that.
[^1]: it's finally being fixed in 3.15 by PEP 686
https://docs.python.org/3.15/whatsnew/3.15.html#whatsnew315-utf8-default1 parent 7b1b32d commit 52b7c6c
1 file changed
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
199 | | - | |
| 199 | + | |
200 | 200 | | |
201 | 201 | | |
202 | 202 | | |
| |||
224 | 224 | | |
225 | 225 | | |
226 | 226 | | |
227 | | - | |
| 227 | + | |
228 | 228 | | |
229 | 229 | | |
230 | 230 | | |
| |||
0 commit comments