Merge pull request #13 from smarie/feature/12_sequences_and_scalars

Feature/12 sequences and scalars

Author: smarie

ci_tools/.gitignore

@@ -11,4 +11,5 @@ pydocstyle>=5.1.1,<6
 pycodestyle>=2.6.0,<3
 mccabe>=0.6.1,<1
 naming>=0.5.1,<1
-pyflakes>=2.2,<3
+pyflakes>=2.2,<3
+genbadge[flake8]

ci_tools/flake8-requirements.txt

@@ -1,5 +1,9 @@
 # Changelog
 
+### 1.1.0 - Now load objects from sequences and scalars too
+
+ - Objects (subclasses of `YamlAble` or `YamlObject2`) can now be loaded from both mappings, sequences and scalars. Codecs (subclasses of `YamlCodec`) can also support this feature. Fixes [#12](https://github.com/smarie/python-yamlable/issues/12)
+
 ### 1.0.4 - better type hinting (mypy)
 
  - Most type hints have been fixed, in particular for `@yaml_info`. Fixes [#11](https://github.com/smarie/python-yamlable/issues/11).

docs/changelog.md

@@ -26,6 +26,10 @@ In addition `yamlable` provides a way to create Yaml codecs for several object t
 
 ## Usage
 
+### 1. The (recommended) `YamlAble` way
+
+#### a. Creating the class
+
 Let's make a class yaml-able: we have to
 
  - inherit from `YamlAble`
@@ -38,7 +42,7 @@ from yamlable import yaml_info, YamlAble
 @yaml_info(yaml_tag_ns='com.yamlable.example')
 class Foo(YamlAble):
 
-    def __init__(self, a, b):
+    def __init__(self, a, b="hey"):
         """ Constructor """
         self.a = a
         self.b = b
@@ -67,6 +71,8 @@ That's it! Let's check that our class is correct and allows us to create instanc
 Foo - {'a': 1, 'b': 'hello'}
 ```
 
+#### b. Dumping and loading to/from YAML
+
 Now let's dump and load it using `pyyaml`:
 
 ```python
@@ -105,7 +111,44 @@ a: 1
 b: hello
 ```
 
-See [Usage](./usage) for other possibilities of `yamlable`.
+#### c. Support for sequences and scalars
+
+Objects can also be loaded from YAML sequences:
+
+```python
+>>> print(yaml.safe_load("""
+!yamlable/com.yamlable.example.Foo
+- 0
+- hey
+"""))
+
+Foo - {'a': 0, 'b': 'hey'}
+```
+
+The default implementation of `__from_yaml_sequence__` (that you may wish to override in your subclass), is to call
+the constructor with the sequence contents as positional arguments.
+
+The same also works for scalars:
+
+```python
+>>> print(yaml.safe_load("""
+!yamlable/com.yamlable.example.Foo 0
+"""))
+
+Foo - {'a': "0", 'b': 'hey'}
+```
+
+The default implementation of `__from_yaml_scalar__` (that you may wish to override in your subclass), is to call
+the constructor with the scalar as first positional argument.
+
+!!! warning "Scalars are not resolved"
+    As can be seen in the above example, scalars are not auto-resolved when constructing an object from a scalar. So an
+    integer `0` is actually received as a string `"0"` by `from_yaml_scalar`.
+
+
+#### d. What if you can not modify the class ?
+
+See [Usage](./usage#yamlcodec) for another possibility offered by `yamlable`: creating a codec to handle YAML for several classes at once, typically classes that you cannot modify.
 
 
 ## Main features / benefits

docs/index.md

@@ -4,6 +4,8 @@ You have seen in the [main page](./index) a small example to understand the conc
 
 ## `YamlCodec`
 
+### 1. Writing a codec class
+
 Sometimes you do not have the possibility to change the classes of the objects that you wish to encode/decode. In this case the solution is to write an independent codec, inheriting from `YamlCodec`. Once again this feature leverages the `multi_constructor` and `multi_representer` concepts available in the `PyYaml` internals, but with `YamlCodec` it becomes a bit easier to do.
 
 Let's assume that the following two classes are given and can not be modified:
@@ -84,13 +86,17 @@ class MyCodec(YamlCodec):
         return types_to_yaml_tags[type(obj)], vars(obj)
 ```
 
+### 2. Registering a codec
+
 When you codec has been defined, it needs to be registerd before being usable. You can specify with which `PyYaml` Loaders/Dumpers it should be registered, or use the default (all): 
 
 ```python
 # register the codec
 MyCodec.register_with_pyyaml()
 ```
 
+### 3. Using a codec
+
 Finally let's test that the codec works:
 
 ```python
@@ -111,3 +117,10 @@ assert dump(b) == by
 assert f == load(fy)
 assert b == load(by)
 ```
+
+### 4. Sequences and scalars
+
+Objects can be loaded from sequences and scalars, in addition to dictionaries. To support this possibility, you simply need to fill the class methods:
+
+ - `from_yaml_sequence` for sequences
+ - `from_yaml_scalar` for scalars

docs/usage.md

@@ -1,8 +1,10 @@
 site_name: yamlable
 # site_description: 'A short description of my project'
 repo_url: https://github.com/smarie/python-yamlable
-docs_dir: .
-site_dir: ../site
+# docs_dir: .
+# site_dir: ../site
+# default branch is main instead of master now on github
+edit_uri : ./edit/main/docs
 nav:
     - Home: index.md
     - Usage details: usage.md

docs/mkdocs.yml mkdocs.ymldocs/mkdocs.yml renamed to mkdocs.yml

@@ -1,6 +1,5 @@
 from itertools import product
 from json import dumps
-
 import logging
 
 import nox  # noqa
@@ -140,10 +139,10 @@ def flake8(session: PowerSession):
     """Launch flake8 qualimetry."""
 
     session.install("-r", str(Folders.ci_tools / "flake8-requirements.txt"))
-    session.install("genbadge[flake8]")
-    session.run2("pip install -e .[flake8]")
+    session.run2("pip install .")
 
     rm_folder(Folders.flake8_reports)
+    Folders.flake8_reports.mkdir(parents=True, exist_ok=True)
     rm_file(Folders.flake8_intermediate_file)
 
     # Options are set in `setup.cfg` file

noxfile.py

@@ -6,6 +6,8 @@
 from abc import ABCMeta
 from collections import OrderedDict
 
+from yaml import ScalarNode, SequenceNode, MappingNode
+
 try:
     # Python 2 only:
     from StringIO import StringIO as _StringIO  # type: ignore  # noqa
@@ -27,7 +29,7 @@ def __exit__(self, exception_type, exception_value, traceback):
 import six
 
 try:  # python 3.5+
-    from typing import Union, TypeVar, Dict, Any
+    from typing import Union, TypeVar, Dict, Any, Sequence
 
     Y = TypeVar('Y', bound='AbstractYamlObject')
 
@@ -49,6 +51,30 @@ class AbstractYamlObject(six.with_metaclass(ABCMeta, object)):
     Default implementation uses vars(self) and cls(**dct), but subclasses can override.
     """
 
+    # def __to_yaml_scalar__(self):
+    #     # type: (...) -> Any
+    #     """
+    #     Implementors should transform the object into a scalar containing all information necessary to decode the
+    #     object as a YAML scalar in the future.
+    #
+    #     Default implementation raises an error.
+    #     :return:
+    #     """
+    #     raise NotImplementedError("Please override `__to_yaml_scalar__` if you wish to dump instances of `%s`"
+    #                               " as yaml scalars." % type(self).__name__)
+    #
+    # def __to_yaml_sequence__(self):
+    #     # type: (...) -> Sequence[Any]
+    #     """
+    #     Implementors should transform the object into a Sequence containing all information necessary to decode the
+    #     object as a YAML sequence in the future.
+    #
+    #     Default implementation raises an error.
+    #     :return:
+    #     """
+    #     raise NotImplementedError("Please override `__to_yaml_sequence__` if you wish to dump instances of `%s`"
+    #                               " as yaml sequences." % type(self).__name__)
+
     def __to_yaml_dict__(self):
         # type: (...) -> Dict[str, Any]
         """
@@ -67,6 +93,52 @@ def __to_yaml_dict__(self):
         # Default: return vars(self) (Note: no need to make a copy, pyyaml does not modify it)
         return vars(self)
 
+    @classmethod
+    def __from_yaml_scalar__(cls,      # type: Type[Y]
+                             scalar,   # type: Any
+                             yaml_tag  # type: str
+                             ):
+        # type: (...) -> Y
+        """
+        Implementors should transform the given scalar (read from yaml by the pyYaml stack) into an object instance.
+        The yaml tag associated to this object, read in the yaml document, is provided in parameter.
+
+        Note that for YamlAble and YamlObject2 subclasses, if this method is called the yaml tag will already have
+        been checked so implementors do not have to validate it.
+
+        Default implementation returns cls(scalar)
+
+        :param scalar: the yaml scalar
+        :param yaml_tag: the yaml schema id that was used for encoding the object (it has already been checked
+            against is_json_schema_id_supported)
+        :return:
+        """
+        # Default: call constructor with positional arguments
+        return cls(scalar)  # type: ignore
+
+    @classmethod
+    def __from_yaml_sequence__(cls,      # type: Type[Y]
+                               seq,      # type: Sequence[Any]
+                               yaml_tag  # type: str
+                               ):
+        # type: (...) -> Y
+        """
+        Implementors should transform the given Sequence (read from yaml by the pyYaml stack) into an object instance.
+        The yaml tag associated to this object, read in the yaml document, is provided in parameter.
+
+        Note that for YamlAble and YamlObject2 subclasses, if this method is called the yaml tag will already have
+        been checked so implementors do not have to validate it.
+
+        Default implementation returns cls(*seq)
+
+        :param seq: the yaml sequence
+        :param yaml_tag: the yaml schema id that was used for encoding the object (it has already been checked
+            against is_json_schema_id_supported)
+        :return:
+        """
+        # Default: call constructor with positional arguments
+        return cls(*seq)  # type: ignore
+
     @classmethod
     def __from_yaml_dict__(cls,      # type: Type[Y]
                            dct,      # type: Dict[str, Any]
@@ -200,14 +272,72 @@ def load_yaml(cls,                  # type: Type[Y]
 
 
 def read_yaml_node_as_dict(loader, node):
+    # type: (...) -> OrderedDict
     """
     Utility method to read a yaml node into a dictionary
 
     :param loader:
     :param node:
     :return:
     """
-    loader.flatten_mapping(node)
-    pairs = loader.construct_pairs(node, deep=True)  # 'deep' allows the construction to be complete (inner seq...)
+    # loader.flatten_mapping(node)
+    # pairs = loader.construct_pairs(node, deep=True)  # 'deep' allows the construction to be complete (inner seq...)
+    pairs = loader.construct_mapping(node, deep=True)  # 'deep' allows the construction to be complete (inner seq...)
     constructor_args = OrderedDict(pairs)
     return constructor_args
+
+
+def read_yaml_node_as_sequence(loader, node):
+    # type: (...) -> Sequence
+    """
+    Utility method to read a yaml node into a sequence
+
+    :param loader:
+    :param node:
+    :return:
+    """
+    seq = loader.construct_sequence(node, deep=True)  # 'deep' allows the construction to be complete (inner seq...)
+    return seq
+
+
+def read_yaml_node_as_scalar(loader, node):
+    # type: (...) -> Any
+    """
+    Utility method to read a yaml node into a sequence
+
+    :param loader:
+    :param node:
+    :return:
+    """
+    value = loader.construct_scalar(node)
+    return value
+
+
+def read_yaml_node_as_yamlobject(
+    cls,      # type: Type[AbstractYamlObject]
+    loader,
+    node,     # type: MappingNode
+    yaml_tag  # type: str
+):
+    # type: (...) -> AbstractYamlObject
+    """
+    Default implementation: loads the node as a dictionary and calls __from_yaml_dict__ with this dictionary
+
+    :param loader:
+    :param node:
+    :return:
+    """
+    if isinstance(node, ScalarNode):
+        constructor_args = read_yaml_node_as_scalar(loader, node)
+        return cls.__from_yaml_scalar__(constructor_args, yaml_tag=yaml_tag)  # type: ignore
+
+    elif isinstance(node, SequenceNode):
+        constructor_args = read_yaml_node_as_sequence(loader, node)
+        return cls.__from_yaml_sequence__(constructor_args, yaml_tag=yaml_tag)  # type: ignore
+
+    elif isinstance(node, MappingNode):
+        constructor_args = read_yaml_node_as_dict(loader, node)
+        return cls.__from_yaml_dict__(constructor_args, yaml_tag=yaml_tag)  # type: ignore
+
+    else:
+        raise TypeError("Unknown type of yaml node: %r. Please report this to `yamlable` project." % type(node))