Add --null-threshold to the docs

Closes: #20

Author: waveform80

docs/manual.rst

@@ -15,7 +15,7 @@ Synopsis
 
     structa [-h] [--version] [-f {auto,csv,json,yaml}] [-e ENCODING]
             [--encoding-strict] [--no-encoding-strict]
-            [-F INT] [-M NUM] [-B NUM] [-E NUM] [--str-limit NUM]
+            [-F INT] [-M NUM] [-B NUM] [-E NUM] [-N NUM] [--str-limit NUM]
             [--hide-count] [--show-count] [--hide-lengths] [--show-lengths]
             [--hide-pattern] [--show-pattern]
             [--hide-range] [--show-range {hidden,limits,median,quartiles,graph}]
@@ -91,6 +91,11 @@ Optional Arguments
     the pattern from being reported; the proportion of "empty" data permitted
     in a field (default: 99%)
 
+.. option:: -N NUM, --null-threshold NUM
+
+    The proportion of values permitted to be null without preventing type
+    analysis (default: 99%)
+
 .. option:: --str-limit NUM
 
     The length beyond which only the lengths of strs will be reported; below

docs/tutorial_basic.rst

@@ -154,14 +154,15 @@ bad threshold mechanism only applies to bad data *within* a homogenous type
 (typically bad string representations of numeric or boolean types).
 
 
-Missing Data (``--empty-threshold``)
-====================================
+Missing Data (``--empty-threshold`` and ``--null-threshold``)
+=============================================================
 
-Another type of "bad" data commonly encountered is empty strings which are
-typically used to represent *missing* data, and (predictably) structa has
-another knob that can be twiddled for this: :option:`structa
---empty-threshold`. The following script generates a list of strings of
-integers in which most of the strings (~70%) are blank:
+Another type of "bad" data commonly encountered is empty strings and nulls
+which are typically used to represent *missing* data, and (predictably) structa
+has more knobs that can be twiddled for this: :option:`structa
+--empty-threshold` and :option:`structa --null-threshold`. The following script
+generates a list of strings of integers in which most of the strings (~70%) are
+blank:
 
 .. literalinclude:: examples/mostly-blank.py
    :caption: mostly-blank.py
@@ -174,11 +175,13 @@ normal:
     $ python3 mostly-blank.py | structa
     [ str of int range=0..100 pattern="d" ]
 
-This is because the default for :option:`structa --empty-threshold` is 99% or
-0.99. If the proportion of blank strings in a field exceeds the empty
-threshold, the field will simply be marked as a string without any further
-processing. Hence, when we re-run this script with the setting turned down to
-50%, the output changes:
+This is because the default for both :option:`structa --empty-threshold` and
+:option:`structa --null-threshold` is 99% or 0.99.
+
+If the proportion of blank strings in a field exceeds the empty threshold, the
+field will simply be marked as a string without any further processing. Hence,
+when we re-run this script with the setting turned down to 50%, the output
+changes:
 
 .. code-block:: console
 
@@ -191,6 +194,11 @@ processing. Hence, when we re-run this script with the setting turned down to
     "100" value, but because it's now considered a string (not a string of
     integers), "100" sorts before "99" alphabetically.
 
+Likewise, if the proportion of null values in a field exceeds the null
+threshold, the field will simply be marked as "value" (an arbitrary mix of
+types), because structa assumes there aren't enough values to accurately
+represent the type of the field.
+
 It is also worth nothing that, by default, structa strips whitespace from
 strings prior to analysis. This is probably not necessary for the vast majority
 of modern datasets, but it's a reasonably safe default, and can be controlled