update README.md and rand docstring

Author: rfourquet

.gitignore

@@ -1,3 +1,5 @@
 *.jl.cov
 *.jl.*.cov
 *.jl.mem
+.*~
+README.html

README.md

@@ -7,40 +7,51 @@
 This package explores a possible extension of `rand`-related
 functionalities (from the `Random` module); the code is initially
 taken from https://github.com/JuliaLang/julia/pull/24912.
-Note that type piracy is commited!
+Note that type piracy is committed!
 While hopefully useful, this package is still experimental, and
-hence unstable. Design or implementation contributions are welcome.
+hence unstable. User feedback, and design or implementation contributions are welcome.
 
-This does mainly 3 things:
+This does essentially 4 things:
 
 1) define distribution objects, to give first-class status to features
    provided by `Random`; for example `rand(Normal(), 3)` is equivalent
    to `randn(3)`; other available distributions: `Exponential`,
-   `CloseOpen` (for generation of floats in a close-open range),
-   `Uniform` (which can wrap an implicit uniform distribution),
-   `make` (to combine distribution for objects made of multiple
-   scalars, like `Pair`, `Tuple`, or `Complex`, or for containers);
+   `CloseOpen` (for generation of floats in a close-open range) and friends,
+   `Uniform` (which can wrap an implicit uniform distribution);
 
-2) define generation of some containers filled with random values
+2) define `make` methods, which can combine distributions for objects made of multiple scalars, like
+   `Pair`, `Tuple`, or `Complex`, or describe how to generate more complex objects, like containers;
+
+3) extend the `rand([rng], [S], dims)` API to allow the generation of other containers than arrays
    (like `Set`, `Dict`, `SparseArray`, `String`, `BitArray`);
 
-3) define a `Rand` iterator, which produces lazily random values.
+4) define a `Rand` iterator, which produces lazily random values.
 
 
 Point 1) defines a `Distribution` type which is incompatible with the
 "Distributions.jl" package. Input on how to unify the two approaches is
 welcome.
-Point 2) goes somewhat against the trend in `Base` to create
-containers using their constructors -- which by the way may be
-achieved with the `Rand` iterator from point 3).
-Still, I like the terser approach here, as it simply generalizes
-to other containers the __current__ `rand` API creating arrays.
-See the issue linked above for a discussion on those topics.
-
-For convenience, the following objects from `Random` are re-exported
+
+Point 2) is really the core of this package. `make` provides a vocabulary to define the generation
+of "scalars" which require more than one argument to be described, e.g. pairs from `1:3` to `Int`
+(`rand(make(Pair, 1:3, Int))`) or regular containers (e.g. `make(Array, 2, 3)`). The point of
+calling `make` rather than putting all the arguments in `rand` directly is simplicity and
+composability: the `make` call always occurs as the second argument to `rand` (or first if the RNG
+is omitted). For example, `rand(make(Array, 2, 3), 3)` creates an array of matrices.
+Of course, `make` is not necessary, in that the same can be achieved with an ad hoc `struct`,
+which in some cases is clearer (e.g. `Normal(m, s)` rather than something like `make(Float64, Val(:Normal), m, s)`).
+
+Point 3) allows something like `rand(1:30, Set, 10)` to produce a `Set` of length `10` with values
+from `1:30`. The idea is that `rand([rng], [S], Cont, etc...)` should always be equivalent to
+`rand([rng], make(Cont, [S], etc...))`. This design goes somewhat against the trend in `Base` to create
+containers using their constructors -- which by the way may be achieved via the `Rand` iterator from
+point 4). Still, I like the terse approach here, as it simply generalizes to other containers the
+_current_ `rand` API creating arrays. See the issue linked above for a discussion on these topics.
+
+For convenience, the following names from `Random` are re-exported
 in this package: `rand!`, `AbstractRNG`, `MersenneTwister`,
 `RandomDevice` (`rand` is in `Base`). Functions like `randn!` or
-`bitrand` are considered to be obsoleted by this package so are not
+`randstring` are considered to be obsoleted by this package so are not
 re-exported. It's still needed to import `Random` separately in order
 to use functions which don't extend the `rand` API, namely
 `randsubseq`, `shuffle`, `randperm`, `randcycle`, and their mutating
@@ -51,11 +62,14 @@ There is not much documentation for now: `rand`'s docstring is updated,
 and here are some examples:
 
 ```julia
-julia> rand(CloseOpen()) # like rand(Float64)
+julia> rand(CloseOpen(Float64)) # equivalent to rand(Float64)
 0.7678877639669386
 
-julia> rand(CloseOpen(1.0, 10.0)) # generation in [1.0, 10.0)
-4.309057677479184
+julia> rand(CloseClose(1.0f0, 10)) # generation in [1.0f0, 10.0f0]
+6.62467f0
+
+julia> rand(OpenOpen(2.0^52, 2.0^52+1)) == 2.0^52 # exactness not guaranteed for "unreasonable" values!
+true
 
 julia> rand(Normal(0.0, 10.0)) # explicit μ and σ parameters
 -8.473790458128912
@@ -66,21 +80,24 @@ julia> rand(Uniform(1:3)) # equivalent to rand(1:3)
 julia> rand(make(Pair, 1:10, Normal())) # random Pair, where both members have distinct distributions
 5 => 0.674375
 
-julia> rand(make(Pair{Number, Any}, 1:10, Normal())) # specify the Pair type
+julia> rand(make(Pair{Number,Any}, 1:10, Normal())) # specify the Pair type
 Pair{Number,Any}(1, -0.131617)
 
 julia> rand(Pair{Float64,Int}) # equivalent to rand(make(Pair, Float64, Int))
 0.321676 => -4583276276690463733
 
-julia> rand(make(Tuple, 1:10, Normal()))
-(9, 1.3407309364427373)
+julia> rand(make(Tuple, 1:10, UInt8, OpenClose()))
+(9, 0x6b, 0.34900083923775505)
 
 julia> rand(Tuple{Float64,Int}) # equivalent to rand(make(Tuple, Float64, Int))
 (0.9830769470405203, -6048436354564488035)
 
 julia> rand(make(NTuple{3}, 1:10)) # produces a 3-tuple with values from 1:10
 (5, 9, 6)
 
+julia> rand(make(NTuple{N,UInt8} where N, 1:3, 5))
+(0x02, 0x03, 0x02, 0x03, 0x02)
+
 julia> rand(make(NTuple{3}, make(Pair, 1:9, Bool))) # make calls can be nested
 (2 => false, 8 => true, 7 => false)
 
@@ -96,39 +113,52 @@ julia> rand(Normal(ComplexF64)) # equivalent to randn(ComplexF64)
 julia> rand(Set, 3)
 Set([0.717172, 0.78481, 0.86901])
 
-julia> rand(1:9, Set, 3)
+julia>  rand!(ans, Exponential())
+Set([0.7935073925105659, 2.593684878770254, 1.629181233597078])
+
+julia> rand(1:9, Set, 3) # if you try `rand(1:3, Set, 9)`, it will take a while ;-)
 Set([3, 5, 8])
 
+julia> rand(Dict{String,Int8}, 2)
+Dict{String,Int8} with 3 entries:
+  "vxybIbae" => 42
+  "bO2fTwuq" => -13
+
 julia> rand(make(Pair, 1:9, Normal()), Dict, 3)
 Dict{Int64,Float64} with 3 entries:
   9 => 0.916406
   3 => -2.44958
   8 => -0.703348
 
-julia> rand(0.3, 9) # equivalent to sprand(9, 0.3)
+julia> rand(SparseVector, 0.3, 9) # equivalent to sprand(9, 0.3)
 9-element SparseVector{Float64,Int64} with 3 stored entries:
   [1]  =  0.173858
   [6]  =  0.568631
   [8]  =  0.297207
 
-julia> rand(Normal(), 0.3, 2, 3) # equivalent to sprandn(2, 3, 0.3)
+julia> rand(Normal(), SparseMatrixCSC, 0.3, 2, 3) # equivalent to sprandn(2, 3, 0.3)
 2×3 SparseMatrixCSC{Float64,Int64} with 2 stored entries:
   [2, 1]  =  0.448981
   [1, 2]  =  0.730103
 
+# like for Array, sparse arrays enjoy to be special cased: `SparseVector` or `SparseMatrixCSC` can be omitted:
+
+julia> rand(make(make(1:9, 0.3, 2, 3), 0.1, 4)) # possible, bug ugly output when non-empty :-/
+4-element SparseVector{SparseMatrixCSC{Int64,Int64},Int64} with 0 stored entries
+
 julia> rand(String, 4) # equivalent to randstring(4)
 "5o75"
 
-julia> rand("123", String, 4) # String considered as a container
+julia> rand("123", String, 4) # like above, String creation with the "container" syntax ...
 "2131"
 
+julia> rand(make(String, 3, "123")) # ... which is as always equivalent to a call to make
+"211"
+
 julia> rand(String, Set, 3) # String considered as a scalar
 Set(["0Dfqj6Yr", "ILngfcRz", "HT5IEyK3"])
 
-julia> rand(make(String, 3, "123"))
-"211"
-
-julia> rand(BitArray, 3) # equivalent to bitrand(3)
+julia> rand(BitArray, 3) # equivalent to, but unfortunately more verbose than, bitrand(3)
 3-element BitArray{1}:
   true
   true
@@ -150,8 +180,20 @@ julia> julia> rand(Bernoulli(0.2), BitVector, 10) # using the Bernoulli distribu
 julia> rand(1:3, NTuple{3}) # NTuple{3} considered as a container, equivalent to rand(make(NTuple{3}, 1:3))
 (3, 3, 1)
 
+julia> rand(1:3, Tuple{Int,UInt8, BigFloat}) # works also with more general tuple types
+(3, 0x02, 2.0)
+
+julia> RandomExtensions.random_staticarrays() # poor man's conditional modules!
+# ugly warning
+
+julia> rand(make(MVector{2,AbstractString}, String), SMatrix{3, 2})
+3×2 SArray{Tuple{3,2},MArray{Tuple{2},AbstractString,1,2},2,6} with indices SOneTo(3)×SOneTo(2):
+ ["SzPKXHFk", "1eFXaUiM"]  ["RJnHwhb7", "jqfLcY8a"]
+ ["FMTKcBY8", "eoYtNntD"]  ["FzdD530L", "ux6sWGMU"]
+ ["fFJuUtJQ", "H2mAQrIV"]  ["pt0OYFJw", "O0fCfjjR"]
+
 julia> Set(Iterators.take(Rand(RandomDevice(), 1:10), 3)) # RNG defaults to Random.GLOBAL_RNG
-Set([9, 2, 6])
+Set([9, 2, 6]) # note that the set could end up with less than 3 elements if `Rand` generates duplicates
 
 julia> collect(Iterators.take(Uniform(1:10), 3)) # distributions can be iterated over, using Random.GLOBAL_RNG implicitly
 3-element Array{Int64,1}:
@@ -160,10 +202,10 @@ julia> collect(Iterators.take(Uniform(1:10), 3)) # distributions can be iterated
   5
 ```
 
-In some cases, the `Rand` iterator can provide some efficiency gains compared to
-repeated calls to `rand`, as it uses the same mechanism as non-scalar generation.
-For example, given `a = zeros(10000)`,
-`a .+ Rand(1:1000).()` will be faster than `a .+ rand.(Ref(1:1000))`.
+In some cases, the `Rand` iterator can provide efficiency gains compared to
+repeated calls to `rand`, as it uses the same mechanism as array generation.
+For example, given `a = zeros(1000)` and `s = BitSet(1:1000)`,
+`a .+ Rand(s).()` is three times faster than `a .+ rand.(Ref(s))`.
 
 Note: as seen in the examples above, `String` can be considered as a scalar or as a container (in the `rand` API).
 In a call like `rand(String)`, both APIs coincide, but in `rand(String, 3)`, should we construct a `String` of
@@ -173,3 +215,19 @@ most useful (and offers the tersest API to compete with `randstring`).
 But as this package is still unstable, this choice may be revisited in the future.
 Note that it's easy to get the result of the second interpretation via either `rand(make(String), 3)`,
 `rand(String, (3,))` or `rand(String, Vector, 3)`.
+
+How to extend: the `make` function is meant to be extensible, and there are some helper functions
+which make it easy, but the internals are not fully settled. By default, `make(T, args...)` will
+create a `Make{find_type(T, args...)}` object, say `m`, which contain `args...` as fields. For type
+stable code, the `rand` machinery likes to know the exact type of the object which will be generated by
+`rand(m)`, and `find_type(T, args...)` is supposed to return that type. For example,
+`find_type(Pair, 1:3, UInt) == Pair{Int,UInt}`.
+Then just define `rand` for `m` like documented in the `Random` module, e.g.
+`rand(rng::AbstractRNG, sp::SamplerTrivial{<:Make{P}}) where {P<:Pair} = P(rand(sp[].x), rand(sp[].y))`.
+
+This package started out of frustration with the limitations of the `Random` module. Besides
+generating simple scalars and arrays, very little is supported out of the box. For example,
+generating a random `Dict` is too complex. Moreover, there are too many functions for my taste:
+`rand`, `randn`, `randexp`, `sprand` (with its exotic `rfn` parameter), `sprandn`, ~~`sprandexp`~~,
+`randstring`, `bitrand`, and mutating counterparts (but I believe `randn` will never go away, as
+it's so terse). I hope that this package can serve as a starting point towards improving `Random`.

src/RandomExtensions.jl

@@ -63,18 +63,18 @@ Pick a random element or collection of random elements from the set of values sp
 
 `S` usually defaults to [`Float64`](@ref).
 
-If `C...` is not specified, `rand` produces a scalar. Otherwise, `C...` can be:
+If `C...` is not specified, `rand` produces a scalar. Otherwise, `C` can be:
 
 * a set of integers, or a tuple of `Int`, which specify the dimensions of an `Array` to generate;
-* `(Array, dims...)`: same as above, but with `Array` specified explicitely
-* `(p::AbstractFloat, m::Integer, [n::Integer])...`, which produces a sparse array of dimensions `(m, n)`,
-  in which the probability of any element being nonzero is independently given by `p`
-* `(String, [n=8])...`, which produces a random `String` of length `n`; the generated string consists of `Char`
-  taken from a predefined set like `randstring`, and can be specified with the `S` parameter.
-* `(Dict, n)...`, which produces a `Dict` of length `n`; `S` must then specify the type of its elements,
-  e.g. `make(Pair, Int, 2:3)`;
-* `(Set, n)...`, which produces a `Set` of length `n`;
-* `(BitArray, dims...)...`, which produces a `BitArray` with the specified dimensions.
+* `(Array, dims...)`: same as above, but with `Array` specified explicitely;
+* `(p::AbstractFloat, m::Integer, [n::Integer])`, which produces a sparse array of dimensions `(m, n)`,
+  in which the probability of any element being nonzero is independently given by `p`;
+* `(String, [n=8])`, which produces a random `String` of length `n`; the generated string consists of `Char`
+  taken from a predefined set like `randstring`, and can be specified with the `S` parameter;
+* `(Dict, n)`, which produces a `Dict` of length `n`; if `Dict` is given without type parameters,
+  then `S` must be specified;
+* `(Set, n)` or `(BitSet, n)`, which produces a set of length `n`;
+* `(BitArray, dims...)`, which produces a `BitArray` with the specified dimensions.
 
 For `Array`, `Dict` and `Set`, a less abstract type can be specified, e.g. `Set{Float64}`, to force
 the type of the result regardless of the `S` parameter. In particular, in the absence of `S`, the