diff --git a/dev/distributions/index.html b/dev/distributions/index.html index acda63a4..d2f77e58 100644 --- a/dev/distributions/index.html +++ b/dev/distributions/index.html @@ -20,4 +20,4 @@ dist: Beta{Float64}(α=2.0, β=2.0) transform: Bijectors.Logit{Float64, Float64}(0.0, 1.0) )
julia> tdist isa UnivariateDistributiontrue

We can the then compute the logpdf for the resulting distribution:

julia> # Some example values
-       x = rand(dist)0.5399772085282485
julia> y = tdist.transform(x)0.1602508973049689
julia> logpdf(tdist, y)-0.9936557123566945
+ x = rand(dist)0.4232850890988823
julia> y = tdist.transform(x)-0.30930213431480513
julia> logpdf(tdist, y)-1.0284736919439514 diff --git a/dev/examples/index.html b/dev/examples/index.html index 42d93e3b..75e9340b 100644 --- a/dev/examples/index.html +++ b/dev/examples/index.html @@ -25,36 +25,36 @@ # Construct the transform
julia> bs = bijector.(dists); # constrained-to-unconstrained bijectors for dists
julia> ibs = inverse.(bs); # invert, so we get unconstrained-to-constrained
julia> sb = Stacked(ibs, ranges) # => Stacked <: Bijector # Mean-field normal with unconstrained-to-constrained stacked bijectorStacked(Any[Inverse{Bijectors.Logit{Float64, Float64}}(Bijectors.Logit{Float64, Float64}(0.0, 1.0)), Base.Fix1{typeof(broadcast), typeof(exp)}(broadcast, exp), Inverse{Bijectors.SimplexBijector}(Bijectors.SimplexBijector())], Any[1:1, 2:2, 3:4], Any[1:1, 2:2, 3:5])
julia> td = transformed(d, sb);
julia> y = rand(td)5-element Vector{Float64}: - 0.6163722476466892 - 9.373289960405257 - 0.4208112321889897 - 0.37191903747263666 - 0.2072697303383737
julia> 0.0 ≤ y[1] ≤ 1.0true
julia> 0.0 < y[2]true
julia> sum(y[3:4]) ≈ 1.0false

Normalizing flows

A very interesting application is that of normalizing flows.[1] Usually this is done by sampling from a multivariate normal distribution, and then transforming this to a target distribution using invertible neural networks. Currently there are two such transforms available in Bijectors.jl: PlanarLayer and RadialLayer. Let's create a flow with a single PlanarLayer:

julia> d = MvNormal(zeros(2), ones(2));
julia> b = PlanarLayer(2)PlanarLayer(w = [-0.5844906356280568, 0.8536896958372493], u = [1.8597450283737451, 1.707621659026127], b = [-0.7357339609843876])
julia> flow = transformed(d, b)MultivariateTransformed{DiagNormal, PlanarLayer{Vector{Float64}, Vector{Float64}}}(
+ 0.19288841082079267
+ 2.2680700004576613
+ 0.4172221602490952
+ 0.12834634043085497
+ 0.4544314993200498
julia> 0.0 ≤ y[1] ≤ 1.0true
julia> 0.0 < y[2]true
julia> sum(y[3:4]) ≈ 1.0false

Normalizing flows

A very interesting application is that of normalizing flows.[1] Usually this is done by sampling from a multivariate normal distribution, and then transforming this to a target distribution using invertible neural networks. Currently there are two such transforms available in Bijectors.jl: PlanarLayer and RadialLayer. Let's create a flow with a single PlanarLayer:

julia> d = MvNormal(zeros(2), ones(2));
julia> b = PlanarLayer(2)PlanarLayer(w = [1.3166111312621989, -0.45396486179549106], u = [0.6550778733553086, -1.062997006793278], b = [-1.4444376455505716])
julia> flow = transformed(d, b)MultivariateTransformed{DiagNormal, PlanarLayer{Vector{Float64}, Vector{Float64}}}(
 dist: DiagNormal(
 dim: 2
 μ: [0.0, 0.0]
 Σ: [1.0 0.0; 0.0 1.0]
 )
 
-transform: PlanarLayer(w = [-0.5844906356280568, 0.8536896958372493], u = [1.8597450283737451, 1.707621659026127], b = [-0.7357339609843876])
+transform: PlanarLayer(w = [1.3166111312621989, -0.45396486179549106], u = [0.6550778733553086, -1.062997006793278], b = [-1.4444376455505716])
 )
julia> flow isa MultivariateDistributiontrue

That's it. Now we can sample from it using rand and compute the logpdf, like any other Distribution.

julia> y = rand(rng, flow)2-element Vector{Float64}:
- -0.5903424409371388
-  0.49722246387316493
julia> logpdf(flow, y)         # uses inverse of `b`-2.052383202362973

Similarily to the multivariate ADVI example, we could use Stacked to get a bounded flow:

julia> d = MvNormal(zeros(2), ones(2));
julia> ibs = inverse.(bijector.((InverseGamma(2, 3), Beta())));
julia> sb = stack(ibs...) # == Stacked(ibs) == Stacked(ibs, [i:i for i = 1:length(ibs)]ERROR: MethodError: no method matching length(::Inverse{Bijectors.Logit{Float64, Float64}})
+ -0.801991460032661
+  1.3191132324552892
julia> logpdf(flow, y)         # uses inverse of `b`-2.1768055950540184

Similarily to the multivariate ADVI example, we could use Stacked to get a bounded flow:

julia> d = MvNormal(zeros(2), ones(2));
julia> ibs = inverse.(bijector.((InverseGamma(2, 3), Beta())));
julia> sb = stack(ibs...) # == Stacked(ibs) == Stacked(ibs, [i:i for i = 1:length(ibs)]ERROR: MethodError: no method matching length(::Inverse{Bijectors.Logit{Float64, Float64}})
 
 Closest candidates are:
   length(!Matched::Core.Compiler.InstructionStream)
    @ Base show.jl:2777
-  length(!Matched::Distributed.WorkerPool)
-   @ Distributed /opt/hostedtoolcache/julia/1.10.2/x64/share/julia/stdlib/v1.10/Distributed/src/workerpool.jl:139
-  length(!Matched::Tables.DictRowTable)
-   @ Tables ~/.julia/packages/Tables/NSGZI/src/dicts.jl:118
+  length(!Matched::DataStructures.DiBitVector)
+   @ DataStructures ~/.julia/packages/DataStructures/95DJa/src/dibit_vector.jl:40
+  length(!Matched::Distributions.DirichletStats)
+   @ Distributions ~/.julia/packages/Distributions/UaWBm/src/multivariate/dirichlet.jl:186
   ...
julia> b = sb ∘ PlanarLayer(2)ERROR: UndefVarError: `sb` not defined
julia> td = transformed(d, b);
julia> y = rand(rng, td)2-element Vector{Float64}:
- 0.5312643077244523
- 0.7814019022450341
julia> 0 < y[1]true
julia> 0 ≤ y[2] ≤ 1true

Want to fit the flow?

julia> using Zygote
+ 1.343424613487856
+ 1.509474913938528
julia> 0 < y[1]true
julia> 0 ≤ y[2] ≤ 1false

Want to fit the flow?

julia> using Zygote
        
        # Construct the flow.
julia> b = PlanarLayer(2)
        
-       # Convenient for extracting parameters and reconstructing the flow.PlanarLayer(w = [0.08273242274438262, -0.2967247970574345], u = [1.9269144896391301, 0.16261701900716385], b = [-0.641972243513082])
julia> using Functors
julia> θs, reconstruct = Functors.functor(b);
+       # Convenient for extracting parameters and reconstructing the flow.PlanarLayer(w = [0.010660031347162877, 0.6433870123713419], u = [1.4861037078819206, 1.8085976568191868], b = [1.2072803384193682])
julia> using Functors
julia> θs, reconstruct = Functors.functor(b);
        
        # Make the objective a `struct` to avoid capturing global variables.
julia> struct NLLObjective{R,D,T}
            reconstruct::R
@@ -71,7 +71,7 @@
        
        # Initial loss.
julia> @info "Initial loss: $(f(θs...))"
        
-       # Train using gradient descent.[ Info: Initial loss: 3746.2796848582443
julia> ε = 1e-3;
julia> for i in 1:100
+       # Train using gradient descent.[ Info: Initial loss: 3476.274699061798
julia> ε = 1e-3;
julia> for i in 1:100
            ∇s = Zygote.gradient(f, θs...)
            θs = map(θs, ∇s) do θ, ∇
                θ - ε .* ∇
@@ -80,8 +80,8 @@
        
        # Final loss
julia> @info "Finall loss: $(f(θs...))"
        
-       # Very simple check to see if we learned something useful.[ Info: Finall loss: 2812.989207270324
julia> samples = rand(transformed(f.basedist, f.reconstruct(θs)), 1000);
julia> mean(eachcol(samples)) # ≈ [0, 0]2-element Vector{Float64}:
- -0.020301432266073213
- -0.007100282616286068
julia> cov(samples; dims=2)   # ≈ I2×2 Matrix{Float64}:
-  0.949914   -0.0615582
- -0.0615582   1.03093

We can easily create more complex flows by simply doing PlanarLayer(10) ∘ PlanarLayer(10) ∘ RadialLayer(10) and so on.

+ # Very simple check to see if we learned something useful.[ Info: Finall loss: 2840.3913066613277
julia> samples = rand(transformed(f.basedist, f.reconstruct(θs)), 1000);
julia> mean(eachcol(samples)) # ≈ [0, 0]2-element Vector{Float64}: + 0.017282488896742932 + -0.031428288028033914
julia> cov(samples; dims=2) # ≈ I2×2 Matrix{Float64}: + 0.97715 -0.0344529 + -0.0344529 0.937849

We can easily create more complex flows by simply doing PlanarLayer(10) ∘ PlanarLayer(10) ∘ RadialLayer(10) and so on.

diff --git a/dev/index.html b/dev/index.html index 3d858707..b9130a58 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Home · Bijectors
Edit on GitHub

Bijectors.jl

This package implements a set of functions for transforming constrained random variables (e.g. simplexes, intervals) to Euclidean space. The 3 main functions implemented in this package are the link, invlink and logpdf_with_trans for a number of distributions. The distributions supported are:

  1. RealDistribution: Union{Cauchy, Gumbel, Laplace, Logistic, NoncentralT, Normal, NormalCanon, TDist},
  2. PositiveDistribution: Union{BetaPrime, Chi, Chisq, Erlang, Exponential, FDist, Frechet, Gamma, InverseGamma, InverseGaussian, Kolmogorov, LogNormal, NoncentralChisq, NoncentralF, Rayleigh, Weibull},
  3. UnitDistribution: Union{Beta, KSOneSided, NoncentralBeta},
  4. SimplexDistribution: Union{Dirichlet},
  5. PDMatDistribution: Union{InverseWishart, Wishart}, and
  6. TransformDistribution: Union{T, Truncated{T}} where T<:ContinuousUnivariateDistribution.

All exported names from the Distributions.jl package are reexported from Bijectors.

Bijectors.jl also provides a nice interface for working with these maps: composition, inversion, etc. The following table lists mathematical operations for a bijector and the corresponding code in Bijectors.jl.

OperationMethodAutomatic
b ↦ b⁻¹inverse(b)✓
(b₁, b₂) ↦ (b₁ ∘ b₂)b₁ ∘ b₂✓
(b₁, b₂) ↦ [b₁, b₂]stack(b₁, b₂)✓
x ↦ b(x)b(x)×
y ↦ b⁻¹(y)inverse(b)(y)×
x ↦ log｜det J(b, x)｜logabsdetjac(b, x)AD
x ↦ b(x), log｜det J(b, x)｜with_logabsdet_jacobian(b, x)✓
p ↦ q := b_* pq = transformed(p, b)✓
y ∼ qy = rand(q)✓
p ↦ b such that support(b_* p) = ℝᵈbijector(p)✓
(x ∼ p, b(x), log｜det J(b, x)｜, log q(y))forward(q)✓

In this table, b denotes a Bijector, J(b, x) denotes the Jacobian of b evaluated at x, b_* denotes the push-forward of p by b, and x ∼ p denotes x sampled from the distribution with density p.

The "Automatic" column in the table refers to whether or not you are required to implement the feature for a custom Bijector. "AD" refers to the fact that it can be implemented "automatically" using automatic differentiation.

+Home · Bijectors
Edit on GitHub

Bijectors.jl

This package implements a set of functions for transforming constrained random variables (e.g. simplexes, intervals) to Euclidean space. The 3 main functions implemented in this package are the link, invlink and logpdf_with_trans for a number of distributions. The distributions supported are:

  1. RealDistribution: Union{Cauchy, Gumbel, Laplace, Logistic, NoncentralT, Normal, NormalCanon, TDist},
  2. PositiveDistribution: Union{BetaPrime, Chi, Chisq, Erlang, Exponential, FDist, Frechet, Gamma, InverseGamma, InverseGaussian, Kolmogorov, LogNormal, NoncentralChisq, NoncentralF, Rayleigh, Weibull},
  3. UnitDistribution: Union{Beta, KSOneSided, NoncentralBeta},
  4. SimplexDistribution: Union{Dirichlet},
  5. PDMatDistribution: Union{InverseWishart, Wishart}, and
  6. TransformDistribution: Union{T, Truncated{T}} where T<:ContinuousUnivariateDistribution.

All exported names from the Distributions.jl package are reexported from Bijectors.

Bijectors.jl also provides a nice interface for working with these maps: composition, inversion, etc. The following table lists mathematical operations for a bijector and the corresponding code in Bijectors.jl.

OperationMethodAutomatic
b ↦ b⁻¹inverse(b)✓
(b₁, b₂) ↦ (b₁ ∘ b₂)b₁ ∘ b₂✓
(b₁, b₂) ↦ [b₁, b₂]stack(b₁, b₂)✓
x ↦ b(x)b(x)×
y ↦ b⁻¹(y)inverse(b)(y)×
x ↦ log｜det J(b, x)｜logabsdetjac(b, x)AD
x ↦ b(x), log｜det J(b, x)｜with_logabsdet_jacobian(b, x)✓
p ↦ q := b_* pq = transformed(p, b)✓
y ∼ qy = rand(q)✓
p ↦ b such that support(b_* p) = ℝᵈbijector(p)✓
(x ∼ p, b(x), log｜det J(b, x)｜, log q(y))forward(q)✓

In this table, b denotes a Bijector, J(b, x) denotes the Jacobian of b evaluated at x, b_* denotes the push-forward of p by b, and x ∼ p denotes x sampled from the distribution with density p.

The "Automatic" column in the table refers to whether or not you are required to implement the feature for a custom Bijector. "AD" refers to the fact that it can be implemented "automatically" using automatic differentiation.

diff --git a/dev/search/index.html b/dev/search/index.html index b018e18c..bccecbcb 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,2 @@ -Search · Bijectors

Loading search...

    +Search · Bijectors

    Loading search...

      diff --git a/dev/transforms/index.html b/dev/transforms/index.html index 350f196a..83d1363f 100644 --- a/dev/transforms/index.html +++ b/dev/transforms/index.html @@ -5,10 +5,10 @@ 2.71828 2.71828 2.71828 2.71828
      julia> logabsdetjac(elementwise(exp), x)4.0
      julia> with_logabsdet_jacobian(elementwise(exp), x)([2.718281828459045 2.718281828459045; 2.718281828459045 2.718281828459045], 4.0)

      These methods also work nicely for compositions of transformations:

      julia> transform(elementwise(log ∘ exp), x)2×2 Matrix{Float64}:
  1.0  1.0
- 1.0  1.0

      Unlike exp, some transformations have parameters affecting the resulting transformation they represent, e.g. Logit has two parameters a and b representing the lower- and upper-bound, respectively, of its domain:

      julia> using Bijectors: Logit
      julia> f = Logit(0.0, 1.0)Bijectors.Logit{Float64, Float64}(0.0, 1.0)
      julia> f(rand()) # takes us from `(0, 1)` to `(-∞, ∞)`0.5592372239915828

      User-facing methods

      Without mutation:

      Bijectors.transform — Function
      transform(b, x)

      Transform x using b, treating x as a single input.

      source
      Bijectors.logabsdetjac — Function
      logabsdetjac(b, x)

      Return log(abs(det(J(b, x)))), where J(b, x) is the jacobian of b at x.

      source
      with_logabsdet_jacobian

      With mutation:

      Bijectors.transform! — Function
      transform!(b, x[, y])

      Transform x using b, storing the result in y.

      If y is not provided, x is used as the output.

      source
      Bijectors.logabsdetjac! — Function
      logabsdetjac!(b, x[, logjac])

      Compute log(abs(det(J(b, x)))) and store the result in logjac, where J(b, x) is the jacobian of b at x.

      source
      Bijectors.with_logabsdet_jacobian! — Function
      with_logabsdet_jacobian!(b, x[, y, logjac])

      Compute transform(b, x) and logabsdetjac(b, x), storing the result in y and logjac, respetively.

      If y is not provided, then x will be used in its place.

      Defaults to calling with_logabsdet_jacobian(b, x) and updating y and logjac with the result.

      source

      Implementing a transformation

      Any callable can be made into a bijector by providing an implementation of ChangeOfVariables.with_logabsdet_jacobian(b, x).

      You can also optionally implement transform and logabsdetjac to avoid redundant computations. This is usually only worth it if you expect transform or logabsdetjac to be used heavily without the other.

      Similarly with the mutable versions with_logabsdet_jacobian!, transform!, and logabsdetjac!.

      Working with Distributions.jl

      Bijectors.bijector — Function
      bijector(d::Distribution)

      Returns the constrained-to-unconstrained bijector for distribution d.

      source
      Bijectors.transformed — Method
      transformed(d::Distribution)
-transformed(d::Distribution, b::Bijector)

      Couples distribution d with the bijector b by returning a TransformedDistribution.

      If no bijector is provided, i.e. transformed(d) is called, then transformed(d, bijector(d)) is returned.

      source

      Utilities

      Bijectors.elementwise — Function
      elementwise(f)

      Alias for Base.Fix1(broadcast, f).

      In the case where f::ComposedFunction, the result is Base.Fix1(broadcast, f.outer) ∘ Base.Fix1(broadcast, f.inner) rather than Base.Fix1(broadcast, f).

      source
      Bijectors.isinvertible — Function
      isinvertible(t)

      Return true if t is invertible, and false otherwise.

      source
      Bijectors.isclosedform — Method
      isclosedform(b::Transform)::bool
-isclosedform(b⁻¹::Inverse{<:Transform})::bool

      Returns true or false depending on whether or not evaluation of b has a closed-form implementation.

      Most transformations have closed-form evaluations, but there are cases where this is not the case. For example the inverse evaluation of PlanarLayer requires an iterative procedure to evaluate.

      source

      API

      Bijectors.Transform — Type

      Abstract type for a transformation.

      Implementing

      A subtype of Transform of should at least implement transform(b, x).

      If the Transform is also invertible:

      • Required:
        • Either of the following:
          • transform(::Inverse{<:MyTransform}, x): the transform for its inverse.
          • InverseFunctions.inverse(b::MyTransform): returns an existing Transform.
        • logabsdetjac: computes the log-abs-det jacobian factor.
      • Optional:
        • with_logabsdet_jacobian: transform and logabsdetjac combined. Useful in cases where we can exploit shared computation in the two.

      For the above methods, there are mutating versions which can optionally be implemented:

      source
      Bijectors.Bijector — Type

      Abstract type of a bijector, i.e. differentiable bijection with differentiable inverse.

      source
      Bijectors.Inverse — Type
      inverse(b::Transform)
-Inverse(b::Transform)

      A Transform representing the inverse transform of b.

      source

      Bijectors

      Bijectors.CorrBijector — Type
      CorrBijector <: Bijector

      A bijector implementation of Stan's parametrization method for Correlation matrix: https://mc-stan.org/docs/2_23/reference-manual/correlation-matrix-transform-section.html

      Basically, a unconstrained strictly upper triangular matrix y is transformed to a correlation matrix by following readable but not that efficient form:

      K = size(y, 1)
+ 1.0  1.0

      Unlike exp, some transformations have parameters affecting the resulting transformation they represent, e.g. Logit has two parameters a and b representing the lower- and upper-bound, respectively, of its domain:

      julia> using Bijectors: Logit
      julia> f = Logit(0.0, 1.0)Bijectors.Logit{Float64, Float64}(0.0, 1.0)
      julia> f(rand()) # takes us from `(0, 1)` to `(-∞, ∞)`-3.831545221580385

      User-facing methods

      Without mutation:

      with_logabsdet_jacobian

      With mutation:

      Bijectors.transform! — Function
      transform!(b, x[, y])

      Transform x using b, storing the result in y.

      If y is not provided, x is used as the output.

      source
      Bijectors.logabsdetjac! — Function
      logabsdetjac!(b, x[, logjac])

      Compute log(abs(det(J(b, x)))) and store the result in logjac, where J(b, x) is the jacobian of b at x.

      source
      Bijectors.with_logabsdet_jacobian! — Function
      with_logabsdet_jacobian!(b, x[, y, logjac])

      Compute transform(b, x) and logabsdetjac(b, x), storing the result in y and logjac, respetively.

      If y is not provided, then x will be used in its place.

      Defaults to calling with_logabsdet_jacobian(b, x) and updating y and logjac with the result.

      source

      Implementing a transformation

      Any callable can be made into a bijector by providing an implementation of ChangeOfVariables.with_logabsdet_jacobian(b, x).

      You can also optionally implement transform and logabsdetjac to avoid redundant computations. This is usually only worth it if you expect transform or logabsdetjac to be used heavily without the other.

      Similarly with the mutable versions with_logabsdet_jacobian!, transform!, and logabsdetjac!.

      Working with Distributions.jl

      Bijectors.bijector — Function
      bijector(d::Distribution)

      Returns the constrained-to-unconstrained bijector for distribution d.

      source
      Bijectors.transformed — Method
      transformed(d::Distribution)
+transformed(d::Distribution, b::Bijector)

      Couples distribution d with the bijector b by returning a TransformedDistribution.

      If no bijector is provided, i.e. transformed(d) is called, then transformed(d, bijector(d)) is returned.

      source

      Utilities

      Bijectors.elementwise — Function
      elementwise(f)

      Alias for Base.Fix1(broadcast, f).

      In the case where f::ComposedFunction, the result is Base.Fix1(broadcast, f.outer) ∘ Base.Fix1(broadcast, f.inner) rather than Base.Fix1(broadcast, f).

      source
      Bijectors.isclosedform — Method
      isclosedform(b::Transform)::bool
+isclosedform(b⁻¹::Inverse{<:Transform})::bool

      Returns true or false depending on whether or not evaluation of b has a closed-form implementation.

      Most transformations have closed-form evaluations, but there are cases where this is not the case. For example the inverse evaluation of PlanarLayer requires an iterative procedure to evaluate.

      source

      API

      Bijectors.Transform — Type

      Abstract type for a transformation.

      Implementing

      A subtype of Transform of should at least implement transform(b, x).

      If the Transform is also invertible:

      • Required:
        • Either of the following:
          • transform(::Inverse{<:MyTransform}, x): the transform for its inverse.
          • InverseFunctions.inverse(b::MyTransform): returns an existing Transform.
        • logabsdetjac: computes the log-abs-det jacobian factor.
      • Optional:
        • with_logabsdet_jacobian: transform and logabsdetjac combined. Useful in cases where we can exploit shared computation in the two.

      For the above methods, there are mutating versions which can optionally be implemented:

      source
      Bijectors.Bijector — Type

      Abstract type of a bijector, i.e. differentiable bijection with differentiable inverse.

      source
      Bijectors.Inverse — Type
      inverse(b::Transform)
+Inverse(b::Transform)

      A Transform representing the inverse transform of b.

      source

      Bijectors

      Bijectors.CorrBijector — Type
      CorrBijector <: Bijector

      A bijector implementation of Stan's parametrization method for Correlation matrix: https://mc-stan.org/docs/2_23/reference-manual/correlation-matrix-transform-section.html

      Basically, a unconstrained strictly upper triangular matrix y is transformed to a correlation matrix by following readable but not that efficient form:

      K = size(y, 1)
 z = tanh.(y)
 
 for j=1:K, i=1:K
@@ -32,12 +32,12 @@
 [w1'w1 w1'w2 ... w1'wn;
  w2'w1 w2'w2 ... w2'wn;
  ...
-]

      The diagonal elements are given by wk'wk = 1, thus x is a correlation matrix.

      Every step is invertible, so this is a bijection(bijector).

      Note: The implementation doesn't follow their "manageable expression" directly, because their equation seems wrong (7/30/2020). Insteadly it follows definition above the "manageable expression" directly, which is also described in above doc.

      source
      Bijectors.LeakyReLU — Type
      LeakyReLU{T}(α::T) <: Bijector

      Defines the invertible mapping

      x ↦ x if x ≥ 0 else αx

      where α > 0.

      source
      Bijectors.Stacked — Type
      Stacked(bs)
+]

      The diagonal elements are given by wk'wk = 1, thus x is a correlation matrix.

      Every step is invertible, so this is a bijection(bijector).

      Note: The implementation doesn't follow their "manageable expression" directly, because their equation seems wrong (7/30/2020). Insteadly it follows definition above the "manageable expression" directly, which is also described in above doc.

      source
      Bijectors.LeakyReLU — Type
      LeakyReLU{T}(α::T) <: Bijector

      Defines the invertible mapping

      x ↦ x if x ≥ 0 else αx

      where α > 0.

      source
      Bijectors.Stacked — Type
      Stacked(bs)
 Stacked(bs, ranges)
 stack(bs::Bijector...)

      A Bijector which stacks bijectors together which can then be applied to a vector where bs[i]::Bijector is applied to x[ranges[i]]::UnitRange{Int}.

      Arguments

      • bs can be either a Tuple or an AbstractArray of 0- and/or 1-dimensional bijectors
        • If bs is a Tuple, implementations are type-stable using generated functions
        • If bs is an AbstractArray, implementations are not type-stable and use iterative methods
      • ranges needs to be an iterable consisting of UnitRange{Int}
        • length(bs) == length(ranges) needs to be true.

      Examples

      b1 = Logit(0.0, 1.0)
 b2 = identity
 b = stack(b1, b2)
-b([0.0, 1.0]) == [b1(0.0), 1.0]  # => true
      source
      Bijectors.RationalQuadraticSpline — Type
      RationalQuadraticSpline{T} <: Bijector

      Implementation of the Rational Quadratic Spline flow [1].

      • Outside of the interval [minimum(widths), maximum(widths)], this mapping is given by the identity map.
      • Inside the interval it's given by a monotonic spline (i.e. monotonic polynomials connected at intermediate points) with endpoints fixed so as to continuously transform into the identity map.

      For the sake of efficiency, there are separate implementations for 0-dimensional and 1-dimensional inputs.

      Notes

      There are two constructors for RationalQuadraticSpline:

      • RationalQuadraticSpline(widths, heights, derivatives): it is assumed that widths,

      heights, and derivatives satisfy the constraints that makes this a valid bijector, i.e.

      • widths: monotonically increasing and length(widths) == K,
      • heights: monotonically increasing and length(heights) == K,
      • derivatives: non-negative and derivatives[1] == derivatives[end] == 1.
      • RationalQuadraticSpline(widths, heights, derivatives, B): other than than the lengths, no assumptions are made on parameters. Therefore we will transform the parameters s.t.:
      • widths_new ∈ [-B, B]ᴷ⁺¹, where K == length(widths),
      • heights_new ∈ [-B, B]ᴷ⁺¹, where K == length(heights),
      • derivatives_new ∈ (0, ∞)ᴷ⁺¹ with derivatives_new[1] == derivates_new[end] == 1, where (K - 1) == length(derivatives).

      Examples

      Univariate

      julia> using StableRNGs: StableRNG; rng = StableRNG(42);  # For reproducibility.
+b([0.0, 1.0]) == [b1(0.0), 1.0]  # => true
      source
      Bijectors.RationalQuadraticSpline — Type
      RationalQuadraticSpline{T} <: Bijector

      Implementation of the Rational Quadratic Spline flow [1].

      • Outside of the interval [minimum(widths), maximum(widths)], this mapping is given by the identity map.
      • Inside the interval it's given by a monotonic spline (i.e. monotonic polynomials connected at intermediate points) with endpoints fixed so as to continuously transform into the identity map.

      For the sake of efficiency, there are separate implementations for 0-dimensional and 1-dimensional inputs.

      Notes

      There are two constructors for RationalQuadraticSpline:

      • RationalQuadraticSpline(widths, heights, derivatives): it is assumed that widths,

      heights, and derivatives satisfy the constraints that makes this a valid bijector, i.e.

      • widths: monotonically increasing and length(widths) == K,
      • heights: monotonically increasing and length(heights) == K,
      • derivatives: non-negative and derivatives[1] == derivatives[end] == 1.
      • RationalQuadraticSpline(widths, heights, derivatives, B): other than than the lengths, no assumptions are made on parameters. Therefore we will transform the parameters s.t.:
      • widths_new ∈ [-B, B]ᴷ⁺¹, where K == length(widths),
      • heights_new ∈ [-B, B]ᴷ⁺¹, where K == length(heights),
      • derivatives_new ∈ (0, ∞)ᴷ⁺¹ with derivatives_new[1] == derivates_new[end] == 1, where (K - 1) == length(derivatives).

      Examples

      Univariate

      julia> using StableRNGs: StableRNG; rng = StableRNG(42);  # For reproducibility.
 
 julia> using Bijectors: RationalQuadraticSpline
 
@@ -74,7 +74,7 @@
 julia> b([-1., 5.])
 2-element Vector{Float64}:
  -1.5660106244288925
-  5.0

      References

      [1] Durkan, C., Bekasov, A., Murray, I., & Papamakarios, G., Neural Spline Flows, CoRR, arXiv:1906.04032 [stat.ML], (2019).

      source
      Bijectors.Coupling — Type
      Coupling{F, M}(θ::F, mask::M)

      Implements a coupling-layer as defined in [1].

      Examples

      julia> using Bijectors: Shift, Coupling, PartitionMask, coupling, couple
+  5.0

      References

      [1] Durkan, C., Bekasov, A., Murray, I., & Papamakarios, G., Neural Spline Flows, CoRR, arXiv:1906.04032 [stat.ML], (2019).

      source
      Bijectors.Coupling — Type
      Coupling{F, M}(θ::F, mask::M)

      Implements a coupling-layer as defined in [1].

      Examples

      julia> using Bijectors: Shift, Coupling, PartitionMask, coupling, couple
 
 julia> m = PartitionMask(3, [1], [2]); # <= going to use x[2] to parameterize transform of x[1]
 
@@ -101,7 +101,7 @@
 Shift([2.0])
 
 julia> with_logabsdet_jacobian(cl, x)
-([3.0, 2.0, 3.0], 0.0)

      References

      [1] Kobyzev, I., Prince, S., & Brubaker, M. A., Normalizing flows: introduction and ideas, CoRR, (), (2019).

      source
      Bijectors.NamedTransform — Type
      NamedTransform <: AbstractNamedTransform

      Wraps a NamedTuple of key -> Bijector pairs, implementing evaluation, inversion, etc.

      Examples

      julia> using Bijectors: NamedTransform, Scale
+([3.0, 2.0, 3.0], 0.0)

      References

      [1] Kobyzev, I., Prince, S., & Brubaker, M. A., Normalizing flows: introduction and ideas, CoRR, (), (2019).

      source
      Bijectors.NamedTransform — Type
      NamedTransform <: AbstractNamedTransform

      Wraps a NamedTuple of key -> Bijector pairs, implementing evaluation, inversion, etc.

      Examples

      julia> using Bijectors: NamedTransform, Scale
 
 julia> b = NamedTransform((a = Scale(2.0), b = exp));
 
@@ -111,7 +111,7 @@
 (a = 2.0, b = 1.0, c = 42.0)
 
 julia> (a = 2 * x.a, b = exp(x.b), c = x.c)
-(a = 2.0, b = 1.0, c = 42.0)
      source
      Bijectors.NamedCoupling — Type
      NamedCoupling{target, deps, F} <: AbstractNamedTransform

      Implements a coupling layer for named bijectors.

      See also: Coupling

      Examples

      julia> using Bijectors: NamedCoupling, Scale
+(a = 2.0, b = 1.0, c = 42.0)
      source
      Bijectors.NamedCoupling — Type
      NamedCoupling{target, deps, F} <: AbstractNamedTransform

      Implements a coupling layer for named bijectors.

      See also: Coupling

      Examples

      julia> using Bijectors: NamedCoupling, Scale
 
 julia> b = NamedCoupling(:b, (:a, :c), (a, c) -> Scale(a + c));
 
@@ -121,4 +121,4 @@
 (a = 1.0, b = 8.0, c = 3.0)
 
 julia> (a = x.a, b = (x.a + x.c) * x.b, c = x.c)
-(a = 1.0, b = 8.0, c = 3.0)
      source
      +(a = 1.0, b = 8.0, c = 3.0)source