Try different prior & shifted bimodal distribution

[lorenzkuhn]

No description provided.

[lorenzkuhn]

The goal here was to investigate some of the initialisation issues of our bimodal posterior toy problem.

Shifted Posterior

The first idea was to modify our model from:
mu ~ N(0, 1)
y ~ N(mu^2, sigma^2)
to:
mu ~ N(0, 1)
y ~ N((mu+1)^2, sigma^2)
Observing data points around +4, this will shift the expected true posterior modes from -2 and +2 to -3 and +1, for example. The problem of having to choose an initialisation for the guide and a suitable learning rate for Adam is thus the same as for the original problem.

Shifted Prior

The second idea we had to break the symmetry of the posterior was to move shift the prior on mu to N(1,1) instead of N(0,1), for the same observations around +4.
Looking into this right now, looks like it makes a relatively small difference so far.