-ELBO(qt, p) plot

[gideonite]

This would be a proxy for the error of our approximation. We want it to go down.

[lorenzkuhn]

Hey @gideonite, the idea here was to estimate E_{q_t}[log p] after adding each new component to the approximation, correct?

I did that for our bimodal posterior toy example and got the following results. Using six components, we get the following approximation:

The pink function is the entire resulting approximation, the other functions are the pdfs of the approximation components. The smaller the max of the pdf, the earlier the component was added to the approximation.

Plotting ELBO(q_t, p) after adding each component we get the following:

As expected, we see that E_{q_t}[log p] goes up as the approximation is closer to the true posterior, and goes down, as new components are added that make the approximation worse.

[lorenzkuhn]

Add -E[log q_t] w.r.t q_t

[lorenzkuhn]

Updated my initial comment above with the new plot.

[lorenzkuhn]

What happens if we use more than 6 components?