general inference on a graph of dependant variables is NP-hard to do exactly. So you use a sampling based method to converge towards the right solution and stop when it looks about right. However, for many smaller problems there is an exact analytic solution.
Real problems in the wild are a mix of both types of problem, and intractable part and a part you can do precisely.
In an ideal world you would use the iterative methods ONLY for the parts that don't have an exact solution available. In practice once you have developed the general iterative solution, you might as well just use that for the easy parts too, as you are just burdening yourself with more development work and more scope for bugs by writing two inference implementations.
A language that allowed inference to be switched at no development cost it would be amazing
Real problems in the wild are a mix of both types of problem, and intractable part and a part you can do precisely.
In an ideal world you would use the iterative methods ONLY for the parts that don't have an exact solution available. In practice once you have developed the general iterative solution, you might as well just use that for the easy parts too, as you are just burdening yourself with more development work and more scope for bugs by writing two inference implementations.
A language that allowed inference to be switched at no development cost it would be amazing