Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

How is that different that testing a bunch of hypotheses forwards and determining which ones result in the end/starting data?


Because:

(1) Even very simple linear models in effect encode uncountably many hypotheses (i.e., y = a x + b where a and b are real numbers). You can't just pull (a,b) out of a hat until the agreement looks good, because the hat is too big.

One tool is, obviously, analytic solutions (where possible) like least squares. Beyond that, iterative methods like the EM algorithm. Beyond that, sampling-based methods like MCMC. You can view MCMC as your idea ("generate a bunch of hypotheses and test them"), but with the change that it does not ignore the old hypotheses when generating a new one to test, it uses an old hypothesis to seed a new one, using a probabilistic update rule.

In the presence of a model, you can basically compile (in the CS sense) an inference program that deduces model parameters from input data. The project to do this has been worked on, in fits and starts, since the late 1940s, by a series of geniuses/visionaries.

(2) Once you have a multi-level model, with many parameters, trading off what you gain by tweaking one parameter versus another parameter becomes hard. If the data is

  (x1,y1), ..., (xN,yN)
and you generate approximations Z1 and Z2 using two different models:

  M1: (x1,Z1_1), ..., (xN,Z1_N), i.e. Z1_i = M1(x_i), and
  M2: (x1,Z2_1), ..., (xN,Z2_N), i.e. Z2_i = M2(x_i)
what is the correct way to gauge agreement of Z1 versus Z2 to the target y? Only working through the model will tell you. For many reasons, least squares it not always suitable.

So even if you were content to keep generating hypotheses and checking agreement, you wouldn't know how to measure agreement.


That's one way to do it, it's just tremendously inefficient in general.

Really grossly impressively inefficient.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: