Of course you are right that stock markets do go up and down. I'm looking at this mathematically, looking at historic gains. The AVERAGE 10 year gain is 9.2%. HOWEVER, you are right, there will be years where you could lose a lot - 2008, 1929, etc., all come to mind. If you dollar cost average and don't panic sell, then you'll still make more money over the long term in index funds.
But emotionally, it can totally stress you out. From an emotional POV, for a lot of people, it IS better to pay off the house. And, you have to have a place to live, so if you have a paid off house... well, that really does give you security. Paying off your house early is a lower risk strategy, but index funds over the LONG TERM provide higher rewards.
Anyway, thanks for the comment, because this is an important factor in the economic calculus - for many people, lowering risk is a much more important factor.