Consumers aren't as confident about the economy as they were earlier this year. That could mean more worries for everyone.