Not-So-Clumpy Dark Matter Poses Cosmological Challenge