When exercising your body, what you may consider to be challenging, your body may find it to be something totally fine to do (in other words, not much of a challenge). Perhaps you shouldn't trust your "Perception Of Exertion"; you may need to push harder than you think to keep on reaping rewards.
Overall, if you find that you are not working out hard enough, begin intensifying your workout. For example, if you walk every day, walk a longer distance AND faster. If your daily exercise is riding your bike, try pedaling uphill and shifting to a more difficult gear. It may be that your daily workout consists of a dance class; try taking a more advanced level dance class.
Adding weights to your cardio activity is also a good way to intensify your workout; such as walking with hand weights, or if using a jump rope, add ankle weights.
Exercise should make you sweat, make your muscles burn, and leave you with a feeling of accomplishment. If you aren’t feeling these things, then perhaps you need to push yourself a bit more!
You may also consider getting yourself a heart rate monitor and checking it frequently.