If you wanted to work out to lose weight, with your body only burning fat for energy, to lose only one pound you would have to run or walk about 35 miles.*
In contrast, if your body only burned carbohydrate (glycogen), that same pound could be lost after only 18 miles of walking or running.*
You could say the same thing if you wanted to lose weight by by cutting calories instead of exercise. For example, refraining from eating all together would see the pound lost in about two days, assuming that you weighed around 120 pounds to begin with.
By switching to glycogen for fuel that same pound could be trimmed in just one day!* Fat is an extremely energy-dense substance, packing many calories into each gram, ounce, or pound, so it is much easier to burn carbos instead.
A gram of fat packs nine calories, while a gram of glycogen holds only four. As a result, you can burn a lot of fat calories without having much impact on your weight. Our bodies are set up that way for a good reason, strange as it may seem. Throughout evolutionary times humans have been nomadic, constantly moving in search of 'greener pastures'.
Unfortunately, humans have also been prone to boom-and-bust cycles. Going from having plenty of food to near starvation in a very short amount of time, a way to store lots of energy with very little weight was necessary to survival. This role was nicely filled by fat. The flabby, foamy material draping our internal organs, lying like a blanket beneath our skin, is relatively light and rich in calories.
The consequence of this is that if you manage to drop in weight quickly, it's probably not fat that is being lost.