Always. I grew up in NS where the current is so strong, and the water so cold, people die all the time. I have lost too many family members off of fishing boats off the coast. The tides are so volatile that fresh water outlets can vary 15-20 feet of depth in a few hours. Not wearing your lifejacket is asking to die.
Here in MA, that is not the case, but it's just so alien to be on the water without a jacket on that I still always wear it. Anyone in my canoe wears one or they don't get in. We use vest type Class III PFDs that do not restrict movement at all, have a ton of pockets, and are new every third year. There is no excuse for not wearing one.
As for the calm shallows, I once saw a guy fishing without a jacket in about 5' of water. He got some huge bite, somehow managed to get excited and flipped his canoe, and something whacked his temple. He was nearly out cold, face down, in 5' of flat water. If he were alone he might have died for no reason at all.