Technically yes, you can leave Canada immediately after you become a Canadian citizen. What’s the catch?
Well, at your citizenship ceremony you must hand over your PR Card. Without a PR Card and without a Canadian passport, you will have trouble flying back to Canada. If you’re driving, this may not be a big deal, but if you are travelling any other way, you should get your Canadian passport first and then travel.