No you will not get in trouble.
Your visa is a “travel document” which lets you come to Canada. Your work permit is your status document. So as long as you stay in Canada during the validity of your work permit you do not need to renew your visa. You only need a new visa if you leave Canada and wish to return (by air/train/bus/ferry).