The answer is there in your question. You cannot give everyone unlimited rights and freedoms without imposing on others'.
What would happen if Canadian rights and freedoms were without limits? (of course they are limited to protect others rights and freedoms)