After 20 turns you can cancel it savely. To do this, open negotiations ("we would like to propose a deal"), then click on "Active". It will show you all your active contracts with the number of turns before it can be cancelled peacefully. If there is no number, it exists for more than 20 turns and it can be cancelled. Click on the item you want to cancel.
You can also change your preferences to "always renegotiate after 20 turns". That way you will be reminded automatically. This method has the disadvantage though, that from then on, the deal will last another 20 turns, whereas if you hadn't touched it, you could cancel it whenever you want.
Hope that helps