You shouldn't have to, but you should anyway. Sometimes it's just a rogue province. I ALWAYS message the monarch of a kingdom when we get ops or attacks into our wars. The majority of the time, it was an accident, and we get an apology. We don't see them after that.

In the case where it isn't an accident, at least you'll know for sure, and can take appropriate action. Razing into their war usually discourages that kind of behavior in the future.