“That logic” didn’t talk about what is okay and what isn’t okay. It talks about what is privilege.

Veganism doesn’t talk about what is okay, either. It talks about what is vegan. Causing suffering is not necessarily not vegan. A vegan can defend themselves. A vegan can even defend their crops. What is not vegan is exploitation in the broadest sense: getting a benefit from the use of something.

You are expected to value life even if you are not vegan. Though, non-vegans have a funny way of showing it.