You have a baby, and what if he is allergic to your cats? Horror is what. Separation is very difficult. Especially with family children.