I see people bring up this point, sometimes. Besides Elethian's response that there were about ten years in which he was not active as a spy--at least as a double agent--I just point out that spies in real life do not take vows of celibacy. It must be a difficult dance, and certainly risky to the loved ones as well as to the spy, but the same is true with any risky profession, e.g., law enforcement. As I understand it, the main limit on the intimacy side is that the spy can't vent about work the way most people would; life is compartmentalized. But relationships are still possible.