You could have something like a chess clock which mutes the other person. I am sure the technology would be very simple.It's called not having both of them work remotely.
Or just a simple volume adjuster. If volume is already coming in from one point, the microphone volume is turned off from the other input.