Don't do it.  It may be OK but (1) all you have to do is turn both on once and you may regret it and (2) there is no assurance that, even in standby, the output impedance of one amp will strongly shunt the other.

Two suggestions:

1.  Get a switch.

2.  Use a preamp with a HT-bypass function.

Kal