Alertmanager supports configuration to create a cluster for high availability. This can be configured using the --cluster-* flags.

It's important not to load balance traffic between Prometheus and its Alertmanagers, but instead, point Prometheus to a list of all Alertmanagers.