ZPA + round robin DNS load balanced RDP environment

We’re experiencing connection issues over ZPA with our load balanced RDP environment. What it SEEMS like it happening is when a user connects, they’re dropped into server #1. Then user disconnects. User reconnects, and the connection broken attempts to put them onto server #2, the connection fails. If the user manually tries to instead connect to server #1, the connection succeeds.

The way this is setup:

rd.internaldomain.com - round robin DNS record that points to RDP server #1 and RDP server #2.
rdpserver1.internaldomain.com - RDP server #1
rdpserver2.internaldomain.com - RDP server #2
remote.domain.com - DNS record pointing to the RDP gateway/connection broker

In the ZPA logs, I see zero connection attempts to anything in the environment other than remote.domain.com. I assume this is because all the traffic is simply proxied through the RDP gateway server. This is making it hard to troubleshoot, because I can’t find any failed traffic, etc. No hits in Diagnostics to the round robin record, etc.

Is there anything that would cause an issue with this setup with ZPA? Is there a way to configure it more optimally for use with ZPA? While I can find no failures in the logs, we know the issue has something to do with ZPA because when bypassing ZPA, the connection works fine 100% of the time.

remote.domain.com is the publicly available FQDN for the gateway/broker? This is what the user connects to, which then forwards onto the .internaldomain.com servers?
If the user is connecting to the gateway/broker - whether through ZPA or not - I wouldn’t expect them to subsequently make a connection to the actual server directly - I could be wrong though.
One thing to check - RDP tends to want to get an ICMP response back from the server before making a connection. Is ICMP enabled on the segment, and in policy to allow access?