Zscaler troubleshooting tools for connectivity and performance/slowness issues

,

Hello,

Here is a fast list of some of Zscaler troubleshooting tools primary for ZIA:

  1. The first is the Zscaler Analyzer that everyone can download to test the load time and performance of a web page through the Zscaler cloud.

https://zmtr.zscaler.com/

  1. The second tool is a web tool and it is the web site https://ip.zscaler.com/ where everyone can see to which Zscaler Gateway they are connected. Be carefull as ip.zscaler.com shouldn’t be excluded from the PAC file (like *.zscaler.com “Direct” can make this happen) and also it looks at XFF header so this should be enable under Locations if GRE or IPSEC is used and not the zscaler connector app.

https://ip.zscaler.com/

  1. Another usefull web tool is the Zscaler cloud performance tool that is to measure if there is an issue between the user and the ZIA edge but again don’t bypass it in the PAC file and the Zscaler Analyzer can be used after this tool to check the connection to the web page itself. For some reason the tool does not open with Mozzila for me but with Chrome and Edge there is no issue.
  1. The Zscaler ZDX is a payed feature that is really good for testing issues with cloud application like salesforce etc.
  1. The final web tool is trust zscaler where it can be checked for known issues is parts of the Zscaler Cloud and the ZIA edge gateways.
  1. Another nice web tool is config Zscaler just for basic info about future zscaler data center locations etc.

https://config.zscaler.com/zscaler.net/cenr

  1. Also the Zscaler client connector logs and diagnostics can be checked and the Zscaler Insights logs from the admin portal for Policy action, SSL Inspected, SSL Policy Reason, Proxy Latency etc. If the Sandbox action is to scan an unknown file and then to let the user download it (not allow an scan first site), this can also cause latency expecially if there is an issue with the sandbox in that region but the Trust Zscaler site should mention if there is sandbox issue at the region.

https://help.zscaler.com/zia/documentation-knowledgebase/analytics/dashboards-reports-and-logs/logs

  1. For general ZIA troubleshooting and proxy PAC file troubleshooting or client Connector error codes ( for mozilla and zscaler client connector there is a special article Configuring Firefox Integration for Zscaler Client Connector | Zscaler ):

https://help.zscaler.com/zia/troubleshooting

  1. For ZPA the logs are even more detailed like if a AD user , AD group is blocked by the Access Policy, PolicyProcessingTime , CAProcessingTime etc. between the private application and the user .
  1. For investigating issue with Zscaler ZPA app connectors. You can send session commands to the Zscaler Connectors from the ZPA admin portal or if using ZPA private edge also commands can be send to it like ping etc. Also the user can log into the Zscaler App connector as it is Linux and do tcpdump, ssldump, ping etc but do not forget that ECC SSL ciphers can’t be decrypted with ssldump.
  1. Sometimes tools like wireshark are still needed as for example the dropbox application being blocked by ZIA and having to bypass all the domains that google uses for sync of the dropbox but the Zscaler Insight logs not showing all the blocked traffic or even allowing SSL traffic that can’t be decrypted not helping (Solved: Syncing issue with Zscaler - Dropbox Community). Or using F12 or fiddler or HTTPwatch to capture web traffic from the client for maybe investigating SAML issues when the Zscaler connector is not used.

Also check the zscaler client connector release notes for known bugs with your version or with ZPA, For some reason the ZIA release notes do not include known issue information.

The Zscaler TAC can do advanced Zscaler debugs on the Edge gateways if nothing helps.

  1. Also you can consider using suffix, _FX in your PAC file as when many users are using the same Public EDGE this can cause issues. Another solution is to use a private/virtual edge but it is more expensive.

Use the suffix, _FX to the ${COUNTRY_GATEWAY_HOST} variable in the PAC file for the PAC server to dynamically issue the gateway hosts within a country based on the client fingerprints, i.e. all users coming from a single egress IP address are given a gateway host from a pool of healthy gateway hosts. The fingerprint is used to ensure that a single device continues its session to the same gateway host.

Use the following syntax to include the ${COUNTRY_GATEWAY_HOST_FX} variable in your PAC file:

return “PROXY ${COUNTRY_GATEWAY_HOST_FX}:80; PROXY ${COUNTRY_SECONDARY_GATEWAY_HOST_FX}:80; DIRECT”;

You can also use the _FX suffix with the subcloud variables. For example, ${COUNTRY_GATEWAY.<Subcloud>.<Zscaler cloud>.net_HOST_FX} and ${COUNTRY_SECONDARY.GATEWAY.<Subcloud>.<Zscaler cloud>.net_HOST_FX}.

The _FX suffix provides load balancing for multiple gateway hosts depending on the HTTP headers (useragent, x-forwarded-for, and z-client). This variable is effective only for Zscaler Client Connector clients because the z-client ID is different for each user.


  1. Traffic bypass or SSL bypass can always be tested if it helps. Or bypass the malware, sandbox if you see issues with file scaning and contact Zscaler TAC. The PAC bypass for tunnel 2.0 works only for web traffic.
  1. Sometimes for better performance zscaler private/virtual edges are needed. Also the user can log into the Private/Virtual Edges as it is Linux and do tcpdump, ssldump, ping etc. or use the Zsclaler TAC support script for debug on Private/Virtual edges “ZSINSTANCE=/” , where the client source IP address is selected, so that only its session is captured but better ask the TAC for it.
  1. For ZIA with tunnel 2.0 specific issues sometimes the DNS should not enter the tunnel 2.0 or the web tool @manuel mentioned in the comments can be used to test the tunnel.
1 Like

And to complete your quite comprehensive list:
If you use ZCC and Tunnel 2.0 be sure to checkout ZCC built in diagnostic tool located at “127.0.0.1:9000?ztest?q=username@company.com” (see also Slowness on the network - #2 by jalomari, kudos to @jalomari for this - quite old - post).

BR
Manuel

3 Likes

This last one is just amazing !!! Never heard or seen it before. So useful.

1 Like

It seems something that Zscaler has not shared like the “ZSINSTANCE=/” script. @manuel thanks for sharing it as if anyone knows any more not documented options please share :slight_smile: