Thanks for your response. Below are answers to your questions and some more detail. I am very eager to get this resolved and will provide any detail you require. Note that as this has been escalated to very high level JTAC engineers - each JTAC engineer has put a lot of time and effort into this but unfortunately the issue is still very prevalent. It is not due to lack of effort from Juniper. - Since when did this issue start? From the best of our knowledge it has ALWAYS been this way. We are only now getting a great deal of complaints as users who had been migrated from our old IPsec platform to this new platform were really not using this new platform until we disabled their old accounts. So now all users, no matter what, are using the Junipers and the complaints are growing daily. - Any upgrade on the IVE OS / Any changes on the network / Any changes on the load balancer? No changes to IVE - we are running 7.0R5.1 for many months now. We cannot upgrade any higher as it wreaks havoc with the "logoff on connect" feature that about 50% of our users use. When we upgrade the new version of NC does not keep the setting for "logoff on connect" so all users that have it enabled have to re-enable it. We can't push policy for "logoff on connect" as only 50% use it and we can't hit everyone... Note that I escalated this with JTAC and in the next release they hope to have this issue fixed as well. No changes at all to network. Only changes to Load Balancer was to set persistence - which per Juniper Admin Guide we should have always had on! - and then recently we upped the load balancer inactivity timer to 12 hours instead of 30 minutes - What load balancer are you using there and have you involved your load balancer team on to this and if yes, what do they say? We are using F5 load balancers and the F5 team is fully on top of this and they see absolutely no issue with the F5 deployment and we use these F5 devices for a ton of load balancing for many other applications with no issue. - Do you see any logs on the load balancer regarding this? We do not have F5 logs that show exactly what is happening and we would have to set up a sniffer trace to understand if there is truly an issue but it is very important to understand that we have no issues with load balancing any other servers/apps and more importantly our UK and Singapore solutions are set to 100/0 on the load balancer where if the primary goes down in UK or Singapore the traffic is sent to the United States - not the secondary in the UK or Singapore cluster. We see absolutely no evidence of users in UK or Singapore bouncing back and forth between UK/Singapore and the US. That speaks volumes here as it really points us to the Juniper devices/clusters. The secondary devices in UK and Singapore are not used in any way. shape, or form via our network, load balancer, etc. yet we see users boucning over to them in the Juniper logs. Also, the load balancer logs shows absolutely no instances of any of the monitored devices going down. So the LB is not moving anyone to another IP. - Are there any time pattern you see this issue? (Peak hours / Non-peak hours / Anytime / Always) Always - Is there any reason why JTAC feels it's your load balancer issue? (Any specific logs pointing to it?) I'm going to try to sum up what JTAC is saying and I hope I capture it 100% properly... They say that although state tables are shared within a cluster, Juniper has no mechanism for moving users back and forth between the clusters unless one of the clusters goes down. <<tnone of the bosxes are ever down even momentarily per montioring tools, event logs, etc>> So JTAC keeps thinking it is our load balancer even though we supply evidence that it cannot be the load balancer. <<For example - sorry to repeat - LB is set to 100/0 with persistence and inactivity at 12 hours in UK and Singapore and the LB does not even know about the IP addresses of the secondary devices out there and failover is to US. So LB in this situation really can't be involved.>> JTAC keeps thinking that the user - on initial sign-in - resolves the name to one member IP and then somehow starts resolving to the other member IP. They also said that when Host Checker performs its 60 minute interval scan to ensure AV is running and user is still aligned with security policy it could reach out to the user and when the response comes back it could be going to the other cluster member and therefore the member that the user is on thinks it does not get a response and blows the user away. Because they see the users bouncing back and forth between boxes they think this is the cause of the drops. However, the LB now keeps the user going to the same IP for 12 hours with persistence and the maximum session length set in the IVE is 12 hours so during the life of the session the user will resolve to the same IP. On top of that, when a user connects with Network Connect their hosts table is configured by Juniper to use the IP address of the IVE that they connected to. So really, once a user is connected, they should always resolve via their hosts file to the correct IP. PLEASE NOTE - We have SA 6500s in US and UK and SA 4500s in Singapore and in all cases, nearly all users are using Network Connect so we cannot turn off acceleration per your detail. <<<Please "DO NOT" disable the SSL hard acceleration if you have many NC (Network Connect) users. Since our NC (Network Connect) client uses 3DES/AES by default, by disabling acceleration, there could be a noticeable performance impact on the system. You may not see a process restart per-se, but packet loss will be greater (for NC) and the page response time will be slower (for core).>>>> HOWEVER:::: Either late last year or early this year we were getting complaints of slow response and JTAC suggested that we turn OFF hardware acceleration. So if your points in your post are accurate, could that be why we are still having slow response reports from our end users???? If all of our users are using Network Connect and we have hardware acceleration disabled are we adding to the slow response misery???? See JTAC response about this below: It has to do with SSL Acceleration option. This feature is enabled by default on all SA6500 devices. It is used to offload SSL operation from the main CPUÓ Due to some issues seen in the past with enabling the feature, the vendor recommends disabling it. Below is a statement we received from the vendor: We are recommending that it be disabled for two reasons, out of an abundance of caution, and secondly, it will have no impact on the functionality of the box, you do not require it for the box to work, but you will potentially see a CPU hit of 20-30% as the device approaches maximum user load. PR for the potential SSL issue is 562746.
... View more