RPC over DCOM using WSAM/JSAM

[KevinW_]

I have a major problem with one of the applications we use here. Not the best in the world, id even go as far to say it was totally rubbish, but still our company still requires it to be used if people are connecting through SAM.

I have had no problems with other applications working at all, but this one fires up an .exe file which then makes a call to DCOM to fire off an RPC request to the host which can be seen in DNS. I always get an error back saying "Unable to connect to destination host. RPC server is not available". I think this is because its trying to use the DCOM subsystem to make the call to the server, instead of the actual .exe file making the RPC request. This causes SAM not to work, as it expects the actual .exe file to make the connection on the session and not DCOM.

So my question to someone is: Has anyone managed to get RPC over DCOM working for SAM? And "change the application" is not an option

Any pointers would be appreciated.

[ruc_]

Have you tried adding the destination IP of the RPC request to the "wsam allowed servers list" under role options? In this mode WSAM will not care which exe generated the traffic as long as the destination IP matches.

[KevinW_]

I have tried adding the hostname of the server, but not the IP address. I did notice that the client does not even try and contact the hostname of the server, even though I can resolve it and DNS is working fine, but I will try adding the IP address instead.

Tried this on a Net Connect role too, and appears to work ok. Its although the SAM application is not capturing the request for DCOM to use RPC.

I will try this and post back.

[KevinW_]

Tried this out and still does not work. Getting the same windows error message "the dcom subsystem could not connec to server (servername). Check that the server is available blah blah blah"

Using a command prompt resolves the server name ok, and again, using the net connect client then the RPC request gets sent through. I am not sure what I am supposed to see in the SAM client itself. I assume it would be svchost.exe trying to make the call to the server, but I am not seeing anything in the client.

Any advice would be good.

[dcvers_]

Try using a tool like the freeware TDImon to determine which processes are sending the network traffic.
Run it when not connected via WSAM to see a normal connection. Also you can run it when connected with WSAM to confirm everything is going through WSAM.

It should also be noted that WSAM has a list of bypass appications which it won't process. There is a default set and you can add you own. The default ones are list in a config file somewhere on the client machine (can't remember where). Having said this using allowed servers usually gets round this list.

[ruc_]

One limitations about WSAM is that it can secure traffic only for applications that rely on Winsock for making network connections, so if your app or RPC or DCOM (whichever is making the actual network calls) does not use winsock then WSAM will not be able to secure traffic. One of ways of finding if winsock is being used is with the help of the tool 'process explorer ' from sysinternals suite. Using this tool you check what dll are being loaded by the application and in the list if you see ws2_32.dll then you know winsock is being used.

[KevinW_]

Thanks guys.

This was going to be my next step, just a little busy at the moment. Usually, I would use something like strace on unix to trace out the application so ill get one of these tools installed and then find out what its doing and then let you know.

[KevinW_]

Ok, changed this message..

the application is using winsock. Also using all the comctl32.ocx files. Next step is to have a look at this with the SAM client active.

Message Edited by KevinW on 02-03-2009 02:13 AM

[KevinW_]

this remains unresolved. I have submitted an RFC to juniper to change the way SAM works on these type of applications.