How Siteminder Agent interacts with Policy Server

 I found this below discussion in one of the forums and Hope this will be useful in understanding the flow,
 Discussion 1:
============
1.When user enters the application url in the browser, it goes to the web server, if web agent is installed on the web server, The request is first intercepted by the webagent.conf file
2. In web agent.conf file it finds the path of the smhost.conf file and the request goes to smhost.conf file.
3.In smhost.conf file it finds the policy server which it needs to contact.
4.Now Web server contacts the policy server through the ports 4444(1,2,3) that are mentioned in smhost.conf file. This is done by LLAWP(Low Level Agent Worker Process)
5.LLAWP sends the request to policy server asking for initializing the communication between the web agent and policy server. In the request web agents send the details of trusted host that is present in smhost.conf file
6.When policy server receives the request it checks whether it can trust the request by seeing whether the trusted host present is present in policy store and if it belongs to web server from which it got the request.
7. This initial request sent by web agent is encrypted by the shared secret present in smhost file. So the request is securely and only policy server can decry-pt it.
8. If policy server trusts the request it starts the communication with the web agent. This communication will be started HLA process from policy server. HLA process uses HCO that web agent used to register with policy server.
9. When the communication from policy server reaches the web agent it decrypts the communication using shared secret and in communication it will have details about the HCO which have started the communication, it compares if HCO in communication is same as HCO present in smhost.conf file.
10.This way the policy server and web agent handshake with each other and sets the communication.Now Web agent sends the details of the ACO present in webagent.conf to policy server.
11.The policy server now takes the this ACO and it will see if it is present in policy store, if it is present it will get the agent name present in ACO and also the properties of the agent present in ACO.
12. Now policy server asks for the resource and web agents sends the resource it go from browser.
13. policy server gets the list of all the realms present in policy store which are linked to the agent that is present in the ACO. It compares the resource it got from web agent is present in any realm in the list (Here it sees the longest match) and gets the realm which has this resource and policy server see's whether this realm is configured for protected or not protected. If not protected , it will say web agent it is not protected.
14.if it is protected, it will whether the request that got from browser has smsession. If no smsession then policy server fetches the authscheme from realm and gives the corresponding login page to web agent and it gives it to browser.
This the process that will occur to check whether the resource or request got from browser is protected or not.
Note: All the communication that occur after hand shake between policy server and web agent is in encrypted for security. The encryption and decryption is done using keys present in key store.

Discussion 2:
=========
WebAgent Initialization:
========================

When you register the agent with the policy server thru smreghost command, The SmHost.conf file gets created and SmHost.conf file contains information that the Web Agent uses to make initial connections to the Policy Servers to which it is associated.it also creates a trustedHost object into the policy store.which you specify at the time of registration.
Trusted Host is a client computer where one or more Web Agents is installed. It handles the connection to the Policy Server. The term trusted host refers to the physical system. You can have more than one trusted host on a physical server, but each must be identified by a unique name.

Now when you enable the webagent and bounce the webserver, the initial connection flow goes like this, the webagent reads the WebAgent.conf file and looks for the path of SmHost.conf file, from that path it reads the SmHost.conf file and tries to connect to the IP Address of the policy Server m/c which are present there. once that is done, it then fetches the trustedHost object from the policy store (The policy Store is connected with the policy Server). Once that is done then a successful TCP connection is established between the webAgent and the policy server. the WebAgent then never uses the SmHost.conf file.


Before i expalin the request processing part, let me explain the connection between WebAgent and the WebServer. There is a Http connection between the WebAgent and the WebServer. As soon as you configure the webagent with the webserver, remember we specify the webagent binaries inside the configuration of webserver. so when you bounce the webserver after configuration, it loads the webagent DLL / Binaries into the webserver process. This is how a Http connection is made.

Request Processing:
===================
Now when user access a resource from the browser say http://www.sample.com/mysite/abc.html . The WebAgent filters this request from the web server module and after parsing the host and the resource name webagent sends the resource name to the policy server thru the TCP socket connection in form of packets. for e.g. ACO, AgentName, HostName, Client IP, resourceName etc. each in a seperate encrypted packet.
Now Policy Server decrypts the each packet and it knows the agent name which has sent the request, from the agent name it fetches all the matching Realm Object and from the Realm object it checks the resource which has been associated with a specific realm, it then checks that if the resource is protected in a realm and if it is protected, it sends a challange response back to the webagent over the same TCP socket connection, WegAgent who is listing over that socket connection reads the response and throw a chalange to the user in the browser.
This is how a basic IsProtected request is processed.


One thing i want to mention that each request goes seperately, IsProtected request is different then IsAuthenticated Request is different and IsAuthorize request is different.
There are three different requests.

SSL Handshake

The Secure Sockets Layer (SSL) protocol uses a combination of public-key and symmetric-key encryption. Symmetric-key encryption is much faster than public-key encryption; however, public-key encryption provides better authentication techniques. An SSL session always begins with an exchange of messages called the SSL handshake. The handshake allows the server to authenticate itself to the client by using public-key techniques, and then allows the client and the server to cooperate in the creation of symmetric keys used for rapid encryption, decryption, and tamper detection during the session that follows. Optionally, the handshake also allows the client to authenticate itself to the server.

The steps involved in the SSL handshake are as follows (note that the following steps assume the use of the cipher suites listed in Cipher Suites with RSA Key Exchange: Triple DES, RC4, RC2, DES): 

1. The client sends the server the client's SSL version number, cipher settings, session-specific data, and other information that the server needs to communicate with the client using SSL.
2. The server sends the client the server's SSL version number, cipher settings, session-specific data, and other information that the client needs to communicate with the server over SSL. The server also sends its own certificate, and if the client is requesting a server resource that requires client authentication, the server requests the client's certificate.
3. The client uses the information sent by the server to authenticate the server (see Server Authentication for details). If the server cannot be authenticated, the user is warned of the problem and informed that an encrypted and authenticated connection cannot be established. If the server can be successfully authenticated, the client proceeds to step 4.
4. Using all data generated in the handshake thus far, the client (with the cooperation of the server, depending on the cipher being used) creates the pre-master secret for the session, encrypts it with the server's public key (obtained from the server's certificate, sent in step 2), and then sends the encrypted pre-master secret to the server.
5. If the server has requested client authentication (an optional step in the handshake), the client also signs another piece of data that is unique to this handshake and known by both the client and server. In this case, the client sends both the signed data and the client's own certificate to the server along with the encrypted pre-master secret.
6. If the server has requested client authentication, the server attempts to authenticate the client (see Client Authentication for details). If the client cannot be authenticated, the session ends. If the client can be successfully authenticated, the server uses its private key to decrypt the pre-master secret, and then performs a series of steps (which the client also performs, starting from the same pre-master secret) to generate the master secret.
7. Both the client and the server use the master secret to generate the session keys, which are symmetric keys used to encrypt and decrypt information exchanged during the SSL session and to verify its integrity (that is, to detect any changes in the data between the time it was sent and the time it is received over the SSL connection).
8. The client sends a message to the server informing it that future messages from the client will be encrypted with the session key. It then sends a separate (encrypted) message indicating that the client portion of the handshake is finished.
9. The server sends a message to the client informing it that future messages from the server will be encrypted with the session key. It then sends a separate (encrypted) message indicating that the server portion of the handshake is finished.
10. The SSL handshake is now complete and the session begins. The client and the server use the session keys to encrypt and decrypt the data they send to each other and to validate its integrity.
11. This is the normal operation condition of the secure channel. At any time, due to internal or external stimulus (either automation or user intervention), either side may renegotiate the connection, in which case, the process repeats itself.

 An SSL-enabled client goes through these steps to authenticate a server's identity:

1. Is today's date within the validity period? The client checks the server certificate's validity period. If the current date and time are outside of that range, the authentication process does not go any further. If the current date and time are within the certificate's validity period, the client goes on to step 2.
2. Is the issuing Certificate Authority (CA) a trusted CA? Each SSL-enabled client maintains a list of trusted CA certificates. This list determines which server certificates the client will accept. If the distinguished name (DN) of the issuing CA matches the DN of a CA on the client's list of trusted CAs, the answer to this question is yes, and the client goes on to step 3. If the issuing CA is not on the list, the server is not authenticated unless the client can verify a certificate chain ending in a CA that is on the list.
3. Does the issuing CA's public key validate the issuer's digital signature? The client uses the public key from the CA's certificate (which it found in its list of trusted CAs in step 2) to validate the CA's digital signature on the server certificate that is being presented. If the information in the server certificate has changed since it was signed by the CA, or if the CA certificate's public key doesn't correspond to the private key that was used by the CA to sign the server certificate, the client does not authenticate the server's identity. If the CA's digital signature can be validated, the client treats the server's certificate as a valid "letter of introduction" from that CA and proceeds. At this point, the client has determined that the server certificate is valid. It is the client's responsibility to take step 4 before it takes step 5.
4. Does the domain name in the server's certificate match the domain name of the server itself? This step confirms that the server is actually located at the same network address that is specified by the domain name in the server certificate. Although step 4 is not technically part of the SSL protocol, it provides the only protection against a form of security attack known as a "Man-in-the-Middle Attack." Clients must perform this step and must refuse to authenticate the server or establish a connection if the domain names do not match. If the server's actual domain name matches the domain name in the server certificate, the client goes on to step 5.
5. The server is authenticated. The client proceeds with the SSL handshake. If the client does not get to step 5 for any reason, the server that is identified by the certificate cannot be authenticated, and the user is warned of the problem and informed that an encrypted and authenticated connection cannot be established.

Also,Found a interesting explanation in Stackoverflow,

First, understand that there are generally two steps in HTTPs communication.

1) Generate a shared symmetric key which can only be known by client and server, no one else knows it

2) With this shared symmetric key, client and server is able to safely communicate with each other without worrying about information being intercepted and decrypted by others.

So the question becomes, how can the client and server generate a secret shared key without being known by others in this open internet? This is the asymmetric algorithm coming to rescue, a demo flow is like below:

-- Client receives public key from server.

-- Client generates a key string "DummySharedKey" which will be later used as shared key, and encrypt it into "7$&^^%####LDD!!@" with server's public key, Man-in-the-middle might be able to intercept the encrypted data, but the data is useless to him as the data can only be decrypted by sever's private key.

-- Server receives the encrypted key string "7$&^^%####LDD!!@", and decrypt it into "DummySharedKey" with its private key

Above key exchange steps makes sure that only Client and Server know the shared key is "DummySharedKey".

So it's critical to understand that it is Client's responsibility to generate the shared key, NOT SERVER! (i think this is what confused you)

How the Agent Reads SiteMinder Cookies

How the Agent Reads SiteMinder Cookies

• Web Agents use agent keys to encrypt and decrypt SiteMinder cookies so the data they contain can be read. The Agent uses the key to encrypt cookiesbefore sending them to a user’s browser and to decrypt cookies received fromother Web Agents.
• All Web Agents need to be aware of the same keys, and the keys must be set to the same value for all Agents communicating with a Policy Server. This rule is particularly important for Agents in a single sign-on environment. To ensure that the keys remain secure, the Policy Server performs a key rollover. A key rollover is the process of generating new keys, encrypting them, and distributing them to all Web Agents within a SiteMinder environment.
• When a Web Agent starts up and makes a management call request, the Policy Server supplies the current set of keys. Each time that the Web Agent polls the Policy Server, the agent again makes the management call. The Web Agent receives the updated keys.

The Policy Server provides two types of keys:

• Dynamic Keys—A dynamic key is generated by a Policy Server algorithm and distributed to other connected Policy Servers and their associated WebAgents. Dynamic keys can be rolled over at a regular interval, or by usingthe Key Management dialog box of the Policy Server User Interface.

• Static Keys—A static key remains the same indefinitely, and can be generated by a Policy Server algorithm or configured manually. SiteMinder uses this type of key for a subset of features that require information to be stored in cookies over extended periods of time.

Automated key changes ease the process of managing agent keys for largeSiteMinder installations that share a single key store. A key store is a storage location for all key information; all agents access the key store to obtain the current keys. For Agents that are configured for single sign-on, the key storemust be replicated and shared across all Policy Servers in the single sign-onenvironment. Automating key changes also ensures the integrity of the keys.

Agent Key Dynamic Rollovers

You can use the SiteMinder Key Management dialog box of the Policy ServerUser Interface to configure dynamic Agent key rollover. Web Agents poll thePolicy Server for key updates at regular intervals. If keys have been updated,Web Agents pick up the changes during polling. The default polling time is 30seconds, but can be configured by changing the PSPollInterval parameter of a Web Agent.

When a Web Agent detects that a key rollover has occurred, the Agent retrieves new values for the following Agent keys:

• Old Key—Last value used for the dynamic Agent key before the currentvalue.

• Current Key—Value of the current dynamic Agent key.

• Future Key—Next value that will be used as the current key in a dynamicAgent key rollover.

• Static Key—A long-term key that the Agent can use for SiteMinder featuresthat need to identify a user and maintain this information for long periods of time. Static keys also support cookie encryption for single sign-on whendynamic keys are not enabled.

Web Agents require multiple keys to preserve cookie data and ensure a smoothtransition between old keys and new keys.

Key Stores

When the Policy Server generates dynamic keys, it saves and maintains thesekeys in the key store. The key store is a repository from which all Web Agents retrieve the most current keys. The key store may be part of a SiteMinder policy store or maintained as a standalone key store.


Source :http://vaibhav181.wordpress.com

Analyze Siteminder Logs

Here is the understanding of analyzing Siteminder Event logs

If the event category is authentication  or authorization , the format is:

lpszEvent lpszHostName lpszTimeString "szClientIp szUserName" "szAgentName szAction szResource" [szTransactionId] [nReason] szStatusMsg

The above format is described as follows,

• lpszEvent. The name (type) of the access event:

  SmLogAccessEvent_AuthAccept : lpszEvent = "AuthAccept"
  SmLogAccessEvent_AuthReject : lpszEvent = "AuthReject"
  SmLogAccessEvent_AuthAttempt : lpszEvent = "AuthAttempt"
  SmLogAccessEvent_AuthChallenge : lpszEvent = "AuthChallenge"
  SmLogAccessEvent_AzAccept : lpszEvent = "AzAccept"
  SmLogAccessEvent_AzReject : lpszEvent = "AzReject"
  SmLogAccessEvent_AdminLogin : lpszEvent = "AdminLogin"
  SmLogAccessEvent_AdminLogout : lpszEvent = "AdminLogout"
  SmLogAccessEvent_AdminReject : lpszEvent = "AdminReject"
  SmLogAccessEvent_AuthLogout : lpszEvent = "AuthLogout"
  SmLogAccessEvent_ValidateAccept : lpszEvent = "ValidateAccept"
  SmLogAccessEvent_ValidateReject : lpszEvent = "ValidateReject"
  

• lpszHostName. The name of the host.
• lpszTimeString. The timestamp of the occurrence of the event, in the format: [//:::
  ]. For example: [27/Jun/2000:11:27:29 -0500]
• szClientIp. The IP address of the client machine.
• szUserName. The name of the user.
• szAgentName. The name of the agent.
• szAction. The action associated with the resource.
• szResource. The accessed resource.
• [szTransactionId]. A string that contains: idletime=.
• [nReason]. The reason associated with the event. Reasons are enumerated in Sm_Api_Reason_t, which is in SmApi.h.
• szStatusMsg. The message associated with the event. The message depends on the event type, as shown in in the following table:

For example:

AuthAccept testbox [27/Jun/2000:11:27:29 -0500] "190.158.4.90 uid=scarter,ou=people,o=airius.com" "testagent GET /test/index.html" [idletime=3600;maxtime=7200;authlevel=5;] [0]

In this example,

• lpszEvent is AuthAccept
• lpszHostName is testbox
• lpszTimeString is [27/Jun/2000:11:27:29 -0500]
• szClientIp is 190.158.4.90
• szUserName is uid=scarter,ou=people,o=airius.com
• szAgentName is testagent
• szAction is GET
• szResource is /test/index.html
• [szTransactionId] is [idletime=3600;maxtime=7200;
      authlevel=5;]
• [nReason] is [0]
• szStatusMsg is not specified.

Siteminder Impersonation

Definition: Impersonation is a method where a Priveleged User assumes the identity of another user,without losing the privileged users session.

Example:

Admin URL : www.xyz.com  

Target URL: www.xyz.com/imperso/targetURL

EndUser Application : www.xyz.com/application

Impersonator : Priveleged User who can assume the identity of another user

Impersonatee :A User whose identity is assumed by the Impersonator

Request Flow:

1)Impersonator logs into Admin URL www.xyz.com which is protected by Siteminder .

Siteminder creates a SMSESSION for the Impersonator.

2)Impersonator clicks on the Target URL: www.xyz.com/imperso/targetURL which initiates Impersonation Journey since the Target URL is protected by Impersonation Authentication Scheme

Impersonation Authentication Scheme prompts for Impersonatee’s user name

3)Impersonator lands to the Target URL  as a Impersonatee

Siteminder creates a SMSESSION for Impersonatee and saves the Impersonators session as SMSAVEDSESSION

4)Now Impersonator assumes the identity of Impersonatee and access the End User Application.

5)Impersonator clicks logout to end Impersonation journey which does the following,

SMSAVEDSESSION Cookie of Impersonator gets restored to SMSESSION and SMSAVEDSESSION returns a NULL value

 

Siteminder Configurations:

In the above example ,Consider Imperso.fcc,Impersologout.fcc and TargetURL are placed under www.xyz.com/imperso/

Realm 1:Create a realm for /imperso/

Authentication Scheme : Impersonation Auth Scheme [www.xyz.com/imperso/Imperso.fcc ]

Rules : Get/Post,ImpersoStart,ImpersoStartUser

Realm 2 for Admin URL[It will be an existing realm which creates a SMSESSION for the Impersonator]

Authentication Scheme: HTML Forms Authentication Scheme

Rules:Get/Post [existing Rules]

Realm 3 for EndUser Application :[Existing Realm]

Rules:Create a ImpersoStart and ImpersoStartUser Actions for all the each existing Rules.

Policies:

Impersonator Policy :

Add all the ImpersoStart Rules under this policy.

User Group:Only Impersonator

Impersonatee policy:

Add all the ImpersoStartUser Rules under this policy.

User Group:Only Impersonatee

Access policy:

Add Get/Post Rule from Realm 1

As part of Impersonation,Imperso.fcc file as to be protected,Since FCC files are ignored as part of IgnoreExt parameter in ACO,Create the following:

OverrideIgnoreExt = /Imperso.fcc

Hope this Documentation Helps!!!Any Queries please comment

SM Session Cookie

SMSESSION cookie represents user's session and contains:

- User's ID

- ID of the directory or database from which the user was authenticated

- SMSESSIONSPEC

- Unique Session ID (a hash of the GUID of the logged in user)

- other proprietary information

The Web Agent is responsible for validating an SMSESSION cookie's integrity, session expiration timeouts, and session revocations.

The session-spec field of the SMSESSION cookie is encrypted as it is transmitted by the Web Agent to the Policy Server to take decision about authentication and authorization. By decoding the SMSESSION cookie, you get the information about:

SM_AGENTAPI_ATTR_USERDN

SM_AGENTAPI_ATTR_SESSIONSPEC

SM_AGENTAPI_ATTR_SESSIONID

SM_AGENTAPI_ATTR_USERNAME

SM_AGENTAPI_ATTR_CLIENTIP

SM_AGENTAPI_ATTR_DEVICENAME

SM_AGENTAPI_ATTR_IDLESESSIONTIMEOUT

SM_AGENTAPI_ATTR_MAXSESSIONTIMEOUT

SM_AGENTAPI_ATTR_STARTSESSIONTIME

SM_AGENTAPI_ATTR_LASTSESSIONTIME

Troubleshoot the Siteminder Agent for Sharepoint

Introduction

Siteminder Web Agent for Sharepoint is a quite complicated product to implement.

You will face many issues while installing it for the first time or even after. So it’s important to manage all the tools to debug and troubleshoot all the issues you will face.

The Agent For Sharepoint has many different products bundled together so you will have many places to take a look in order to find the black sheep.

Enable / Disable logs

Affwebservices logs

Edit $AGENTHOME/Tomcat/webapps/affwebservices/WEB-INF/classes/LoggerConfig.properties
Change the value of :
TracingOn to Y
LoggingOn to Y

Claims web services log

Edit $AGENTHOME/Tomcat/webapps/ClaimsWS/WEB-INF/classes/LoggerConfig.properties
Change the value of :
TracingOn to Y
LoggingOn to Y

Apache HTTP Server logs

Open $AGENTHOME/httpd/conf/httpd.conf
Change value of :
JkLogLevel to debug
LogLevel to debug

Tomcat logs

Open $AGENTHOME/proxy-engine/conf/server.conf and change the value of :
loglevel to 4
After all of these modifications, please restart Agent For Sharepoint in order to make the changes effective.

Path of logs

Affwebservices logs

$AGENTHOME/proxy-engine/logs/federation.log (and its trace in the same directory)

Claims Web Service logs

$AGENTHOME/proxy-engine/logs/claimswebservice.log (and its trace in the same directory)

Apache HTTP Server logs

$AGENTHOME/httpd/logs/error_log

Tomcat logs

$AGENTHOME/proxy-engine/logs/server.log


Source : http://guireg.com/2012/11/01/troubleshoot-the-siteminder-agent-for-sharepoint/