SBL-SVR-01015

Applies to:

Siebel Email Response - Version 7.8 [19038] to 8.1.1.8 SIA [23012] [Release V7 to V8]
Information in this document applies to any platform.

Symptoms

On Siebel 8.1.1.3 using Communications Outbound Manager component to send out Outbound Requests.

When Siebel tries to send out outbound requests using CommOutboundMgr component it fails to send the emails. The following errors can be observer on Communication Outbound Manager component task for the outbound request with event trace log level increased:
SrmRouting SrmRoutingSubevent 4 0000006d50341600:0 2012-08-22 01:00:45 SNDREQ
 IDS  : FLAGS [1] : REQID [2-4OFLL-1BFG] : SRBSESSID [a5f315e8200003SBLGW] : SRMCONNID [11]
 SRC  : SRC_ID [] : SRC_KEY []
 DEST : ROUTE_ID [FSMSrvr] : ROUTE_KEY [] : ROUTE_SRVR [] : LANG_CODE [ENU] : CODE_PAGE [1200]
GenericLog GenericError 1 0000006d50341600:0 2012-08-22 01:00:45 (srmconn.cpp (2571) err=1180682 sys=0) SBL-NET-01034: The SISNAPI connection was closed by the peer.
GenericLog GenericError 1 0000006d50341600:0 2012-08-22 01:00:45 (srmconn.cpp (2313) err=1311735 sys=0) SBL-SVR-01015: Could not send message due to (null)
GenericLog GenericError 1 0000006d50341600:0 2012-08-22 01:00:45 (fsmclientstub.cpp (355) err=1311735 sys=0) SBL-SVR-01015: Could not send message due to (null)
GenericLog GenericError 1 0000006d50341600:0 2012-08-22 01:00:45 Not connected to FSM
CommSrvr CommSrvrWarning 2 0000006d50341600:0 2012-08-22 01:00:45 Cannot open request state file CommOutboundMgr2-11DPAN3.CommSrvr.crf, error 1.
GenericLog GenericError 1 0000006d50341600:0 2012-08-22 01:00:45 Not connected to FSM
CommSrvr CommSrvrWarning 2 0000006d50341600:0 2012-08-22 01:00:45 Cannot remove request state file CommOutboundMgr2-11DPAN3.CommSrvr.crf, error 1.
After this error all other Outbound Request processed by this CommOubountMgr component instance fail with similar error:
GenericLog GenericError 1 0001a54c5033249c:0 2012-08-22 01:01:29 Not connected to FSM
GenericLog GenericError 1 0001a54c5033249c:0 2012-08-22 01:01:29 Not connected to FSM
CommSrvr CommSrvrWarning 2 0001a54c5033249c:0 2012-08-22 01:01:29 Cannot open request state file CommOutboundMgr2-11DPAMA.CommSrvr.crf, error 1.
GenericLog GenericError 1 0001a54c5033249c:0 2012-08-22 01:01:29 Not connected to FSM
CommSrvr CommSrvrWarning 2 0001a54c5033249c:0 2012-08-22 01:01:29 Cannot remove request state file CommOutboundMgr2-11DPAMA.CommSrvr.crf, error 1.
The only way to recover is to restart the CommOutboundMgr component.

Cause

The cause of the problem is due a communication problem between the CommOutboundMgr and FSMSrvr component through SRBroker component.
When processing Outbound Request a CommOutboundMgr component instance opens a persistent connection with FSMSrv. In case this connection is broken the CommOuboundMgr keep trying to use the broked connection through SRBroker componet and the outbound request fail when it tries to create the internal control file "CommOutboundMgr<Outbound Request Id>.CommSrvr.crf" file in Siebel File System.
The connection could be broke due a FSMSrvr component crash or restarted or a SRBroker crash or restart for example.

Solution

Once the CommOutboundMgr persistent connection with FSMSrvr component is broken, the only way to recover it is restarting the CommOutboundMgr component instance that lost the connection with FSMSrvr.
BUG:10577765 - "[CR#12-1V7F8QN][FR#12-1V7F8R9] PROVIDE FSM RECONNECT FUNCTIONALITY FOR OBJECT" type Enhancement Request was opened to provide the ability to Communication Outbound Manager component to recover from a broken persistent connection for synchronous server requests to FSMSrvr but has not been implemented.
Another BUG:14305830 - COMMUNICATIONS OUTBOUND REQUESTS GIVING ERROR: SBL-CMO-00165 has been opened and a Quick Fix is available for Siebel 8.1.1.5 in order to CommOutboundMgr recover the connection with FSMSrvr.
Patch Number : 14647573 is available on My Oracle Support for 8.1.1.5 Quick Fix 05BU.
The BUG:14305830 has also been opened and target for a permanent fix on Siebel 8.1.1.10 Fix Pack.
In this particular case the Service Request Broker (SRBRoker) component was crashing sporadically on customer environment causing the problems to CommOutboundMgr.

Applies to:

Siebel System Software - Version 7.8.2.2 [19219] and later
z*OBSOLETE: Microsoft Windows Server 2003
Product Release: V7 (Enterprise)
Version: 7.8.2.2 [19219]
Database: Oracle 9i
Application Server OS: Microsoft Windows 2003 Server SP1
Database Server OS: Sun Solaris 9

CHECK ON CURRENCY 2-NOV-2012

Symptoms

SBL-SVR-01015, SBL-SVR-01043, SBL-SMI-00033, SBL-NET-01034, SBL-NET-01033, SBL-ADM-01044, SBL-SCB-00015, SBL-SCB-00011
Each night a cold DB backup is taken on each environment. This is affecting a number of components on the Siebel server:

Transaction Processor
Transaction Merger
Transaction Router
Server Tables Cleanup

These components need to be restarted each morning as the backup is affecting them. Hence scripts were used to shutdown the siebel server 15 mins before backup is taken and another to restart it in the morning. On another environmnet the shutdown script appears to hang (i.e batch job does not complete). All the parameters are correct as the command would not run otherwise.

Cause

Configuration/ Setup

Solution

The customer's script:

D:\sea78\test\siebsrvr\BIN\srvrmgr /g UKGateway /e siebtest /s SiebTest1 /u sadmin /p password /c "shutdown appserver SiebTest1"

In review of the customer's component log files, could see that there was a problem with one or more components shutting down. The "shutdown appserver" command issued from the Server manager command-line is a synchronous process, thus the command will block until all tasks in the server have exited. There could be a problem stopping one or more components, caused either by the components themselves, or caused by the synchronous nature of the shutdown process (ie. one component is waiting on a response from another component, or the Siebel server - that may have already been shutdown).

The customer was recommended to use the Windows command, net stop. The net stop command is used as follows:

net stop <service>

For example:

net stop SiebSrvr

Which would stop the service called SiebSrvr.

Assuming you do not have any problem stopping your Siebel Server service using the Windows' Services utility, then you should not experience a problem with using the net stop command to stop the Siebel Server service.