Developing in WebSphere Message Broker Best practices

Developing in WebSphere Message Broker Best practices

WMB is a good integration software but you need to know some internals for developing suit solution.

1. Avoid using hard wired looping in the message flows, Use PROPAGATE statement instead.

When using hard wired looping in message flow All the resources of the loop participant are stored on the stack storage. Nodes , message trees, variables copied and held in memory.
Therefore while message processing the stack storage of the Execution Group (process) can ran out.
You also probably will get abends and core dumps.

By using the PROPAGATE statement the data won't be copied and stored during the loop iterations. After each iteration all the iteration data will be freed and re-used.

2. Work with reference variable and don't navigate through message tree directly.

SET OutputRoot.XML.Root.Child.Field1 = 1;
SET OutputRoot.XML.Root.Child.Field2 = 2;

For those simple assigning statements about 20 navigation action will be made. And it's only if the OutputRoot element is empty.
If not it can be much more.


If you have complex message transformation and performance is your concern work with Reference variable and Create statement.

DECLARE ref REFERENCE TO OutputRoot;
CREATE LASTCHILD OF ref as ref DOMAIN('XML') NAME 'XML';
CREATE LASTCHILD OF ref as ref NAME 'Root';
CREATE LASTCHILD OF ref as ref NAME 'Child';
CREATE LASTCHILD OF ref NAME 'Field1' VALUE '1';
CREATE LASTCHILD OF ref NAME 'Field2' VALUE '2';

We no using SET statement and since all CREATES specify relative keywords no navigation are required. But we got more code :-(.
You can use the ROW statement to get more code visibility and simplicity combined with SET statement.

remember to work with REFERENCE.

3. Avoid using Environment tree to handle local variables.

All data which stored in the Environment tree are stored in Memory during the whole message flow and not freed after it's out of scope.
so if you need space to store local data for node work with the LocalEnvironment tree. Thus avoiding high memory consumption.
Even if you delete the fields from the Environment tree it still won't free the memory till message flow end processing the message. The broker use pool allocation and reuse an pointed resources of the Environment Tree.

COALESCE Function

As we know,it is used to provide default values.It returns the first not null value.

SET OutputRoot.XMLNSC.COALESCE = COALESCE(InputRoot.XMLNSC.input1,0);
--SET OutputRoot.XMLNSC.COALESCE1 =
COALESCE(InputRoot.XMLNSC.input12,0);
-- <input1>12</input1>
------------------------------------------SET OutputRoot.XMLNSC.employee.(XMLNSC.Attribute)id ='123';
-------UUIDASCHAR,UUIDASBLOB---------------------SET OutputRoot.XMLNSC.emp = UUIDASCHAR;
--SET OutputRoot.XMLNSC.emp1 = UUIDASBLOB;
--------------------------------------------------DECLARE arg1,arg2 INTEGER;
--SET arg1 = InputRoot.XMLNSC.args.arg1;
--SET arg2 = InputRoot.XMLNSC.args.arg2;
--SET OutputRoot.XMLNSC.res.res1 = NULLIF(arg1,arg2);
--CASE WHEN arg1=arg2 THEN NULL ELSE arg1 END
--<args><arg1>2</arg1><arg2>2</arg2></args>


------------------------------------------------------------------------------------------------------------

PASSTHRU Function

The main use of the PASSTHRU function is to issue complex SELECTs, not currently
supported by the broker, to databases like GROUP BY or HAVING clauses which are not supported by ESQL.If we need to use these filter conditions,we can go for Passthru Function.

Ex:- SET OutputRoot.XML.Data.SelectResult.Row[] =
PASSTHRU('SELECT R.* FROM Schema1.Table1 AS R WHERE R.Name = ? OR R.Name =
? ORDER BY Name' TO Database.DSN1 VALUES ('Name1', 'Name4'));

------------------------------------------------------------------------------------------------------------

 PASSTHRU Statement

The main use of the PASSTHRU statement is to issue administrative commands to
databases like Creating/Dropping a table which needs extra privileges.

Ex 1:- PASSTHRU 'CREATE TABLE customer (
                                    CustomerNumber INTEGER,
                                    FirstName VARCHAR(256),
                                    LastName VARCHAR(256),
                                   Street VARCHAR(256),
                                   City VARCHAR(256),
                                   Country VARCHAR(256)
                             )' TO Database.sample;

Ex 2:- PASSTHRU 'DROP TABLE passthru' TO Database.sample;
                     DECLARE myVar CHARACTER;
                     SET myVar = 'SELECT * FROM sample';
                     SET OutputRoot.XMLNSC.root.Data[] = PASSTHRU(myVar);

Default Queues Created by Broker for Aggregate Nodes

•    SYSTEM.BROKER.AGGR.REQUEST
•    SYSTEM.BROKER.AGGR.CONTROL
•    SYSTEM.BROKER.AGGR.REPLY
•    SYSTEM.BROKER.AGGR.UNKNOWN
•    SYSTEM.BROKER.AGGR.TIMEOUT

Default Queues Created by Broker for Collector Nodes

•   SYSTEM.BROKER.EDA.COLLECTIONS
•   SYSTEM.BROKER.EDA.EVENTS 

DB Call in JAVA Compute Node in Message Broker



The main Purpose of this post is to hit a database(in my case DB2) table and retrieve the other details like Company,Salary on issuing the ID as an input.  For this,I have used 

MQInput node-->JavaCompute node-->MQOutput node

 The code in the JCN looks like this:-
.......................................................................
public void evaluate(MbMessageAssembly inAssembly) throws MbException {
MbOutputTerminal out = getOutputTerminal("out");
MbMessage inMessage = inAssembly.getMessage();
MbMessage outMessage = new MbMessage();
MbMessageAssembly outAssembly = new MbMessageAssembly(inAssembly,outMessage);
  try 
 {
   copyMessageHeaders(inMessage, outMessage);
   MbElement inRootElement = inMessage.getRootElement();
   List<MbElement>  idList =  (List) inRootElement.getLastChild().evaluateXPath("/ID");
   int id = (Integer)idList.get(0).getValue();
  
   //Creating Output data
   MbElement outRootElement = outMessage.getRootElement();
   MbElement messageType = outRootElement.getFirstChild().getFirstElementByPath("MessageType");
   messageType.setValue("Response");
   MbElement parser = outRootElement.createElementAsLastChild("MRM");
  
   // Conventional way of connecting to DB.
   Class.forName("com.ibm.db2.jcc.DB2Driver");  //Driver for DB2
   String url = "jdbc:db2://localhost:50000/TEST"; // Database name : TEST
   Connection conn = DriverManager.getConnection(url,"username","password"); 
     
   //Connection conn = getJDBCType4Connection("MYDB",JDBC_TransactionType.MB_TRANSACTION_AUTO); 
   // For this need to configure the configurable service.
  Statement stmt = conn.createStatement
  (ResultSet.TYPE_SCROLL_INSENSITIVE,  
   ResultSet.CONCUR_READ_ONLY);                                                 
  
   ResultSet resultSet = stmt.executeQuery("SELECT COMPANY, SALARY FROM MDEAI.EMP WHERE ID ="+id);
  
   while(resultSet.next())
   {
   parser.createElementAsLastChild(MbElement.TYPE_NAME_VALUE,"COMPANY",resultSet.getString("COMPANY"));
   parser.createElementAsLastChild(MbElement.TYPE_NAME_VALUE,"SALARY",resultSet.getInt("SALARY"));
   }
  
   out.propagate(outAssembly);
  
  } 
  catch (ClassNotFoundException e) 
  {
   e.printStackTrace();
  } 
  catch (SQLException e) 
  {
   e.printStackTrace();
  }
  finally 
  {
   // clear the outMessage even if there's an exception
   outMessage.clearMessage();
  }
 }





Input :
<Request><ID>1234</ID></Request>


Output :
<?xml version="1.0"?>
<Response xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<COMPANY>MICROSOFT</COMPANY>
<SALARY>999999</SALARY>
</Response>

Debugging WebSphere Message Broker flows in MB6

Step 1:- Assign debug port to execution group and enable debugging

You need to assign a debug port to the execution group you want the debugger to attach to and enable debugging. Please remember that this can constitute a security vulnerability since the debug port should be accessible from outside.

Step 2:- Next you should enable debugging;

Step 3:- Go to tthe command console and shoot;

mqsichangeproperties <broker-name> -e <execution-group-name> -o ComIbmJVMManager -n jvmDebugPort -v <port-number>

Step 4:- Restart the broker

Step 5:- Restart the broker toolkit

Step 6:- Create debug configuration

Create a new debug configuration by going to Debug… / Message Broker Debug / Add…

In the new configuration first click the “Select Execution Group…” which will result in the broker toolkit to query execution groups previously configured to discover the ones which have debugging enabled. Choose the execution group you have configured. Enter the debug port you have picked and add the Source folder to the debug configuration.

Step 7 :- Add breakpoints to you flow and use the debug configuration you have created to debug your flows.

Types of Logging in MQ

There are 2 types of Logs in MQ.There are:- 

1. Circular Logging
2. Linear logging

In WebSphere® MQ, the number of files that are required for logging depends on the file size, the number of messages you have received, and the length of the messages. There are two ways of maintaining records of queue manager activities: circular logging and linear logging.

·         

•  Circular logging

Use circular logging if all you want is restart recovery, using the log to roll back transactions that were in progress when the system stopped.

Circular logging keeps all restart data in a ring of log files. Logging fills the first file in the ring, then moves on to the next, and so on, until all the files are full. It then goes back to the first file in the ring and starts again. This continues as long as the product is in use, and has the advantage that you never run out of log files.

WebSphere MQ keeps the log entries required to restart the queue manager without loss of data until they are no longer required to ensure queue manager data recovery. 

·          

• Linear logging

Use linear logging if you want both restart recovery and media recovery (re-creating lost or damaged data by replaying the contents of the log). Linear logging keeps the log data in a continuous sequence of files. Space is not reused, so you can always retrieve any record logged in any log extent that has not been deleted.

As disk space is finite, you might have to think about some form of archiving. It is an administrative task to manage your disk space for the log, reusing or extending the existing space as necessary.

The number of log files used with linear logging can be very large, depending on your message flow and the age of your queue manager. However, there are a number of files that are said to be active. Active files contain the log entries required to restart the queue manager. Collectively, active log files are known as the active log. The number of active log files is usually less than the number of primary log files as defined in the configuration files.

The key event that controls whether a log file is termed active or not is a checkpoint. A WebSphere MQ checkpoint is a point of consistency between the recovery log and object files. A checkpoint determines the set of log files needed to perform restart recovery. Log files that are not active are not required for restart recovery, and are termed inactive. In some cases inactive log files are required for media recovery.

Inactive log files can be archived because they are not required for restart recovery. Inactive log files that are not required for media recovery can be considered as superfluous log files. You can delete superfluous log files if they are no longer of interest to your operation. 

                                 If a new checkpoint is recorded in the second, or later, primary log file, the first file can become inactive and a new primary file is formatted and added to the end of the primary pool, restoring the number of primary files available for logging. In this way the primary log file pool can be seen to be a current set of files in an ever-extending list of log files. Again, it is an administrative task to manage the inactive files according to the requirements of your operation.

                                   Although secondary log files are defined for linear logging, they are not used in normal operation. If a situation arises when, probably due to long-lived transactions, it is not possible to free a file from the active pool because it might still be required for a restart, secondary files are formatted and added to the active log file pool.

                                  If the number of secondary files available is used up, requests for most further operations requiring log activity will be refused with an MQRC_RESOURCE_PROBLEM return code being returned to the application.Both types of logging can cope with unexpected loss of power, assuming that there is no hardware failure.

Overview of Java Compute Node in MB

A JavaCompute node has three output terminals — out, alternate and failure — which enable routing to be performed through the out or alternate terminals. This procedure is not as flexible as the nodes that use ESQL for their processing. The Filter node and the enhanced WebSphere Message Broker V6
Compute node both have more than three output terminals. If you are using Java and require more than three output terminals. a full Java plug-in node would be required. Due to the similarities in structure between the code required for a JavaCompute node and a Java plug-in, it is easy to create a new Java plug-in node based on code previously developed for a JavaCompute node. Doing so can provide a new level of reuse for a team, since pre-built function can be added to a message flow developer’s palette in the form of a new plug-in node.If a JavaCompute node needs to communicate with a database, there are three ways to do it:-

1. MbSQLStatement class, which is part of the Java API specific to WebSphere Message Broker
and which lets a node access any ODBC datasources that WebSphere Message Broker can
connect to. This procedure provides full transactional support between a database and other
recoverable resources that support the XA protocol, such as WebSphere MQ.

2. JDBC type 2 or type 4 connection. In this case WebSphere Message Broker is not able to
provide transactional support for database operations.

3. SQLJ. A new facility that became available in WebSphere Message Broker V6 Fix Pack 1. In
this case WebSphere Message Broker is not able to provide transactional support for database
operations.

• Note-JDBC Type4 Driver is better than ODBC Driver(Type1) in performance.

Benefits of the JavaCompute node

1: Java may be the language of choice for an enterprise or may be a preferred option for productivity
reasons.
2:A key benefit of the JavaCompute node is the wide range of function available in the base Java
classes. In addition, a wide variety of third-party class libraries are available on the Internet.
3:An enterprise may have already developed Java class libraries to perform key business functions.
4:WebSphere Message Broker V6 ships with a number of sample message flows that demonstrate the
use of different class libraries. For example, one sample flow demonstrates basic message
transformation, and another shows how the JavaCompute node can be used to call the Google API.

Purpose of "PROMOTE PROPERTY" in MB:

If we want to reference this promoted property like for (ex.,queue name) because I need its value in different parts of the code and this value can change according the environment where flow is deployed. Also,There is no need to use any CONSTANT which constains same value.