Showing posts with label shell script. Show all posts
Showing posts with label shell script. Show all posts

Sunday, October 25, 2015

Send messages to IBM WebSphere MQ from Unix server using Shell script and JAVA

A standalone shell script to send messages to MQ on an IBM WebSphere MQ server using JAVA program.

The application mostly helps in testing connectivity from Metasolv M6 server to IBM WebSphere connectivity in MQ event generators in WLI.

In order to build this utility please follow below steps.

STEP 1. 


Create a alok_tmp(choose any name) in any directory on your unix box and create src and bin folder and below files.

Folders created inside directory i.e. /opt/app/script are as below

1. alok_tmp
2. alok_tmp/src
3. alok_tmp/bin

Files created inside directory src and bin

1. alok_tmp/sendMQ.sh
2. alok_tmp/src/MQSendUtility.java
3. alok_tmp/bin/requestMessage.xml
4. alok_tmp/bin/mq.properties

Below should be directory structure.


Class file will automatically be created after each time sh file is run.

STEP 2. 


Copy below to Java file to send the messages to MQ on IBM WebSphere server. Code below is self explanatory.

File: MQSendUtility.java

/** 
* This class provides access to the MQSeries queue facilities.  Once 
*
* instantiated, the main application program need only call the 'send'
*
* method to put message on request queue. The send method returns an MQMessage. 
*
* reply queue is still kept in class as earlier it was also used to receive the reply.
*
* but now it is now utilised to send a MQ message only. 
*
* @author Alok Mishra
*
* @version 1.0
*/

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import java.util.Properties;
import com.ibm.mq.MQC;
import com.ibm.mq.MQEnvironment;
import com.ibm.mq.MQException;
import com.ibm.mq.MQMessage;
import com.ibm.mq.MQPutMessageOptions;
import com.ibm.mq.MQQueue;
import com.ibm.mq.MQQueueManager; // Include the MQ package

public class MQSendUtility {
private static Properties props = new Properties();
private MQQueueManager qMgr;
private MQQueue requestQueue;
private MQQueue replyQueue;
public String replyQueueName;
private int openOptions;
private MQMessage storedMessage;

@SuppressWarnings("unchecked")
public MQSendUtility(String hostname, String channel, String qManager,
String aRequestQueue, String aReplyQueue) throws MQException {

try {
MQEnvironment.hostname = hostname;
MQEnvironment.channel = channel;
MQEnvironment.port = Integer.valueOf(props
.getProperty("mq.listen.port"));
MQEnvironment.userID = props
.getProperty("mq.server.security.principal");
MQEnvironment.password = props
.getProperty("mq.server.security.credentials");
MQEnvironment.properties.put(MQC.TRANSPORT_PROPERTY,
MQC.TRANSPORT_MQSERIES);
try {
qMgr = new MQQueueManager(qManager);
} catch (MQException e) {
print("An MQException occurred trying to connect to the QManager.");
Object o = e.exceptionSource;
print("MQException originated from object '" + o.toString());
print("Completion code = " + e.completionCode);
System.exit(1);
}
print("Connected to QManager " + qManager);

replyQueueName = aReplyQueue;

if (aRequestQueue.equals(aReplyQueue)) {
print("Open Options from IF");
openOptions = MQC.MQOO_INPUT_AS_Q_DEF | MQC.MQOO_OUTPUT;
} else {
print("Open Options from ELSE");
openOptions = MQC.MQOO_OUTPUT;
}
print("Open Options"+openOptions);
requestQueue = qMgr.accessQueue(aRequestQueue, openOptions, 
null, // default queue manager
null, // no dynamic q name
null); // no alternate user id

print("Got access to request queue "+requestQueue);

} catch (MQException ex) {
print("MQCommunicator.constructor - MQException occurred : Completion code "
+ ex.completionCode
+ "\n>MQStatus: Reason code "
+ ex.reasonCode + " mesage" + ex.getClass());
System.exit(1);
}

catch (Exception e) {
print("MQCommunicator.constructor - Exception occurred - "
+ e.toString());
System.exit(1);
}
}

public MQMessage send(String buffer) {

try {
MQMessage sendMessage = null;

try {
// Create new MQMessage object
sendMessage = new MQMessage();
} catch (NullPointerException e) {
print("Unable to create new MQMessage");
return null;
}
print("MQMessage created");

sendMessage.format = MQC.MQFMT_STRING;
sendMessage.messageType = MQC.MQMT_REQUEST;
sendMessage.replyToQueueName = replyQueueName;
sendMessage.writeString(buffer);
MQPutMessageOptions pmo = new MQPutMessageOptions();
try {
requestQueue.put(sendMessage, pmo);
} catch (NullPointerException e) {
print("Request Q is null - cannot put message");
return null;
}
print("Message placed on queue");
storedMessage = new MQMessage();
storedMessage.correlationId = sendMessage.messageId;
print("Message ID for sent message = "
+ sendMessage.messageId.toString());
print("Correlation ID stored = "
+ storedMessage.correlationId.toString());

return storedMessage;

} catch (MQException ex) {
print("MQCommunicator.send - MQException occurred : Completion code "
+ ex.completionCode + " Reason code " + ex.reasonCode);
return null;
} catch (java.io.IOException ex) {
print("MQCommunicator.send - IOException occurred: " + ex);
return null;
} catch (Exception ex) {
print("MQCommunicator.send - General Exception occurred: " + ex);
return null;
}
}

public void finalise() {
try {
// Closing the queues
requestQueue.close();

if (requestQueue != replyQueue)
replyQueue.close();

// Disconnect from the queue manager
qMgr.disconnect();

} catch (MQException ex) {
print("MQCommunicator.finalise - MQException occurred : Completion code "
+ ex.completionCode
+ ">MQStatus: Reason code "
+ ex.reasonCode);
}
}

public static void print(String msg) {
PrintWriter message = new PrintWriter(System.out, true);
message.println(">MQStatus: " + msg);
}

private static String getFileContent(String fileName) throws Exception {
File file = new File(fileName);
StringBuffer strBuffer = new StringBuffer();
try {
FileInputStream fileIn = new FileInputStream(file);
int lengthOfFile = (int) file.length();
byte[] outputByte = new byte[lengthOfFile];
// copy binary content to output stream
while (fileIn.read(outputByte) != -1) {
strBuffer.append(new String(outputByte));
}
fileIn.close();
} catch (Exception e) {
print("Exception FileInputStream" + e.getMessage());
e.printStackTrace();
throw e;
}
return strBuffer.toString();
}

public static void main(String args[]) throws Exception {


InputStream file = null;
String propertyFile = "mq.properties";
print("Load property File");
try {
file = MQSendUtility.class.getClassLoader().getResourceAsStream(
propertyFile);
props.load(file);
print("Loaded property File");
String filePath="requestMessage.xml";
String message = MQSendUtility.getFileContent(filePath);
String reqQueueName = props.getProperty("mq.request.queue");
String repQueueName = props.getProperty("mq.reply.queue");
String queueManager = props.getProperty("mq.queue.manager");
String channel = props.getProperty("mq.channel.name");
String qManagerUrl = props.getProperty("mq.provider.url");
print("Calling Constructor");
MQSendUtility util = new MQSendUtility(qManagerUrl, channel,
queueManager, reqQueueName, repQueueName);
MQMessage replyMsg = util.send(message);


} catch (IOException io) {
if (file == null)
print("No input file specified");
} catch (Exception e) {
print("Exception occured in main " + e.getMessage());
} finally {
if (file != null) {
try {
file.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

}

}

STEP 3. 


Copy below to shell script.

File: sendMQ.sh

#!/bin/ksh

#Execute Domain variables
. /opt/app/domains/my_domain/bin/setDomainEnv.sh

# Assuming domain variables don't include com.ibm.mq.jar so include it
CLASSPATH=$CLASSPATH:/opt/app/domains/my_domain/mq-jars/com.ibm.mq.jar
export CLASSPATH

cd /opt/app/script/alok_tmp
javac -d bin -sourcepath src src/MQSendUtility.java

cd /opt/app/script/alok_tmp/bin
java -Dbea.home=$BEA_HOME  MQSendUtility


STEP 4. 


Below is property file & request message file(XML).

File: mq.properties 

mq.provider.url=#IP Address of the IBM WebSphere MQ server i.e. 172.0.0.1
mq.channel.name=#Channel Name.This must be a valid channel on the IBM WebSphere MQ server.
mq.request.queue=# MQ Name where msg need to be put.This should be a valid queue for the specified queue manager.
mq.reply.queue=# reply queue name.This should be a valid queue for the specified queue manager.
mq.queue.manager=# This should be a valid queue manager on the IBM WebSphere MQ server i.e. qmgr123.
mq.listen.port=# Specifies the port of the IBM WebSphere MQ server.
mq.server.security.principal=#Specifies the username who has permission to use this channel.
mq.server.security.credentials=# Specifies the password for the given user.


File: requestMessage.xml

Hello, this is a test message being sent to MQ on IBM websphere server.

STEP 5. 


Now run sendMQ.sh and see output as below.




Hope above helps in testing Metasolv to WebSphere MQ connectivity by checking if M6 application server is able to send a message to MQ , please leave your feedback or query.


Wednesday, April 22, 2015

MetaSolv Weblogic Server/Datasource's monitoring and email notification through python scripts on cron tab

This is sometimes a common requirement to monitor the MetaSolv application servers(weblogic server instances) and datasources which are created at time of MetaSolv installation and other custom datasources to optimize the connection pool size. 

If your development/production WebLogic domain consists of multiple number (eg. 1 to 100 or 200 or so on) of managed servers then, it is difficult to see all server’s state and datasource's state at once. Weblogic Administration console is unable to show all the servers in the domain on a single page. Navigating in between also a time eating process so the monitoring through script will save lot of time and manual effort. 


The architecture of weblogic server differs from project to project. In my case I have 2 domains.


1. DOMAIN1

2. DOMAIN2


and each domain has an admin , a loadbalancer server and 4 managed server like DOMAIN1 has below


1. admin

2. loadBalancer
3. srvr_1.1(managed server) on unix box 1
4. srvr_1.2(managed server) on unix box 1
5. srvr_2.1(managed server) on unix box 2
6. srvr_2.2(managed server) on unix box 2

Similarly DOMAIN2 has the same configuration.


So now I am going to explain you how I am able to put an eye on server and datasource's properties and their state.


I will summarize it in below STEPs and later provide details for each STEP.


STEP 1. Monitoring of weblogic server instances and datasources through  ServerMonitor.py.

STEP 2. Ensure if SMTP is enabled on unix box.
STEP 3. Send email notification script MailServerState.py.
STEP 4. Encapsulate above scripts in a shell script ServerMonitoring.sh.
STEP 5. Put ServerMonitoring.sh on cron tab in unix box.


Now see the details in each STEP.



STEP 1. 


Create a python script to monitor all the servers in a weblogic domain. In the below script I am passing 4 parameters while calling it. Below parameters can be provided from a property file.

i)   User name for Admin server console

ii)  Password for Admin server console 
iii) URL Admin server
iv)  Name of the Domain

The o/p of the file is written in file status.log (/opt/app/Scripts/JMS/status.log) which we can use while sending email notification.



To get the status and other properties of all servers and the datasources in each server in the domain can be obtained with the following steps

1. Connect to the Admin Server
2. Navigate to the domain runtime MBean hierarchy by entering the domainRuntime command domainRuntime().
3. Navigate to ServerRuntimes to fetch the server list.
4. ServerRuntimeMBean provides methods for retrieving runtime information about a server instance.
5. Pass the serverlist to reportDomainHealth() method which iterate the loop and get the state and other mbeans(other run time mbeans like JVMRuntime, ThreadPoolRuntime) of each admin/managed server to provide monitoring properties for each server.
6. Come back to main method and pass the serverlist to reportDataSourceHealth().
7. All the datasource properties can be fetched from ServerRuntimes-->JDBCServiceRuntime-->JDBCDataSourceRuntimeMBeans mbean hieracrhy.
8. In below script it will only display the datasource details for all such datasource which are not running. In case all are working fine it will simply display "All the Datasources in instance are running fine."
9. Comeback to main method and finally disconnect.



file:ServerMonitor.py

# Purpose: To collect server and datasource statistics and save it to a log file

# Author: Alok Mishra

from java.util import Date

import os
adminUser=sys.argv[1]
print adminUser
adminPassword=sys.argv[2]
print adminPassword
adminURL=sys.argv[3]
print adminURL
server=sys.argv[4]
print server

def reportDomainHealth(serverList):

 cmd = "echo 
\================================================================================================================================================================================================ 
>>/opt/app/Scripts/JMS/status.log"
 os.system(cmd)
 cmd = "echo \"%25s %10s %15s %10s %10s %20s %10s %10s %50s" % ("Server","Threads","HoggingThreads", "IdleThreads", "QueueLength", "ThroughPut" ,"Heap_Free","State","HealthState ")+"\" 
>>/opt/app/Scripts/JMS/status.log"
 os.system(cmd)
 cmd = "echo ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 
>>/opt/app/Scripts/JMS/status.log"
 os.system(cmd)
 for i in range(len(serverList)):
  if serverList[i] != 'dr--':
   server_st=get('ServerRuntimes/' + serverList[i] + '/HealthState')
   server_state=get('ServerRuntimes/' + serverList[i] + '/State')
              server_tc=get('ServerRuntimes/' + serverList[i] + '/ThreadPoolRuntime/ThreadPoolRuntime/ExecuteThreadTotalCount')
              server_hog=get('ServerRuntimes/' + serverList[i] + '/ThreadPoolRuntime/ThreadPoolRuntime/HoggingThreadCount')
              server_itc=get('ServerRuntimes/' + serverList[i] + '/ThreadPoolRuntime/ThreadPoolRuntime/ExecuteThreadIdleCount')
          server_ql=get('ServerRuntimes/' + serverList[i] + '/ThreadPoolRuntime/ThreadPoolRuntime/QueueLength')
   server_thru=get('ServerRuntimes/' + serverList[i] + '/ThreadPoolRuntime/ThreadPoolRuntime/Throughput')
   server_hpfp=get('ServerRuntimes/' + serverList[i] + '/JVMRuntime/' + serverList[i] + '/HeapFreePercent')
   cmd ="echo \"%25s %10s %15s %10s %10s %20s %10s %10s %80s" % (serverList[i],str(server_tc),str(server_hog),str(server_itc),str(server_ql),str(server_thru),str
(server_hpfp)+'%',str(server_state),str(server_st))+"\" >>/opt/app/Scripts/JMS/status.log"
   os.system(cmd)
     cmd = "echo 
\================================================================================================================================================================================================ 
>>/opt/app/Scripts/JMS/status.log"
     os.system(cmd)

def reportDataSourceHealth(serverList):

 for i in range(len(serverList)):
  if serverList[i] != 'dr--':
   cmd = "echo \"\t\t\t\t\t\t\t\t\tDataSource Monitoring Details on server:"+serverList[i]+" @ " + today.toString() + " \n\n"+ "\" >>/opt/app/Scripts/JMS/status.log"
   os.system(cmd)
   check="false"   
   dataSourceList=ls('ServerRuntimes/' + serverList[i] +'/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans');
   dataSourceList=dataSourceList.split()  
   for j in range(len(dataSourceList)):
    if dataSourceList[j] != 'dr--':
     ds_name=get('ServerRuntimes/' + serverList[i] + '/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans/'+dataSourceList[j]+'/Name')
     ds_state=get('ServerRuntimes/' + serverList[i] + '/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans/'+dataSourceList[j]+'/State')
     ds_capacity=get('ServerRuntimes/' + serverList[i] + '/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans/'+dataSourceList
[j]+'/CurrCapacityHighCount')
     ds_accc=get('ServerRuntimes/' + serverList[i] + '/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans/'+dataSourceList
[j]+'/ActiveConnectionsCurrentCount')
     ds_achc=get('ServerRuntimes/' + serverList[i] + '/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans/'+dataSourceList
[j]+'/ActiveConnectionsHighCount')
     ds_wshc=get('ServerRuntimes/' + serverList[i] + '/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans/'+dataSourceList
[j]+'/WaitSecondsHighCount')
     ds_wfccc=get('ServerRuntimes/' + serverList[i] + '/JDBCServiceRuntime/' + serverList[i] + '/JDBCDataSourceRuntimeMBeans/'+dataSourceList
[j]+'/WaitingForConnectionCurrentCount')
     if (str(ds_state)!="Running"):
      if (check!="true"):
       cmd = "echo \"%25s %15s %20s %30s %20s %20s %25s" %("Name", "State", "Capacity", "ActiveConnectionsCurrentCount", 
"ActiveConnectionHighCOunt", "WaitSecondsHighCount", "WaitingForConnectionCurrentCount") +"\">>/opt/app/Scripts/JMS/status.log"
       os.system(cmd)
       check="true"
      cmd = "echo \"%25s %15s %20s %30s %20s %20s %25s" %(str(ds_name), str(ds_state), str(ds_capacity), str(ds_accc), str(ds_achc), str(ds_wshc), str
(ds_wfccc)) +"\">>/opt/app/Scripts/JMS/status.log"
      os.system(cmd)
   if(check=="false"):
    cmd = "echo \"All the Datasources in instance are running fine."+ "\" >>/opt/app/Scripts/JMS/status.log"
    os.system(cmd)
   
 cmd = "echo 
\================================================================================================================================================================================================ 
>>/opt/app/Scripts/JMS/status.log"
 os.system(cmd)

today=Date()

if __name__== "main": 
 cmd = "echo \"\t\t\t\t\t\t\t\t\t"+server+" Server Monitoring Details: @ " + today.toString() + " \n\n"+ "\" >>/opt/app/Scripts/JMS/status.log"
 os.system(cmd)
 connect(adminUser,adminPassword,adminURL)
     domainRuntime()
     serverList=ls('ServerRuntimes');
     serverList=serverList.split()
 reportDomainHealth(serverList)
 reportDataSourceHealth(serverList)
 disconnect()


The o/p of script should look as below.





STEP 2. 


Confirm if your email service (SMTP) is working fine.


Run below command and verify the o/p


1. Generally port 25 is reserved for SMTP and to check if port 25 is reserved for SMTP and SMTP is enabled run below command.


telnet localhost 25


if SMTP is enabled you should get below response




Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.


Else you will get below



if SMTP is disabled please go to step 2 and further steps.



2. If SMTP is not enabled run below command


svcadm enable /network/smtp


3. To confirm if SMTP service is running run below comand


svcs -a | grep smtp


4. From command 3 if you get below response 

online         11:04:28 svc:/network/smtp:sendmail

Then your SMTP is enabled


5. Try running below command and you should get email.


echo "something" | mailx -s "hello" -r "abc@xyz.com"--(enter your email if here)



So now by now you know that email service is working on your machine.



STEP 3. 


Create a python script which will send the status.log file attaching in an email with some generic message to notify the team. But before that ensure that SMTP service is configured on your unix box(STEP 2).

Here in the following script you can replace the 'receiver' and 'To' address value given as  abc@xyz.com with mailing address of your supporting MetaSolv monitoring team list by semicolon separation. 


You can also modify below script according to your requirement. If your requirement is to send email only in case of server state is SHUTDOWN or UNKNOWN in such case you can get the server state in below script(refer from above script) and put conditions here. But my requirement is to send email with irrespective of server state after every certain amount of time.


1 thing I want to highlight here I had problems with base64 import so I print the classpath after import of smtplib and check which jython-modules.jar from classpath location it is being taken. As the base64.py in your jython-modules.jar may not have method encodestring. So please ensure it and use method mentioned in your base64.py. 


Below code is quite self explanatory so in case you have issues understanding it , please leave comment.




file:MailServerState.py

#!/usr/bin/python

# Purpose: Send email for server status
# Author: Alok Mishra
import smtplib
import base64
import time
Date = time.ctime(time.time())
filepath = "/opt/app/Scripts/JMS/status.log"
filename= "status.log"

# Read a file and encode it into base64 format

fo = open(filepath, "rb")
filecontent = fo.read()
encodedcontent = base64.encodestring(filecontent)  # base64
sender = 'MetaSolvAdministrator'
reciever = 'abc@xyz.com'
marker = "AUNIQUEMARKER"
body ="""
Hi All, \n\nPlease find the monitoring details for all MetaSolv domains as attached.\n\n
Regards\nThis is an automated email notification. \nPlease do not reply. Contact Application Support Team."""
# Define the main headers.
part1 = """From: MetaSolvAdministrator
To: abc@xyz.com
Subject: WLS status for all M6 Domains of PROD Environment @ %s
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=%s
--%s
""" % (Date,marker, marker)

# Define the message action

part2 = """Content-Type: text/plain
Content-Transfer-Encoding:8bit
%s
--%s
""" % (body,marker)

# Define the attachment section

part3 = """Content-Type: multipart/mixed; name=\"%s\"
Content-Transfer-Encoding:base64
Content-Disposition: attachment; filename=%s
%s
--%s--
""" %(filename, filename, encodedcontent, marker)
message = part1 + part2 + part3

try:

   smtpObj = smtplib.SMTP('localhost')
   smtpObj.sendmail(sender, reciever, message)
   print "Successfully sent email"
except Exception:

   print "Error: unable to send email"


STEP 4.  


Automate every process mentioned above using a shell script.

to Automate the above process I wrote a schell script which will run both the python files and provide the necessary parameters from property file.



Script will do below.


1. Calls the SetWLSEnv.sh to set the enviornment to execute python files.

2. Read the domain.property file.
3. Call the python scripts.



 file: ServerMonitoring.sh

# Purpose: To call python scripts for servermonitoring for all the servers

# Author: Alok Mishra
#! /bin/sh
WL_HOME="/opt/app/mss/bea/wlserver_10.3"
# set up class path
. "${WL_HOME}/server/bin/setWLSEnv.sh"
. /opt/app/Scripts/JMS/domain.properties

echo $username

echo $password
echo $DOMAIN1AdminURL
echo $DOMAIN2AdminURL

cat /dev/null > /opt/app/mssnas/m6config/Scripts/JMS/status.log

java weblogic.WLST /opt/app/Scripts/JMS/ServerMonitor.py $username $password $DOMAIN1AdminURL DOMAIN1
java weblogic.WLST /opt/app/Scripts/JMS/ServerMonitor.py $username $password $DOMAIN2ADMINURL DOMAIN2


Also have a look at domain.property file.

provide user/password and other URLs as per your configurations. 




username=weblogic
password=weblogic
DOMAIN1AdminURL=t3://adminserverURLfor domain1:adminserverPort for domain1
DOMAIN2AdminURL=t3://adminserverURLfor domain1:adminserverPort for domain2


You can save the files as per you convenience but in my case all the files are put together.





STEP 5. 


The Final step is to put the shell script ServerMonitoring.sh on cron job in unix box so that out email can regularly be sent to desired recipients.

to do that do below


1. To open a VI style editor.

crontab -e
2. Write below 
0 0,4,8,12,16,20 * * *  /opt/app/Scripts/JMS/ServerMonitoring.sh
3. Save and exit the editor.

Here completes the whole activity of monitoring weblogic servers and datasources. Now the mentioned email addresses will start receiving the email notifications with attachment status.log in their email after every 4 hours starting 00:00 AM in night which means 6 emails/per day.


Hope above helps in your server monitoring tasks, please leave your feedback or query.