Maximo Consultant's Blog

Category: java

Manipulating Chain of Java Extensions for the upgrade/installation process

17 June 2019 / / 2 Comments

 

I recently had to upgrade a pretty complex system. The original environment includes Maximo and ICD, and two large customization packages, one extended by the other (let’s call them package XXX extended by package YYY). The target system is the latest Maximo + ICD 7.6.1, plus 4 add-ons which include Oil & Gas and Utilities.

Customization was written by 3 different third parties over a long period of time and the source code was lost. This posed some challenges related to preserving customization and I had to spend a bit of time to figure it out. Below are some of the gotchas I learnt after the project:
 
Problem 1: Ensure customization is preserved after the upgrade

After reviewing the SMP folder, I found about 300 extended Java class files, but the product.xml files only cover about 20-30% of them; worse, some data are not even up-to-date. After the initial attempt to correct these files, I decided to simply ignore them, and build new product.xml files from scratch. Below are some of the key steps I had to do:
 
  • List out all extended java class files found in the SMP folder (using the command: dir /s /b  >list.txt) and put them in an Excel sheet
  • Use SQL to search from the DB any usage of custom Java code, put this list into another Excel sheet
  • Match the two sheets above to identify any missing files
  • Use a decompiler to view the code of each file to determine what is the original class it extends, and the type of the class (which I put into: Action, Bean, Cron, Event, Field, Object, Util)
  • For each of the class types, use Excel to generate an XML element as the follow examples:
  • After generating those elements, I put them together into newly created XML files.

Problem 2: Manipulating chain of extension without having Java source code:

The updated process took 6 hours, and after it finished and I can start Maximo. However, I freaked out when I looked into the MboSet class used by various objects. For example, in the WorkOrder object, the class used is psdi.pluss.app.workorder.PlusSWOSet. Initially, I thought the custom classes were wiped out after the upgrade. But after some investigation, I realized that due
to the upgrade and installation of new add-ons, Maximo has updated the classes (through the mean of binary code injection) to modify the chain of extension like this:

Before:  YYYWOSet > XXXWOSet > TloamWOSet > PlusPWOSet > WOSet


After:  PlusSWOSet > PlusGWOSet > YYYWOSet > XXXWOSet > TloamWOSet > PlusPWOSet > WOSet


After spending some time reading various IBM tech notes, I learnt that, in the past, we need to create a file with the exact name as: “a_customer.xml” to maintain metadata about customization. In the newer version (not exactly sure from what version, probably 7.5), we actually SHOULDN’T name it “a_customer.xml”. Because the file name makes it top of the list (in alphabetical order), and thus, becomes the first product to extend the core Maximo classes. For example, if you only have Core Maximo, Oil & Gas Add-on, and your custom package, if you name it a_customer.xml, the O&G package name is plusg, thus the extension chain would become: PlusGWOSet > CUSTOMWOSet > WOSet


If I like my custom class to be the last one that extends everything else, I actually should name it z_customer.xml, or anything that comes last in term of alphabetical
order. So I named the product XML files for the two custom packages z1_XXX
and z2_YYY.

For some unknown reasons, using just file name doesn’t give me the desirable outcome (probably due to  some undocumented logic), I had to use the <depends>
tag inside the two product XML files. From my experiment, by having the <depends> tag, the updatedb process will now ignore the file name rule, which means it doesn’t matter if you name it a_customer or z_customer anymore. The classes in your package will be extended after all packages listed in the <depends>
tag

To illustrate this point, below is the <depends> tag of my original z2_YYY.xml file:

It means the YYY package will extend the XXX package, then extend a bunch of packages (which included in the ICD product).

I updated the <depends> tag of the z2_YYY.xml file as follows:

Notice now I inserted three other packages before z1_XXX. After I updated the files, I ran updatedb process again (Even if there’s no update, updatedb will still update the java class files. With newer Maximo versions, you can run updatedblitepreprocessor.bat to do the same). With this, the updated process displays the product installation order as below:

Checking the class files, I got the extension in the desired order:

YYYWOSet > XXXWOSet > TloamWOSet > PlusSWOSet > PlusGWOSet > PlusPWOSet > WOSet

One more note, during the process, I found some of the files that get extended in an endless loop, for example:


YYYWOSet > PlusSWOSet > PlusGWOSet > TloamWOSet
> PlusPWOSet > XXX > TloamWOSet

This caused Maximo to fail when opening the app and crashed Eclipse when I tried to decompile the file. It turned out the reason is I used the wrong XML tag for the class type i.e. I should have used <Mbo….>  and <MboSet…> for object class rather
than <Class…> tag.

Another note is, due to some bugs or unknown logic, I had to play around a little with the product listed in the <depends> tag to get to the desired order as it doesn’t seem to work exactly as documented.

Hope this helps.

Deploy Maximo 7.6.1 on Websphere 8.5

15 August 2018 / / 7 Comments

Out of curiosity, I installed Maximo 7.6.1 on a VM to explore the new features. Then I found that I kind of like it, so I wanted to deploy it on my host OS to get rid of the VM. The trouble is I have Websphere 8.5.5.3 which comes with Maximo 7.6.0.0. I can run maxinst.bat to create demo data with Oracle 11g without a hiccup, but the deployment of the EAR file to Websphere didn’t go well.

It turned out Maximo 7.6.1 requires Java 1.8 and Websphere 9. It’s not possible to upgrade Websphere from version 8 to 9. We will have to install a new Websphere environment and migrate all of the settings which sounds like a lot of work. I decided to tweak it a little bit to make Maximo 7.6.1 runs on Websphere 8.5 using the following process:

1 – Update Installation Manager: Open IM, then open File > Preference, on the left menu, choose Update, check the Search for Installation Manager updates. Then click Ok. Then back to the main Installation Manager screen, click on “Install”, a wizard will pop up to download and update Installation Manager to the latest version (I got 1.8.9 by the time of writing this)

2 – Update Webshere: Open IM after it is updated, click on Update, choose Websphere Application Server 8.5, then follow the wizard. It will download and update Websphere with the latest fix packs. (I got 8.5.5.13)

3 – Install JDK 1.8: Open IM again, choose Install, then select ‘IBM SDK Java version 8’ to install it to the existing Websphere. After that, I got both Java 1.7 and Java 1.8 in my Websphere folder:

4 – Switch SDK version in Websphere: I used the [WAS_home]/bin/managesdk.cmd tool to change Websphere to use SDK 1.8 following the instruction here. To verify that MXServer uses the correct SDK version, I started MXServer and look at the SystemOut.log file to see the startup log:

5 – Upgrade deploytool: After the above step, I tried to deploy maximo.ear file and it failed. Turned out that the deployment tool of Websphere 8.5 doesn’t work with this new Maximo version. So I cheated by replacing the ‘deploytool’ folder of my Websphere 8.5 environment with the ‘deploytool’ folder I copied from the Websphere 9 environment I got when installing a fresh Maximo 7.6.1 instance in my VM. After that, the deployment process ran without any issues (and it is super fast compared to the previous version, yay!)

6 – Add new JMS queues: after maximo.ear is deployed, I tried to start MXServer but it failed, turned out there are new queues added to Maximo 7.6.1 which are not there with my existing Websphere environment, so I have to manually add those new queues and activation specifications to Websphere.

In summary for this step, I had to add new queues and activation specifications to Websphere. I basically mimic the settings from the Websphere environment I got in my VM. In the end, I added two queues: NOTF and NOTFERR, and two activation specifications: notfact, and notfacterr. Since I use Websphere MQ, I have to added two bus destinations in Websphere MQ as well. If you use the standard WAS default messaging provider, you simply add the two destinations to the ‘intjmsbus‘. I struggled with this step a little bit as the name of these artefacts are case-sensitive but I didn’t pay attention the first time I created those.

After this step, MXServer starts up without any issue and it works like a charm.

OutOfMemory error when deploying Maximo in Websphere

20 July 2018 / / 1 Comment

For every client I work with, I always keep a copy of the SMP folder if I can. And every time I need a standard demo instance with the same configuration as the client, I simply deploy it on my local Websphere and Oracle DB.

My current Websphere version is 8.5.5.3, and I had problems whenever I deploy a Maximo 7.6 instance with lots of add-ons. I gave up the attempt the last few times as I didn’t really need it.

Recently, as I was working with Maximo Flex (on the cloud), having a local instance is necessary as Maximo on the cloud has lots of limitations in terms of what you can do. As such, when I had the same problem deploying Maximo, I had to investigate and fix the issue.

OutOfMemory when deploying big Maximo.ear file to Websphere

Turned out it is an OutOfMemory issue during the deployment process and it took me quite a bit of time to fix it, so I leave the note here as I’m sure many others will have similar problems.

  • PROBLEM: when deploying Maximo 7.6 with many add-ons, Websphere keeps running or stops with a failure.
  • TROUBLESHOOT: if it takes more than 1 hour, I restart the system and re-deploy Maximo. In many cases, it immediately returns a failure. If it returns a failure, I look at the SystemOut.log file under 
[Websphere Home]/AppServer/profiles/ctgDmgr01/logs/dmgr/
  • SOLUTION: if the problem is an OutOfMemory error, it can be tricky as there are many different processes in Websphere using Java. I ended up employing the spray and pray approach:
    • Increase heap size for WAS Admin console to 2GB: edit the wsadmin.bat under [Websphere Home]/AppServer/bin to update this parameter: PERFJAVAOPTION=-Xms512m -Xmx1024m –Xquickstart
    • Increase Application Server heap size: in Websphere admin console, go to Servers section (on the left side menu) -> Server Types -> Websphere Applicaton Servers -> [MXServer] -> Java Process Management (link on the right side menu) -> Process Definition -> Java Virtual Machine. Set both Initial and Maximum heap size to 2048 MB
    • Increase deployment manager heap size: in Websphere admin console, go to System Administration section (on the left side menu) -> Deployment manager -> Java and Process management -> Process definition -> Java Virtual Machine. Set both initial and maximum heap size to 2048 MB
    • Increase heap for deploy tool: edit the ejbdeploy.bat file under [Websphere Home]/AppServer/deploy/tool/itp to update this parameter: EJBDEPLOY_JVM_HEAP=-Xms512  -Xmx1024m

With these updates, I managed to deploy the EAR file. But Maximo and Websphere alone consume 6GB of my laptop memory. I had to remind myself to reduce them back to probably 1024MB each to save valuable real estate.

MboSet performance, Memory Cache, and DB Call

6 June 2018 / / 1 Comment

 

I recently had to look at ways to improve the performance of a custom-built operation in Maximo. Essentially, it is one of the many validation operations take place after a user uploads a PO with a few hundred thousand lines.  The team here already built a high-performance engine to handle the process, but even with it, this particular operation still takes around 15-17 milliseconds to process each line which is too slow for their current standard. Imagine processing 200,000 PO lines, it will take nearly an hour just for this operation alone. There are a few dozen of these operations that need to be executed, plus other standard basic Maximo operations like status change or save, the whole process takes up many hours.
 
With this millisecond operation, many assumptions or standard recommendations for improving performance may not work. In some instances, following the standard recommendations actually make it slower. For example, many Maximo blogs suggest setting a MboSet to DISCARDABLE to reduce memory usage, but if the MboSet is re-used repeatedly, it will need to query from the DB again and again which can be costly. In this case, don’t set the MboSet to DISCARDABLE will allow Maximo to cache data in memory and thus make it faster. 
 
Usually, for performance tuning, a database query is the most expensive operation, and with Maximo, it has something to do with the getMboSet method. Therefore, a better insight into how it works will help us to devise the best strategy for writing code for high-performing operations. In this post, I’ll list out the findings of some of the experiments that I did around MboSet, hopefully will help save you some time if you have to deal with the same problem.

1. mboSet.count() versus getSize() versus isEmpty()
 
If we need to retrieve a MboSet via relationship such as itemSet = asset.getMboSet(“ALL_ITEM”), and we need to check if the result set is empty or not, we can use either itemSet.count() or .getSize() or .isEmpty(). The differences between them are:
  • count() will fire off a “Select count(*) From ITEM Where…” query; isEmpty() will fire off a “Select * From ITEM Where…” query.
  • getSize() will never fire off a DB query. It only returns the size of the mboSet which has been initialised in memory. Thus, if after the itemSet = asset.getMboSet(“ALL_ITEM”), we call cnt = itemSet.getSize(), we will get cnt = 0. After itemSet.moveFirst() is called, getSize will return cnt = 2 (because Maximo initialised one object ahead), and after we have looped through the whole itemSet using itemSet.moveNext(), calling itemSet.getSize() will return the actual size of the whole set.
  • isEmpty(), together with moveFirst(), or moveNext() or getMbo(i) function will call a “Select * From ITEM Where…” query if the itemSet never been initialized. But if the mboSet is already initialised, any subsequent call to those functions will just access the object in memory and thus doesn’t trigger another DB query.

Considering the following piece of code:

With this code, the mboSet.count() method will fire off a “Select count(*)” query. Then, when getMbo() method is called the first time, it will fire off a “Select * From…” query to initialise the mboSet. Although inside DB, the Select count(*) cost much less; from Maximo JVM, with a small table, it takes about the same amount of time (a few milliseconds) to execute the mboSet.count() and the mboSet.getMbo(i) method.  As such, the above piece of code will take about twice the amount of time to execute when compared to a loop that doesn’t use the count() method. This is consistent with the recommendation by Bruno in his blog.

However, considering a scenario when our logic only needs to check if a result set returned is empty or not, with a massive table, calling count() can be much faster than calling isEmpty().

2. Retrieve MboSet via a MXServer object versus via Relationship

When we retrieve a mboSet via relationship, the data will be cached in Maximo’s JVM memory. Thus, if the same relationship on the same mbo object is used again, it doesn’t fire off another DB query, thus improving performance significantly. However, if in our code, instead of using a relationship, we retrieve the mboSet via mxserver.getMboSet(), it will query the DB every time, thus could impact performance if the same mboSet can be re-used (when there is no change to the DB query). Therefore, in this case, it is advisable to retrieve the data via relationship instead. But be aware that in the case the same relationship is used, but with a different where clause, Maximo will have to call the DB to execute a new query, thus there will be no difference in terms of performance.

This leads to a scenario where you don’t want to add a new relationship to the object every time you need some specific operation. In this case, we can define a temporary relationship in our code as in the following sample:

itemSet = asset.getMboSet(“$TEMP_RELATIONSHIP”, “ITEM”, “status = ‘ACTIVE’”)

This will add a new temporary relationship during runtime, and as long as the where clause doesn’t change, the itemSet data is cached and thus the next time you call the getMboSet using this temporary relationship, data will be accessed from memory instead of being queried from the DB.

Sometimes, you will want to shorten your code using the dot notation in mbo.getString(), getInt(), or getBoolean() method etc., Maximo will try to use cached data instead of querying the database, thus considering the following piece of code:

The first getString statement will initialise the mboSet retrieved by the “ASSET_ITEM” relationship. Thus, subsequent getString statements don’t query the DB again. Because of that, in terms of performance, it will be similar to the following piece of code:

item = asset.getMboSet(“ASSET_ITEM”).getMbo(0) if (“ACTIVE”.equals(item.getString(“STATUS”))) { desc = item.getString(“DESCRIPTION”); orderunit = item.getString(“ORDERUNIT”) }

Using a relationship will help cache the data in memory, but if the data is only used once, all it does is increase memory usage and thus could potentially degrade performance instead of giving any benefit. Thus, you should be considering whether data can be re-used. If not, setting the mboSet to DISCARDABLE will ensure memory is freed up after use. So my recommendation for you to deal with performance issues is to always run your own test and make sure the right strategy is used for your scenario.

© 2024 Maximo Consultant's Blog

Theme by Anders Noren — Up ↑