Open Liberty application server and jandex version

Question

Open Liberty application server and jandex version

216 views Asked by stefan.m At 03 January 2023 at 21:44

I want to improve the startup time of my Open Liberty application using Jandex, as described in https://www.ibm.com/support/pages/node/745421 and https://www.ibm.com/support/pages/using-jandex-ibm-websphere-liberty-setting-configuration-enable-jandex-index-reads.

However, it seems that Open Liberty (version 21.0.0.6) is using a rather old version of Jandex. I only managed to get it running with indexes that were created with com.ibm.ws.org.jboss.jandex_1.0.53.jar (containing a Jandex 2.1.2 version), while the current version is Jandex 3.0.5.

I am aware that I might upgrade to OpenLiberty 22.0.0.13 which is the latest version, but this also only contains a Jandex version 2.x.

Is there any way to have OpenLiberty use Jandex 3 version? I assume that simply replacing the com.ibm.ws.org.jboss.jandex_1.0.x.jar file with the original Jandex Jar file will lead to incompatibilities, so is there any other chance?

Edit 2023-01-25: In answer to Thomas Bitonti's reply, here is an excerpt of the startup time of the application when Jandex is active, also showing how slow the Jandex operation is.

Use case one: Jandex disabled: Application startup time: 31 seconds

Use case two: Jandex enabled, first start. Application startup time: 35 seconds

Number of Jars and classes according to OLP Jandex log:

I CWWKC0093I: Jandex coverage of module e2e-leasing-ui-war in application e2e_leasing2_ui_war_war: Jandex indexes were read for 131 out of 298 module locations; Jandex indexes provided 5,118 out of 48,102 module classes.

a more detailed log of use case 2 follows at the end of the question

Use case three: Jandex enabled, second start. Application startup time: 33 seconds

According to the log, the read process of indexes is somehow different, especially if you look at the first line. That might be due to the custom cache mentioned by Thomas?

[1/25/23 23:35:37:730 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module e2e-leasing-ui-war in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 298 module locations; Jandex indexes provided 0 out of 0 module classes.
[1/25/23 23:35:38:562 CET] 00000030 org.apache.cxf.management.jmx.InstrumentationManagerImpl     I registering MBean org.apache.cxf:bus.id=e2e-leasing-ui-war-Server-Bus,type=Bus,instance.id=1043806874: org.apache.cxf.bus.ManagedBus@6a547ce6
[1/25/23 23:35:38:572 CET] 00000030 org.apache.cxf.management.jmx.InstrumentationManagerImpl     I registered org.apache.cxf:bus.id=e2e-leasing-ui-war-Server-Bus,type=Bus,instance.id=1043806874
[1/25/23 23:35:38:610 CET] 00000030 org.apache.cxf.management.jmx.InstrumentationManagerImpl     I registering MBean org.apache.cxf:bus.id=e2e-leasing-ui-war-Server-Bus,WorkQueueManager=Bus.WorkQueueManager,type=WorkQueueManager,instance.id=154287288: org.apache.cxf.bus.managers.WorkQueueManagerImplMBeanWrapper@ee8e0e7
[1/25/23 23:35:38:886 CET] 00000030 org.jboss.weld.Version                                       I WELD-000900: 3.1.4 (Final)
[1/25/23 23:35:39:902 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/classgraph-4.8.78.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes.
[1/25/23 23:35:39:924 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jrap-commons-config-startup-2.123.13.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 3 out of 3 module classes.
[1/25/23 23:35:39:927 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jackson-defensive-2.119.1-LTS.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes.
[1/25/23 23:35:39:929 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/termin-business-api-0.58.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes.
[1/25/23 23:35:40:392 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/produktantrag-business-api-2.2.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 102 out of 102 module classes.
[1/25/23 23:35:40:395 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/xz-1.9.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes.
[1/25/23 23:35:40:396 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/tika-parser-apple-module-2.3.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes.
[1/25/23 23:35:40:402 CET] 00000030 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/journal-client-jrap2-1.63.1.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 17 out of 17 module classes.
...

I attach a more detailed (but of course abbreviated) log of use case two, showing that reading the Jandex index seems to take 14 seconds - a bit too long, in my opinion:

 ride">[1/25/23 23:31:44:383 CET] 00000028 com.ibm.ws.annocache                                         I CWWKC0093I: Jandex coverage of module e2e-leasing-ui-war in application e2e_leasing2_ui_war_war: Jandex indexes were read for 131 out of 298 module locations; Jandex indexes provided 5,118 out of 48,102 module classes. ement.jmx.InstrumentationManagerImpl     I registering MBean org.apache.cxf:bus.id=e2e-leasing-ui-war-Server-Bus,type=Bus,instance.id=1045305938: org.apache.cxf.bus.ManagedBus@205fda98 ement.jmx.InstrumentationManagerImpl     I registered org.apache.cxf:bus.id=e2e-leasing-ui-war-Server-Bus,type=Bus,instance.id=1045305938 ement.jmx.InstrumentationManagerImpl     I registering MBean org.apache.cxf:bus.id=e2e-leasing-ui-war-Server-Bus,WorkQueueManager=Bus.WorkQueueManager,type=WorkQueueManager,instance.id=1435508220: org.apache.cxf.bus.managers.WorkQueueManagerImplMBeanWrapper@2d8d0bdb I WELD-000900: 3.1.4 (Final) I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/classgraph-4.8.78.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 228 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jrap-commons-config-startup-2.123.13.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 3 out of 3 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jackson-defensive-2.119.1-LTS.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 30 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/termin-business-api-0.58.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 19 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/produktantrag-business-api-2.2.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 102 out of 102 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/xz-1.9.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/tika-parser-apple-module-2.3.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 25 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/journal-client-jrap2-1.63.1.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 17 out of 17 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/vorbis-java-tika-0.8.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 8 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/istack-commons-runtime-3.0.12.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 22 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jsoup-1.11.3.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 248 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/efw-services-jrap-drb-client-1.0.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 351 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jrap-commons-bankdaten-1.7.16.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 19 out of 19 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/pdfbox-tools-2.0.15.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 29 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/e2e-leasing-ui-services-5.36.12-SNAPSHOT.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 185 out of 185 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/boilerpipe-1.1.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 76 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/json-20141113.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 31 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/sbi-commons-security-1.7.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 6 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/httpcore-4.4.4.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 254 module classes. I CWWKC0093I: Jandex coverage of module null in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 587 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/fontbox-2.0.15.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 157 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jrap-zsvz-client-jrap2-3.0.1.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 15 out of 15 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/discovery-service-client-jaxrs-jrap2-1.63.1.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 7 out of 7 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/crm-services-client-3.38.0-SNAPSHOT.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 684 out of 684 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/kundendokumente-services-client-jrap1-1.5.8.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 181 out of 181 module classes.classes. I CWWKC0093I: Jandex coverage of module null in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 117 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jrap-soa-jca-2.123.21.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 122 out of 122 module classes. I CWWKC0093I: Jandex coverage of module null in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 48 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jrap-soa-ui-services-2.123.21.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 5 out of 5 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/failureaccess-1.0.1.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 2 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/wiedereinstieg-business-client-jaxrs-jrap2-2.2.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 1 out of 1 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/e2e-ng-business-jaxrs-ui-12.15.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 231 out of 231 module classes. I CWWKC0093I: Jandex coverage of module null in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/user-service-api-1.51.10.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 58 out of 58 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/e2e-security-1.103.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 11 out of 11 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/discovery-service-discovering-jaxrs-jrap2-1.63.1.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 1 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/commons-logging-1.2.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 28 module classes. I CWWKC0093I: Jandex coverage of module null in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 587 module classes. I CWWKC0093I: Jandex coverage of module null in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 68 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/termin-business-api-0.58.0.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jackson-defensive-2.119.1-LTS.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/jrap-commons-config-startup-2.123.13.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes. I CWWKC0093I: Jandex coverage of module /WEB-INF/lib/classgraph-4.8.78.jar in application e2e_leasing2_ui_war_war: Jandex indexes were read for 0 out of 1 module locations; Jandex indexes provided 0 out of 0 module classes.

Original Q&A

There are 2 answers

**Thomas Bitonti** · Answer 1 · 2023-01-24 21:03:41

Open Liberty currently reads Jandex indexes in preference to doing a scan when discovering application annotations. The current code should be able to read index formats up to the latest V2 (2.4.3.Final) Jandex version. Can you provide a sample JAR + Jandex 2 index which isn't being read?

Up to at least v23.0.0.2, Open Liberty does not read Jandex 3 indexes.

From a performance perspective, there are several cases:

Starting an application for the first time, with no Jandex indexes.
Starting an application for the first time, with Jandex indexes.
Starting an application for the second (or subsequent) time, with or without Jandex indexes.

The first case is slowest. In this case, Liberty uses ASM to scan the entire application. Liberty does its best to scan quickly, but, inherently, the scan is costly since all application classes must be read and examined.

The second case is faster. In this case, Liberty does a special read of the Jandex indexes. This is not quite as fast as reading the Liberty internal index format, but should be faster than doing a whole new scan (the first case).

The third case will be fastest. In this case, Liberty reads annotations data from a custom cache format which is tuned to the Liberty annotations discovery requirements.

(There are other cases relating to restarts after making application updates.)

For your testing, can you provide information about the size of the application (counting JARs and classes), and can you provide information about the timing for the three cases, above? I can review this to see if the times match the expected performance.

**Thomas Bitonti** · Answer 2 · 2023-01-26 21:16:18

Can you say what Jandex 2 version was used? Definitely, Jandex 3 indexes are not currently being read. All Jandex 2 version indexes should be read. I'll see if I can reproduce the index read failure.

For a startup with those counts of classes, using Jandex for just over 10% of the classes (5,118 out of 48,102 module classes) will mean a first startup will still be dominated by raw scans.

A second or subsequent start really should be faster. (Excepting very unusual cases: Say, an application with over 500 very small JAR files, in which case the overhead of managing the cache data becomes large.)

Can you see what data is generated in the annotations cache?

That should be a file set under the server work area, with application, module, and jar specific folders. For example:

wlp/usr/servers/test/workarea/org.eclipse.osgi/43/data/anno/A_simpleApp_A/M_simpleApp_M

wlp/usr/servers/<serverName>/workarea/org.eclipse.osgi/<bundleNumber>/data/anno/A_<APP_NAME>_A/M_<MODULE_NAME>_M/...

The bundle number varies, but is usually in the 40's. The key directory is "anno".

If you are interested, these are the trace specifications for annotation caching:

com.ibm.ws.annocache
com.ibm.ws.annocache.query
com.ibm.ws.annocache.state
com.ibm.ws.annocache.jandex

Note that some of these generate a lot of output.

This trace is also useful. Access to the annotations cache is as a container service.

com.ibm.ws.container.service

TechQA.

Open Liberty application server and jandex version

There are 2 answers

Related Questions in JAVA

Related Questions in OPEN-LIBERTY

Related Questions in JANDEX

Popular Questions

Popular Tags

Trending Questions