Thursday, May 26, 2016


 Some customer wanted to setup despite it being out of support.Their 3rd party application only worked on this.So , we had no choice to agree to set this up.We faced issues like our OEL was 5.+ and database was 11gr2. The is certified for 10g database and OEL 4+. We had to work around it to make it work.The product support didnot help much as its not supported any longer.

For using 11g database with SOA i followed,

The only way to ignore OEL check was to run the installer with,

runInstaller -ignoreSysprereqs 

Then came the surprises on issues after issues on simple install.I will list them out as i couldnot find any documentation in oracle support or blog on internet

1)When i login to or ESB, after sometime it throws .

    <MSG_TEXT>ESB bootstrap: Unknown error occured in constructor of ESB resource adapter</MSG_TEXT>
    <SUPPL_DETAIL><![CDATA[oracle.tip.esb.model.repository.exception.RepositoryException: Unable to get connection to Repository
Ensure repository is accessible
        at oracle.tip.esb.model.repository.database.DatabaseRepository.getConnection(Unknown Source)
        at oracle.tip.esb.model.repository.database.DatabaseRepository.getESBParameters(Unknown Source)
        at oracle.tip.esb.server.bootstrap.ESBBaseResourceAdapter.<init>(Unknown Source)

I reinstalled so many times due to this, total waste of time.When i checked log.xml , i found that it was filled with ORA-01017: invalid username/password; logon denied

This was baffling as user /password login worked fine.Then i realized this was 11g database so i need to make password sensitivity false!

Just run this on the database.

This fixed the repository error!!
2) Orbpel issue.
     [echo] == Applying BPEL patch in primary instance "oc4j_soa" ==
     [copy] Copying 1 file to /sid/ias/product/1013/bpel/system/services/lib
     [copy] Copying 1 file to /sid/ias/product/1013/bpel/system/services/lib
     [copy] Copying 1 file to /sid/ias/product/1013/bpel/system/services/lib      [jar] Updating jar: /sid/ias/product/1013/bpel/system/services/lib/orabpel.ear     [echo] Re-deploying orabpel ...
     [echo] Running: java -jar /sid/ias/product/1013/j2ee/home/admin_client.jar deployer:oc4j:opmn://hostname:11722/oc4j_soa oc4jadmin [oc4jadmin-password] -redeploy -keepsettings -bindAllWebApps default-web-site -file /sid/ias/product/1013/bpel/system/services/lib/orabpel.ear -deploymentName orabpel
     [java] Redeploy error: Redeploy failed: Unable to redeploy Application: orabpel does not exist!
I again tried reinstall, notes was of no use.Then i manually deployed it and did a retry on installer.It worked!
Below command can be used to deploy Orabpel.Export java_home etc..
java -jar /sid/ias/product/1013/j2ee/home/admin_client.jar deployer:oc4j:opmn://hostname:11722/oc4j_soa oc4jadmin [oc4jadmin-password] -deploy   -file /sid/ias/product/1013/bpel/system/services/lib/orabpel.ear -deploymentName orabpel -bindAllWebApps
3) Rulehelp issue.
[echo] Running: java -jar /apps/oasprd/product/AS10gR3_BPEL/j2ee/home/admin_client.jar deployer:oc4j:opmn://oa1-iprd-21:6055/BPEL_OC4J_COE oc4jadmin [oc4jadmin-password] -redeploy -keepsettings -bindAllWebApps default-web-site -file /apps/oasprd/product/AS10gR3_BPEL/rules/webapps/rulehelp.ear -deploymentName rulehelp [echo]  [java] Redeploy error: Redeploy failed: Unable to redeploy Application: rulehelp does not exist!
As a work around comment out, in $ORACLE_HOME/bpel/system/services/install/ant-tasks/redeploy.xml, the line 'redeploy-app file "${oracle.home}/rules/webapps/rulehelp.ear" deploymentname='rulehelp'
Acording to bug 9097350 'rulehelp.ear' file is not a vital component that will prevent the install from working. This file only provides online help for the Rules Editor UI, and if needed the .ear file can be deployed at a later date either (i) manually or (ii) using EM to deploy. 
4) There was in issue with one off patch with installer,
To run in silent mode, OPatch requires a response file for Oracle Configuration Manager (OCM).
Run /dwwy8o/ias/product/asmt_1013/OPatch/ocm/bin/emocmrsp to generate an OCM response file. The generated response file can be reused on different platforms and in multiple OPatch silent installs.

To regenerate an OCM response file, rerun  /sid/ias/product/1013/OPatch/ocm/bin/emocmrsp.
I just ran below command and did a retry on the installer,

 $ORACLE_HOME/ccr/bin/setupCCR -s -d

** Installing base package **
Deploying core - Version
Deploying engines - Version
Deploying metricdata - Version
Deploying scripts - Version

Oracle Configuration Manager has been configured in disconnected mode. If the
target ORACLE_HOME is running a database, please refer to the
"Post-installation Database Configuration" section of the OCM Installation
and Administration Guide
( to complete the

View configuration data reports and access valuable configuration best
practices by going to MetaLink.

Monday, October 19, 2015

WebCenter Enterprise Capture Console or Client, Error 404

After installing capture on a windows machine, after installing i get 404 error when i access url or rather try to login.

Solution was from a oracle document, but seems like we need to retarget capture application.

Un-target and re-target the Capture deployment to the Capture Server:

1. Log into the Weblogic Server Console.
2. Click Deployments in the Domain Structure list.
3. Select Capture.
4. Check Capture.
5. Click Change Deployments.
6. Uncheck the Capture_Server (or cluster) check box.
7. Click Yes. This will save the un-target for the Capture deployment to the Capture Server.
8. Check Capture from the list of deployments again.
9. Select Change Targets.
10. Check the Capture_Server (or cluster) check box
11. Click Yes.

Tuesday, September 22, 2015

Oracle Impdp Failed on RAC with ORA-31640, ORA-19505, ORA-27037, Linux-x86_64 Error: 2: No such file or directory

We were doing a import on 3 node RAC (exadata)using IMPDP.We obviously used prallelism, but faced below error.First look indicated that its related to OS/Filesystem. But its related to how impdp worked on a RAC node.

Additional information: 3
ORA-31693: Table data object "ORACLE_ORASDPM"."MESSAGE" failed to load/unload and is being skipped due to error:
ORA-31640: unable to open dump file "/oracle/oracle_soa_schemas_exp.dmp" for read
ORA-19505: failed to identify file "/oracle/oracle_soa_schemas_exp.dmp"
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory

When you are using a cluster database, it is possible for datapump workers to start on any of the available nodes when using parallelism. In our case dump file was local to node1, so node2 and 3 were unable to access it. If the nodes that start the workers do not have access to the datapump directory they will be unable to process the files.

Impdp seems to have option "Cluster=N" .
This will remove the other nodes from the impdp process.

Friday, May 29, 2015

OIM 11g quartz scheduler logging

 We had some issues in OIM quartz scheduler.So, we had to enable logging for it.The process for it is slightly different.

- Go to  OIM_ORACLE_HOME/server/config
- Create a with content,

handlers=java.util.logging.FileHandler, java.util.logging.ConsoleHandler
# default log level
# Quartz logger level
# log file name for the quartz log
# formatter
# limit the size of the file
# recycle

In the add to the JAVA_OPTIONS,


This will need the OIM managed server restart and the log file will get created in domain home with name quartzX.log

Oracle Directory Integration Platform(DIP)

   I had encountered Oracle directory integration platform in 2008, when i was working to integrate EBS HR module with OID 10g.It had some complex profile setup and i did many tests on it and found lot of issues and worked with oracle support/dev almost for a year !!Finally it worked !

   Nowadays, I manage a team which works on Fusion middleware products.The IDM suite specifically is getting complex day by day with addition of products or each version having architecture changes.But one thing which remains almost same is the OID.

   Though there are some changes ofcourse which can be read here. In 10g where dip was a process, in 11g its a j2ee application.It can be used to develop repository integration solution with other ldap directories.Oracle Directory Integration Platform  provides Synchronization Service and Provisioning Services.


EBS to OID bidirectional integration

I had a query from an internal team , they had setup EBS 12.2 to oid 11g integration.A user created in EBS was populating to oid ,but user created in oid was not syncing up in EBS.

So, the above problem statement makes veteran Fusion experts know where to look, let me crack a simple to do list for dummies.

Came across a superb blog ,

1)Issue in EBS to OID sync, Look in the EBS profile options.

Applications SSO Enable OID Identity Add Event
  • When profile 'Applications SSO Enable OID Identity Add Event' value is 'Enabled', users created in OID are automatically created in E-Business and subscribed to the E-Business instance.
  • When profile 'Applications SSO Enable OID Identity Add Event' value is 'Disabled', users created in OID will not be automatically created in E-Business. They can be created in E-Business (and subscribed to it) only after provsubtool or OIDDAS Edit Service Recipient page is used to subscribe existing users to the particular E-Business instance. 
  • Application SSO LDAP Synchronization must be ENABLED regardless of provisioning type
2)Issue with oid user to sync with EBS , there are dip profiles which are created by default and are bidirectional by default.
  • Log in to Oracle Enterprise Manager Fusion Middleware Control.
  • In the navigation panel on the left, click or expand the Identity and Access entry and then select the DIP component that contains the profile you want to edit.
  • Click the DIP Server menu, point to Administration, and then click Synchronization Profiles. The Manage Synchronization Profiles appears displaying a list of the existing profiles.
  • Here it turned to be simple case of misconfiguration of OID details on em, i.e worng port on server properties of DIP application.This can be changed via command line or in the em console.
This is very very basic way to troubleshoot, there can be lot many things wrong.

Wednesday, May 20, 2015

OIM error (org.jgroups.protocols.UDP._send)

  We found this error in logs when customer reported it.

[APP: oim#] failed sending message to null (58 bytes)[[
java.lang.Exception: dest=/ (61 bytes)
at org.jgroups.protocols.UDP._send(
at org.jgroups.protocols.UDP.sendToAllMembers(
at org.jgroups.protocols.TP.doSend(
at org.jgroups.protocols.TP.send(
at org.jgroups.protocols.TP.down(
at org.jgroups.protocols.PING.sendMcastDiscoveryRequest(
at org.jgroups.protocols.PING.sendGetMembersRequest(
at org.jgroups.protocols.Discovery$PingSenderTask$
at java.util.concurrent.Executors$
at java.util.concurrent.FutureTask.runAndReset(

This is known bug and we fixed this by following a workaround given by Oracle.

We just  added below parameter in OIM startup argument.Based on you are starting from console or backend you need to make this change.

Sunday, November 16, 2014

Error while applying BSU patch

 bash-3.2$ ./ -install -patch_download_dir=/prod/oracle/111/utils/bsu/cache_dir/ -patchlist=T5F1 -prod_dir=/prod/oracle/111/wlserver_10.3
Exception in thread "Thread-0" java.lang.NullPointerException
        at com.bea.plateng.patch.PatchTargetHelper.loadPatchProfiles(
        at com.bea.plateng.patch.PatchTarget.<init>(
        at com.bea.plateng.patch.PatchTargetFactory.create(
        at com.bea.plateng.patch.ProductAliasTarget.constructPatchTargetList(
        at com.bea.plateng.patch.ProductAliasTarget.<init>(
        at com.bea.plateng.patch.ProductAliasTargetHelper.getProdAliasTargetList(        at com.bea.plateng.patch.ProductAliasTargetHelper.getAllHomeToProdAliasesTargetMap(        at com.bea.plateng.patch.ProductAliasTargetHelper.checkProfilesInProductAliases(        at com.bea.plateng.patch.Patch$

 I thought this is related to previous error posted in my blog, i.e.patch_backup.xml file having wrong MW home.But this one was different.

The directory  /prod/oracle/111/patch_ocp371/ had wrong owner hence we were getting issues.After changing to correct owner issue was fixed.Make sure the directories inside it also have correct owner and permissions..(/prod/oracle/111/patch_ocp371/profiles/default)

Thursday, September 25, 2014


<Sep 24, 2014 10:48:26 PM CDT> <Error> <Cluster> <BEA-000109> <An error occurred while sending multicast message: Invalid argument Invalid argument
        at Method)
        at weblogic.cluster.MulticastFragmentSocket.sendThrottled(
        at weblogic.cluster.MulticastFragmentSocket.send(
        at weblogic.cluster.FragmentSocketWrapper.send(
        Truncated. see log file for complete stacktrace

We faced this issue on a customer with a cluster and using multicast for some reason.When we tried starting managed server we got this error.


Refer:Doc ID 1360498.1

Even if both protocols are enabled, it is possible to force a specific protocol for multicast using the property Adding this property to JAVA_OPTIONS in the startup script, like, resolves the issue:

This fixed the issue.

Wednesday, August 13, 2014

Store:280105 : Error with file store

 <Aug 14, 2014 5:45:38 AM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Aug 14, 2014 5:45:38 AM UTC> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Aug 14, 2014 5:45:38 AM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
<Aug 14, 2014 5:45:38 AM UTC> <Critical> <Health> <BEA-310006> <Critical Subsystem PersistentStore._WLS_wls_osb2 has failed. Setting server state to FAILED.
Reason: [Store:280105]The persistent file store "_WLS_wls_osb2" cannot open file _WLS_WLS_OSB2000000.DAT.>
<Aug 14, 2014 5:45:39 AM UTC> <Critical> <WebLogicServer> <BEA-000385> <Server health failed. Reason: health of critical service 'PersistentStore._WLS_wls_osb2' failed>
<Aug 14, 2014 5:45:39 AM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Aug 14, 2014 5:45:39 AM UTC> <Error> <> <BEA-000000> <cannot dump threads on this VM:

<Aug 14, 2014 2:58:43 AM UTC> <Error> <Store> <BEA-280061> <The persistent store "_WLS_wls_soa1" could not be deployed: [Store:280105]The persistent file store "_WLS_wls_soa1" cannot open file _WLS_WLS_SOA1000000.DAT. [Store:280105]The persistent file store "_WLS_wls_soa1" cannot open file _WLS_WLS_SOA1000000.DAT.

We were getting these errors on a 4 node cluster for both SOA and OSB, i checked config.xml and couldnot find any file store with this name!! Also we already tried after renaming tmp/cache/data folder.Seems like nothing was working.But i finally found the root cause! It was the files in cluster folder.

-bash-4.1$ ls
_WLS_WLS_OSB1000000.DAT  _WLS_WLS_OSB2000000.DAT
-bash-4.1$ ls
_WLS_WLS_SOA1000000.DAT    _WLS_WLS_SOA2000000.DAT  

Just renamed these are tried starting , then we got local filestore errors, in which case we had to rename every *.DAT file(file store folders are in domain home).After that the server came up