Blog dedicated to Oracle Applications (E-Business Suite) Technology; covers Apps Architecture, Administration and third party bolt-ons to Apps

Wednesday, December 14, 2011

java.lang.NegativeArraySizeException

On starting weblogic on Solaris 10, I got this error

Could not load the performance pack that can take advantage of /dev/(e)poll device due to:
java.lang.NegativeArraySizeException.

I checked ulimit:

ulimit -n
65536

I reduced ulimit to 65534:

ulimit -n 65534

and weblogic started up fine.

I remember reading somewhere that java has array overflow issues if ulimit is set to maximum.

A connection to the server could not be established: null (user=DBA.ADMIN)

This morning I got a call from various people complaining about OTM not allowing them to login. Anyone who tried logging in got this error:

A connection to the server could not be established: null (user=any_valid_otm_user)

I checked $GLOG_HOME/logs/weblogic/console.log.0:


INFO | 2011/12/14 09:40:07 | -- OTM Event: serverReady
INFO | 2011/12/14 09:47:01 | javax.management.MBeanException: MBean invoke failed: weblogic.management.mbeanservers.edit.EditTimedOutException:
INFO | 2011/12/14 09:47:01 |
INFO | 2011/12/14 09:47:01 | javax.management.MBeanException: MBean invoke failed: weblogic.management.mbeanservers.edit.EditTimedOutException:
INFO | 2011/12/14 09:47:01 | at weblogic.rmi.internal.ServerRequest.sendReceive(ServerRequest.java:205)
INFO | 2011/12/14 09:47:01 | at weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:222)
INFO | 2011/12/14 09:47:01 | at javax.management.remote.rmi.RMIConnectionImpl_1033_WLStub.invoke(Unknown Source)
INFO | 2011/12/14 09:47:01 | at weblogic.management.remote.common.RMIConnectionWrapper$16.run(ClientProviderBase.java:876)
INFO | 2011/12/14 09:47:01 | at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
INFO | 2011/12/14 09:47:01 | at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
INFO | 2011/12/14 09:47:01 | at weblogic.security.Security.runAs(Security.java:61)
INFO | 2011/12/14 09:47:01 | at weblogic.management.remote.common.RMIConnectionWrapper.invoke(ClientProviderBase.java:874)
INFO | 2011/12/14 09:47:01 | at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:993)
INFO | 2011/12/14 09:47:01 | at glog.util.jmx.JMXConnection.call(JMXConnection.java:104)
INFO | 2011/12/14 09:47:01 | at glog.util.j2ee.remote.WLManagementHelper$Edit.(WLManagementHelper.java:54)
INFO | 2011/12/14 09:47:01 | at glog.util.j2ee.jdbc.WLConnectionPoolInstance.create(WLConnectionPoolInstance.java:139)
INFO | 2011/12/14 09:47:01 | at glog.util.j2ee.jdbc.ASConnectionPoolInstance.(ASConnectionPoolInstance.java:52)
INFO | 2011/12/14 09:47:01 | at glog.util.datasource.AppServerConnectionPoolInstance.(AppServerConnectionPoolInstance.java:9)
INFO | 2011/12/14 09:47:01 | at glog.util.datasource.AppServerConnectionPool.getNewInstance(AppServerConnectionPool.java:71)
INFO | 2011/12/14 09:47:01 | at glog.util.datasource.AppServerConnectionPool.getInstance(AppServerConnectionPool.java:61)
INFO | 2011/12/14 09:47:01 | at glog.util.datasource.AppServerConnectionPool.reserve(AppServerConnectionPool.java:34)
INFO | 2011/12/14 09:47:01 | at glog.util.datasource.DataSource.reserve(DataSource.java:143)
INFO | 2011/12/14 09:47:01 | at glog.util.datasource.DataSource.reserve(DataSource.java:137)
INFO | 2011/12/14 09:47:01 | at glog.util.jdbc.noserver.T2SharedConnection$ReferencedConnection.addRef(T2SharedConnection.java:301)
INFO | 2011/12/14 09:47:01 | at glog.util.jdbc.noserver.T2SharedConnection$ReferencedConnection.addRef(T2SharedConnection.java:291)
INFO | 2011/12/14 09:47:01 | at glog.util.jdbc.noserver.T2SharedConnection.open(T2SharedConnection.java:57)
INFO | 2011/12/14 09:47:01 | at glog.server.useraccess.UserAccessSessionBean.getUserAccess(UserAccessSessionBean.java:154)
INFO | 2011/12/14 09:47:01 | at glog.server.useraccess.UserAccessSessionServerSideEJBWrapper.getUserAccess(UserAccessSessionServerSideEJBWrapper.java:87)
INFO | 2011/12/14 09:47:01 | at glog.server.useraccess.UserAccessSessionHome_uzvgzz_EOImpl.getUserAccess(UserAccessSessionHome_uzvgzz_EOImpl.java:1600)
INFO | 2011/12/14 09:47:01 | at glog.server.useraccess.UserAccessSessionHome_uzvgzz_EOImpl_WLSkel.invoke(Unknown Source)
INFO | 2011/12/14 09:47:01 | at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
INFO | 2011/12/14 09:47:01 | at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
INFO | 2011/12/14 09:47:01 | at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
INFO | 2011/12/14 09:47:01 | at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
INFO | 2011/12/14 09:47:01 | at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
INFO | 2011/12/14 09:47:01 | at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
INFO | 2011/12/14 09:47:01 | at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
INFO | 2011/12/14 09:47:01 | at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
INFO | 2011/12/14 09:47:01 | at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

I googled and found a few hits and in one of them found this line:

When I'm trying to deploy my project on weblogic I've got this error:
2007-02-09 16:05:15 CET Error Deployer BEA-149181 Deployment Service encountered error while aborting the edit session before start. Initiato
r for this edit session is 'principals=[kernel identity = -1632281053]'. Error is 'weblogic.management.provider.EditNotEditorException: Not edit lock
owner'
weblogic.management.provider.EditNotEditorException: Not edit lock owner

On seeing those words: Not edit lock owner, I realized that the previous day, I was trying to create SFTP File control in OTM Weblogic console. It wasn't letting me create a new one, till I clicked Lock and Edit. So I quickly logged into weblogic console and clicked on Release Configuration button. The subsequent logins to OTM started working after this.

Thursday, December 8, 2011

passwordless ssh doesn't work

On both source and target servers ensure these permissions exist:

$ chmod go-w ~/
$ chmod 700 ~/.ssh
$ chmod 600 ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/id_rsa
$ chmod 644 ~/.ssh/id_rsa.pub
$ chmod 644 ~/.ssh/known_hosts

It didn't work for me in one case where $HOME directory had 777 permission

Looking at /var/log/auth.log on my server revealed the problem: sshd was
refusing to use my public key because my home directory was
group-writable. In order to do pubkey auth, both the home directory
and the .ssh directory must writable only by the owner.

Two parts: first, turn up debugging on your ssh sever. Edit /etc/ssh/sshd_config and increase LogLevel to DEBUG. Then force your ssh server to reload it's config with killall -HUP .

That will cause the server to add much more details to your /var/log/secure and/or /var/log/auth logfiles.

Secondly (actually you cant try this first), increase the debug level on the client side. ssh in to the box with

$ ssh -vvv hostname

and that will print out lots more info about where the process is failing.

If you do turn up the debug level on your ssh server, don't forget to turn it back down when you are finished.


A few more reasons why this fails:

1. Make sure the owner:group on the directories and files is correct:

ls -ld  $HOME/
ls -ld  $HOME/.ssh
ls -ltr $HOME/.ssh

Login as root

chown user:group $HOME
chown user:group $HOME/.ssh
chown user:group $HOME/.ssh/authorized_keys
chown user:group $HOME/.ssh/id_rsa
chown user:group $HOME/.ssh/id_rsa.pub
chown user:group $HOME/.ssh/known_hosts

2. This one took a lot of R&D.  The user's entry was missing in /etc/shadow

Login as root
grep user /etc/shadow

If user doesn't exist in /etc/shadow, ssh will fail.

This is caused, because unix admins sometimes do not use useradd command to add users, they just copy the line from /etc/passwd from an existing server and paste it in new server.  They forget to add the line for the user inside /etc/shadow.  If useradd command is used, it takes care of this automatically.

Monday, November 21, 2011

Where is OA_HTML defined in R12

OA_HTML is defined in $ORA_CONFIG_HOME/10.1.3/j2ee/oacore/config/oc4j.properties

Thursday, November 17, 2011

IGW_BUDGET_CATEGORY_V invalid in R12

If you are getting this in 11i then you can apply this patch as per article
Patch 6629280: APPSST11G:11510CU2: IGW: INVALID GROUP BY EXPRESSION IN VIEW DUE TO CBO IFX



Originally we got 70 invalids after applying R12.1.3 patch 9239090. Oracle advised us to run utlirp.sql, followed by utlrp.sql.

We ran utlirp.sql which invalidates all objects
then ran utlrp.sql which reduced the count to 53
then we ran utlrp.sql which reduced the count to 2:

VIEW IGW_BUDGET_CATEGORY_V
PACKAGE BODY IGW_REPORT_PROCESSING

The package was not compiling because of the invalid view. So I extracted the definition of the view from a working R12.1.3 instance through dbms_metadata:

SQL> connect apps/appspassword
SQL> set heading off
SQL> set long4000
SQL> select dbms_metadata.get_ddl('VIEW','IGW_BUDGET_CATEGORY_V') from dual;


CREATE OR REPLACE FORCE VIEW "APPS"."IGW_BUDGET_CATEGORY_V" ("PROPOSAL_ID", "VERSION_ID", "BUDGET_PERIOD_ID", "LINE_ITEM_ID", "EXPENDITURE_TYPE", "EXPENDITURE_CATEGORY_FLAG", "BUDGET_CATEGORY_CODE", "BUDGET_CATEGORY", "BASE_AMT", "OH_APPLIED_FLAG", "OH_COST", "EB_COST") AS
SELECT pbcv.proposal_id,
pbcv.version_id,
pbcv.budget_period_id,
pbcv.line_item_id,
pbcv.expenditure_type,

pbcv.expenditure_category_flag,
pbcv.budget_category_code,
fl.meaning budget_category,
SUM (pbcv.line_item_cost + pbcv.eb_cost) base_amt,
DECODE (pbcv.oh_cost, 0, 'N', 'Y') oh_applied_flag,
pbcv.oh_cost,
pbcv.eb_cost
FROM igw_budget_complete_v pbcv, igw_lookups_v fl
WHERE pbcv.budget_category_code = fl.lookup_code
AND fl.lookup_type = 'IGW_BUDGET_CATEGORY'
GROUP BY pbcv.proposal_id,
pbcv.version_id,
pbcv.budget_period_id,
pbcv.budget_category_code,
fl.meaning,
pbcv.line_item_id,
pbcv.expenditure_type,
pbcv.expenditure_category_flag,
DECODE (pbcv.oh_cost, 0, 'N', 'Y'),
pbcv.oh_cost,
pbcv.eb_cost

I have corrected the indentation above. You may have to correct indentation if you plan to copy from the output generated by dbms_metadata.

Executed this and we have zero invalids now:

1 CREATE OR REPLACE FORCE VIEW "APPS"."IGW_BUDGET_CATEGORY_V" ("PROPOSAL_ID ", "VERSION_ID", "BUDGET_PERIOD_ID", "LINE_ITEM_ID", "EXPENDITURE_TYPE", "EXPEND ITURE_CATEGORY_FLAG", "BUDGET_CATEGORY_CODE", "BUDGET_CATEGORY", "BASE_AMT", "OH _APPLIED_FLAG", "OH_COST", "EB_COST") AS
2 SELECT pbcv.proposal_id,
3 pbcv.version_id,
4 pbcv.budget_period_id,
5 pbcv.line_item_id,
6 pbcv.expenditure_type,
7 pbcv.expenditure_category_flag,
8 pbcv.budget_category_code,
9 fl.meaning budget_category,
10 SUM (pbcv.line_item_cost + pbcv.eb_cost) base_amt,
11 DECODE (pbcv.oh_cost, 0, 'N', 'Y') oh_applied_flag,
12 pbcv.oh_cost,
13 pbcv.eb_cost
14 FROM igw_budget_complete_v pbcv, igw_lookups_v fl
15 WHERE pbcv.budget_category_code = fl.lookup_code
16 AND fl.lookup_type = 'IGW_BUDGET_CATEGORY'
17 GROUP BY pbcv.proposal_id,
18 pbcv.version_id,
19 pbcv.budget_period_id,
20 pbcv.budget_category_code,
21 fl.meaning,
22 pbcv.line_item_id,
23 pbcv.expenditure_type,
24 pbcv.expenditure_category_flag,
25 DECODE (pbcv.oh_cost, 0, 'N', 'Y'),
26 pbcv.oh_cost,
27* pbcv.eb_cost
28 /

View created.

SQL> alter package IGW_REPORT_PROCESSING compile body;

Package body altered.

SQL> select instance from v$thread;

INSTANCE
--------------------------------------------------------------------------------
exampler12

SQL> select count(*) from dba_objects where status='INVALID';

COUNT(*)
----------
0

Wednesday, November 2, 2011

Use Opera to access edelivery.oracle.com

After Oracle SSO enabled edelivery.oracle.com, we were unable to access the site through Opera browser. Whenever we clicked on Sign On/Register button a screen with Oracle Access Manager would come up with

Error

System error. Please re-try your action. If you continue to get this error, please contact the Administrator.

A google search revealed the answer from OTN. You have to make Opera identify itself as Internet Explorer. It works fine then. This is from http://www.opera.com/support/kb/view/843/

Go to edelivery.oracle.com. Click on Sign in / Register button. Wait for the error to appear.
Right click anywhere on the site, click Edit Site Preferences.
Click the "Network" tab
In the "Browser identification" drop-down, select "Identify as Internet Explorer" or "Identify as Firefox"
Click "OK"
Reload the page

Tuesday, October 18, 2011

UX: useradd: ERROR: Inconsistent password files

While using useradd command to add my user, I got this error:

UX: /usr/sbin/useradd: ERROR: Inconsistent password files. See pwconv(1M).

Logged in as root, I checked the no. of lines in /etc/passwd and /etc/shadow:

# cat /etc/shadow |wc -l
299
# cat /etc/passwd |wc -l
300

vi /etc/passwd

I saw that last line was a blank and removed the blank line.

Using useradd command worked now without issues.

Thursday, September 15, 2011

Weblogic hangs at BEA-002014 IIOP subsystem enabled

After a server reboot, OTM Weblogic would not start up. It was forever waiting after showing these lines in weblogic.log:

####Sep 15, 2011 5:22:38 PM EDT Info IIOP weblogic.justanexample.com gc3-appotms1 [ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)' WLS Kernel 1316121758429 BEA-002014 IIOP subsystem enabled.

A google search with keywords "Weblogic hang BEA-002014 IIOP subsystem enabled" led me to

https://forums.oracle.com/forums/thread.jspa?threadID=923601

Vivek Tripathi gave a solution two years ago in 2009:

Hmmmm ... Last line in the logs are
"<2/07/2009 08:54:10 PM CDT> "

Usually after enabling IIOP subsystem weblogic server initiates "security". I will suggest to rename the "ldap" directory present under location "\servers\\data" and start the server.
This issue may be due to corrupt LDAP data.

I stopped Apache and Weblogic

cd $GLOG_HOME/weblogic/domains/otm/servers/gc3-appotms1/data

mv ldap ldap.old

Restarted Apache and Weblogic

Weblogic came up fine after this.

Tuesday, September 13, 2011

[HTTP:101216]Servlet: "action" failed to preload on startup in Web application: "ccg.war".

While deploying ccg in a freshly installed Weblogic 10.3.5, this error was coming:

[HTTP:101216]Servlet: "action" failed to preload on startup in Web application: "ccg.war". java.lang.NullPointerException at org.apache.commons.digester.Digester.getXMLReader(Digester.java:899) at org.apache.commons.digester.Digester.parse(Digester.java:1647) at org.apache.struts.action.ActionServlet.parseModuleConfigFile(ActionServlet.java:708) at org.apache.struts.action.ActionServlet.initModuleConfig(ActionServlet.java:670) at org.apache.struts.action.ActionServlet.init(ActionServlet.java:329) at javax.servlet.GenericServlet.init(GenericServlet.java:241) at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:283) at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120) at weblogic.servlet.internal.StubSecurityHelper.createServlet(StubSecurityHelper.java:64) at weblogic.servlet.internal.StubLifecycleHelper.createOneInstance(StubLifecycleHelper.java:58) at weblogic.servlet.internal.StubLifecycleHelper.(StubLifecycleHelper.java:48) at weblogic.servlet.internal.ServletStubImpl.prepareServlet(ServletStubImpl.java:539) at weblogic.servlet.internal.WebAppServletContext.preloadServlet(WebAppServletContext.java:1985) at weblogic.servlet.internal.WebAppServletContext.loadServletsOnStartup(WebAppServletContext.java:1959) at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1878) at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:3153) at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1508) at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:482) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119) at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200) at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119) at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27) at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:636) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:205) at weblogic.application.internal.SingleModuleDeployment.activate(SingleModuleDeployment.java:43) at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161) at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79) at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:569) at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:150) at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:116) at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:323) at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:844) at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1253) at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:440) at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:163) at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195) at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13) at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68) at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528) at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209) at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)

The issue is caused by the following setup: Existence of ‘xercesImpl.jar’ file in the ccg-war directory (Location: /WEB-INF/lib) caused errors on the Weblogic 11g application server while deplying CCG applications.

As per My Oracle Support article "CCG 5.5.1 Deployment With Weblogic 10.3.0.1/11g Fails [ID 1265899.1]"

This is explained in the following bug:
Bug 10251676 - CCG 5.5.1 DEPLOYMENT WITH WEBLOGIC 10.3.0.1 FAILS

Solution
1. Delete Weblogic cache and restart it before applying the workaround below.
2. Remove the ‘xercesImpl.jar’ file from ccg-war directory (Location: /WEB-INF/lib)
3. Redeploy CCG applications on Weblogic 11g application server. (Refer to the CCG Install Guide v5.5.1 for detailed instructions.

Weblogic Cache exists in $MW_HOME/user_projects/domains/ccg/servers/AdminServer/tmp
I had named my domain ccg. You can replace that with your domain name in the above path.

So I did this:

Stop Weblogic Server
rm /ccg/WEB-INF/lib/xercesImpl.jar
rm -rf $MW_HOME/user_projects/domains/ccg/servers/AdminServer/tmp
mkdir $MW_HOME/user_projects/domains/ccg/servers/AdminServer/tmp
Start Weblogic Server

The issue was resolved.

GRC An error occurred during the login process

On a freshly installed Oracle EGRC(Enterprise Governance Risk and Compliance) 8.6.3 on OEL 5.6, I got this error in front end:

An error occurred during the login process.

On checking $MW_HOME/grc863/grc/log/grc.log, I found this error:

2011-09-13 14:27:25,529 ERROR [el.Default (self-tuning)'] LoginServiceImpl:233 Error during User Authentication : No Configuration was registered that can handle the configuration named JdbcAuth
java.lang.IllegalArgumentException: No Configuration was registered that can handle the configuration named JdbcAuth
at com.bea.common.security.jdkutils.JAASConfiguration.getAppConfigurationEntry(JAASConfiguration.java:130)
at oracle.apps.grc.security.AuthenticationEnforcer.authenticate(AuthenticationEnforcer.java:77)
at oracle.apps.grc.ui.login.server.LoginServiceImpl.isLogin(LoginServiceImpl.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:562)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:188)
at oracle.apps.grc.ui.common.server.context.AACGRemoteServiceServlet.processCall(AACGRemoteServiceServlet.java:73)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:224)
at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:300)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at oracle.apps.grc.ui.webapp.servlet.ApplicationResourceRequestFilter.doFilter(ApplicationResourceRequestFilter.java:262)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at oracle.security.jps.ee.http.JpsAbsFilter$1.run(JpsAbsFilter.java:111)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.util.JpsSubject.doAsPrivileged(JpsSubject.java:313)
at oracle.security.jps.ee.util.JpsPlatformUtil.runJaasMode(JpsPlatformUtil.java:413)
at oracle.security.jps.ee.http.JpsAbsFilter.runJaasMode(JpsAbsFilter.java:94)
at oracle.security.jps.ee.http.JpsAbsFilter.doFilter(JpsAbsFilter.java:161)
at oracle.security.jps.ee.http.JpsFilter.doFilter(JpsFilter.java:71)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at oracle.dms.servlet.DMSServletFilter.doFilter(DMSServletFilter.java:136)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:27)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3715)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3681)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2277)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2183)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1454)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)


I checked startManagedWeblogic.sh script and found that there was a space before jaas.conf

JAVA_OPTIONS="-Xms8192m -Xmx16384m -Xss512k -XX:MaxPermSize=512m -Doracle.security.jps.auth.debug=true -Doracle.security.jps.auth.debug.verbose=true -Djava.security.debug=access,failure -Djava.security.auth.login.config="/GE/erpgrca1/grcapp/weblogic/grc863/grc/WEB-INF/ jaas.config" -Djava.awt.headless=true -Dweblogic.security.SSL.trustedCAKeyStore="/GE/erpgrca1/grcapp/weblogic/wlserver_10.3/server/lib/cacerts" ${JAVA_OPTIONS}"
export JAVA_OPTIONS

I started Weblogic again and a new error showed up in grc.log:

2011-09-13 14:43:05,456 ERROR [el.Default (self-tuning)'] LoginServiceImpl:233 Error during User Authentication : No Configuration was registered that can handle the configuration named JdbcAuth
java.lang.IllegalArgumentException: No Configuration was registered that can handle the configuration named JdbcAuth
at com.bea.common.security.jdkutils.JAASConfiguration.getAppConfigurationEntry(JAASConfiguration.java:130)
at oracle.apps.grc.security.AuthenticationEnforcer.authenticate(AuthenticationEnforcer.java:77)
at oracle.apps.grc.ui.login.server.LoginServiceImpl.isLogin(LoginServiceImpl.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:562)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:188)
at oracle.apps.grc.ui.common.server.context.AACGRemoteServiceServlet.processCall(AACGRemoteServiceServlet.java:73)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:224)
at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:300)
at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56)
at oracle.apps.grc.ui.webapp.servlet.ApplicationResourceRequestFilter.doFilter(ApplicationResourceRequestFilter.java:262)

I went through the documentation again and realized that I needed to do this:

If you are installing EGRCC to perform analysis in Oracle EBS or PeopleSoft instances:
1. Stop the Administration Server.
2. Open the file StartWebLogic.sh in a text editor. The file resides in the /user_projects/domains//bin directory.
3. Add the following line to the file, replacing with the full path to the grc863 directory you created earlier (see step 5 of ―Preparing Additional Files‖ on page 2-7).

SAVE_JAVA_OPTIONS="-Xms512m -Xmx4096m -Xss512k -XX:MaxPermSize=512m -Doracle.security.jps.auth.debug=true -Doracle.security.jps.auth.debug.verbose=true -Djava.security.debug=access,failure -Djava.security.auth.login.config="/grc/WEB-INF/ jaas.config" -Djava.awt.headless=true -Dweblogic.security.SSL.trustedCAKeystore="/wl server_10.3/server/lib/cacerts" ${JAVA_OPTIONS}"

You may use a maximum memory setting (-Xmx) larger than 4096m if your server has enough memory to support the larger value.
4. Start the Administration Server

Once I did the abve steps, I was able to login to GRC with the default username/password : admin/admin

Wednesday, September 7, 2011

Sendmail connects to localhost only

In a new Solaris build, we go this problem about sendmail. It was connecting on sendmail port 25 only if hostname was localhost or 127.0.0.1. It was not listening on all the other IPs plumbed on the box. Here's what I did to make it listen on all IPs:

Stop sendmail
svcadm disable sendmail

cd /etc/mail
chmod 600 sendmail.cf

Search for this line:
O DaemonPortOptions=NAME=NoMTA4, Family=inet, Addr=127.0.0.1

Remove , Addr=127.0.0.1 so that the line now reads:
O DaemonPortOptions=NAME=NoMTA4, Family=inet

chmod 444 sendmail.cf

Start Sendmail
svcadm enable sendmail

Retest by doing

telnet hostname_of_server 25

If it still doesn't work, do this:

svccfg -s sendmail setprop config/local_only=false

svcadm refresh sendmail

svcadm restart sendmail

Friday, August 26, 2011

OTM HTTP-400 Bad Request

Some users reported HTTP-400 Bad Request errors in ERPOTMD1. Here's what was happening:

1. If the first window you open was an OTM instance, everything worked fine.

2. If you started E-Business Suite and then accessed OTM, HTTP-400 Bad Request would result.

3. This was happening in IE8, Firefox6.

4. Apache error.log showed:

[Wed Aug 24 14:41:43 2011] [error] [client 3.75.208.77] request failed: error reading the headers


I googled and found this:

http://allthingsunix.inside.quest.com/thread.jspa?threadID=4667

This post advised:

request headers can be quite long, try setting

LimitRequestFieldsize 12392


in the server context.

See http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestfieldsize

So I checked the value of this variable in Apache httpd.conf and found that it was set to:

LimitRequestFieldSize 2048
LimitRequestLine 4096


I changed it to

LimitRequestFieldSize 20480
LimitRequestLine 40960


And bounced Apache and Weblogic.

Problem was resolved. For official confirmation from Oracle about this fix, I logged SR. Oracle OTM Development was ok with this change.

Wednesday, August 17, 2011

adgendbc.sh java.sql.SQLException: Invalid number format for port number

Bimal asked me about this one. While running adconfig on an R12 clone, the following error was being reported:

Unique constraint error (00001) is OK if key already exists

Creating the DBC file...
java.sql.SQLException: SO Exception was generated
Database connection to jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=YES)(FAILOVER=YES)(ADDRESS=(PROTOCOL=tcp)(HOST=R1213q2.justanexample.com)(PORT=1553)))(CONNECT_DATA=)) failed
ADD call failed with exit code 1

Updating Server Security Authentication
java.sql.SQLException: Invalid number format for port number
Database connection to jdbc:oracle:thin:@host_name:port_number:database failed
Updating Server Security Authentication failed with exit code 1
Restoring DBC file from backed up location $INST_TOP/appltmp/TXK/R1213q2_Wed_Aug_17_14_32_2011.dbc

adgendbc.sh ended at Wed Aug 17 14:32:04 EDT 2011

adgendbc.sh exiting with status 1
ERRORCODE = 1 ERRORCODE_END
.end std out.

.end err out.
****************************************************

We noticed the blank CONNECT_DATA and did node cleanup and re-ran autoconfig. However it still failed:

Unique constraint error (00001) is OK if key already exists

Creating the DBC file...
java.sql.SQLRecoverableException: No more data to read from socket raised getting GUEST_USER_PWD - ignoring
java.sql.SQLRecoverableException: No more data to read from socket- Database error performing ADD

ADD call failed with exit code 1

Updating Server Security Authentication
java.sql.SQLException: Invalid number format for port number
Database connection to jdbc:oracle:thin:@host_name:port_number:database failed
Updating Server Security Authentication failed with exit code 1
Restoring DBC file from backed up location $INST_TOP/appltmp/TXK/R1213q2_Wed_Aug_17_16_11_2011.dbc

adgendbc.sh ended at Wed Aug 17 16:11:16 EDT 2011

adgendbc.sh exiting with status 1
ERRORCODE = 1 ERRORCODE_END
.end std out.
Unable to update GUEST_USER_PWD in database to GUEST/ORACLE - java.sql.SQLRecoverableException: No more data to read from socket

After a lot of searching in My Oracle Support I found this article: Adgendbc Fails With Database Connection Failure [ID 1302708.1]. It asked us to do this:

The following change to the database parameters allows Autoconfig, and specifically adgenfbc.sh, to complete successfully. Please make this change and then re-run autoconfig.

alter system set JAVA_JIT_ENABLED= FALSE scope = both;

We did this, but the error still appeared.

So we bounced the database. After bouncing the database it worked fine:


$ cd $INST_TOP/admin/install
$ ./adgendbc.sh

adgendbc.sh started at Wed Aug 17 16:20:52 EDT 2011


Enter the APPS username: apps

Enter the APPS password:

SQL*Plus: Release 10.1.0.5.0 - Production on Wed Aug 17 16:21:02 2011

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Enter value for 1: Enter value for 2: Enter value for 3: Connected.
[ APPS_DATABASE_ID ]
Application Id : 0
Profile Value : orpogpq2
Level Name: SITE
INFO : Updated/created profile option value.
.

PL/SQL procedure successfully completed.


Commit complete.

Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, Oracle Label Security, OLAP, Data Mining
and Real Application Testing options


==============================
* * * * DBC PARAMETERS * * * *
==============================
fnd_jdbc_buffer_min=1
fnd_jdbc_buffer_max=5
fnd_jdbc_buffer_decay_interval=300
fnd_jdbc_buffer_decay_size=5
fnd_jdbc_usable_check=false
fnd_jdbc_context_check=true
fnd_jdbc_plsql_reset=false



====================================
* * * * NO CUSTOM PARAMETERS * * * *
====================================



Unique constraint error (00001) is OK if key already exists

Creating the DBC file...
ADD executed successfully - $INST_TOP/appl/fnd/12.0.0/secure/orpogpq2.dbc

Updating Server Security Authentication
AUTHENTICATION SECURE executed successfully - $INST_TOP/appl/fnd/12.0.0/secure/R1213q2.dbc

adgendbc.sh ended at Wed Aug 17 16:21:39 EDT 2011

adgendbc.sh exiting with status 0
ERRORCODE = 0 ERRORCODE_END

Wednesday, July 20, 2011

OTM java.lang.NullPointerException

In OTM we got this error reported by end users:

On many screens for example User Manager > New Button > key in all details > Click Finish.

Users Finder > Results

Results
Errors
Create Another Modify Another

Error

java.lang.NullPointerException

Stack Trace
java.lang.NullPointerException

java.lang.NullPointerException
at glog.util.jdbc.noserver.T2SharedConnection$ReferencedConnection.setAutoCommit(T2SharedConnection.java:421)
at glog.util.jdbc.noserver.T2SharedConnection$ReferencedConnection.release(T2SharedConnection.java:326)
at glog.util.jdbc.noserver.T2SharedConnection.close(T2SharedConnection.java:61)
at glog.database.security.dbrealm.RdbmsRealm.createUser(RdbmsRealm.java:642)
at glog.database.security.dbrealm.RdbmsRealm.createUser(RdbmsRealm.java:652)
at glog.database.security.dbrealm.CachingRealm.newUser(CachingRealm.java:483)
at glog.database.security.SecurityUserImpl.create(SecurityUserImpl.java:65)
at glog.database.security.SecurityUtil.createUser(SecurityUtil.java:52)
at glog.database.security.SecuritySessionBean.createUser(SecuritySessionBean.java:40)
at glog.database.security.SecuritySessionServerSideEJBWrapper.createUser(SecuritySessionServerSideEJBWrapper.java:167)
at glog.database.security.SecuritySessionHome_snlvan_EOImpl.createUser(SecuritySessionHome_snlvan_EOImpl.java:5947)
at glog.database.security.SecuritySessionHome_snlvan_EOImpl_WLSkel.invoke(Unknown Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
at weblogic.rmi.cluster.ClusterableServerRef.invoke(ClusterableServerRef.java:230)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:477)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:473)
at weblogic.rmi.internal.wls.WLSExecuteRequest.run(WLSExecuteRequest.java:118)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)






Fix Errors


This was due to the fact that DB Listener was down. Once DB listener was brought up, this error disappeared.

Sunday, July 17, 2011

opendaemonsocket: daemon MTA-v6: server SMTP socket wedged

After installing a newly available build of Solaris 10, sendmail was disallowing connections from anything but localhost. We tried taking a diff between sendmail.cf of older build and the new sendmail.cf and found that the old sendmail.cf had extra lines for DaemonPortOptions :

O DaemonPortOptions=Name=MTA-v4, Family=inet
O DaemonPortOptions=Name=MTA-v6, Family=inet6
O DaemonPortOptions=Port=587, Name=MSA, M=E

After including the DaemonPortOptions of MTA-v4 and MTA-v6, it started giving this error message:

opendaemonsocket: daemon MTA-v6: server SMTP socket wedged

So we removed the MTA-v6 line, and the above error stopped coming. Meanwhile I had googled this error and found this post on OpenSolaris forms. Based on the information, I reasoned that changing the parameter to false would allow connections from other hosts:

svccfg -s sendmail setprop config/local_only=false
svcadm refresh sendmail
svcadm restart sendmail

This worked perfectly fine. Since we don't use Solaris Zones, there was no need to use the pfexec command.

Thursday, June 9, 2011

Oracle Fusion Apps for Linux x86-64 available for download on edelivery

While looking for something completely different, on edelivery.oracle.com I was surprised to find Oracle Fusion Apps appearing in the drop down of products. Sure enough, Oracle Fusion Apps is now available for download for Linux x86-64:


Monday, June 6, 2011

Weblogic is replacing OC4J as middleware in R12.2

I just read Steven Chan's blog about June 2011 EBS Techstack Roadmap which has some slides on upcoming R12.2. Most notable in R12.2:

Latest middleware: Fusion Middleware 11g
  • WebLogic Server to replace Fusion Middleware 10g Oracle Containers for Java (OC4J)

Latest database: Oracle 11gR2 11.2 Database
  • Online patching via Edition-Based Redefinition
  • Improved performance via database result caching
Oracle has been steadily pushing Weblogic as its chosen Application server for all its products which require an application server. Notable among these products are Oracle Transportation Management (OTM), Oracle Governance, Risk and Compliance (GRC) Suite. Now EBS will also use Weblogic as the application server. Oracle Application Server or OC4J is the terminal release as Oracle Application Server 11g is Oracle Weblogic application server.

So all Oracle Apps techies brush up your skills and learn how Weblogic operates.

Tuesday, May 31, 2011

R12 AppsLocalLogin shows blank page

This issue has already been reported to me twice, so I am documenting this for future:

R12 AppsLocalLogin shows blank page

A check of logs shows this:

cd $LOG_HOME/ora/10.1.3/j2ee/oacore/oacore_default_group_1
vi application.log
Go to the end of the file

11/05/31 15:38:07.918 html: LeakSetting:
global:true,session:false,aggressive:false,stderr:false,appslog:false,corelog:false
ignoring:null
11/05/31 15:39:40.810 html: chain failed
javax.servlet.ServletException: java.lang.RuntimeException: Guest user/pwd does not exist or match: GUEST/ORACLE
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.EvermindPageContext.handlePageThrowable(EvermindPageContext.java:899)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.EvermindPageContext.handlePageException(EvermindPageContext.java:816)
at _AppsLocalLogin._jspService(_AppsLocalLogin.java:303)
at com.orionserver[Oracle Containers for J2EE 10g (10.1.3.5.0) ].http.OrionHttpJspPage.service(OrionHttpJspPage.java:59)
at oracle.jsp.runtimev2.JspPageTable.service(JspPageTable.java:473)
at oracle.jsp.runtimev2.JspServlet.internalService(JspServlet.java:594)
at oracle.jsp.runtimev2.JspServlet.service(JspServlet.java:518)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64)
at oracle.apps.jtf.base.session.ReleaseResFilter.doFilter(ReleaseResFilter.java:26)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:15)
at oracle.apps.fnd.security.AppsServletFilter.doFilter(AppsServletFilter.java:318)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:642)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:391)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:908)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:458)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:313)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].server.http.AJPRequestHandler.run(AJPRequestHandler.java:199)
at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
at oracle.oc4j.network.ServerSocketAcceptHandler.procClientSocket(ServerSocketAcceptHandler.java:234)
at oracle.oc4j.network.ServerSocketAcceptHandler.access$700(ServerSocketAcceptHandler.java:29)
at oracle.oc4j.network.ServerSocketAcceptHandler$AcceptHandlerHorse.run(ServerSocketAcceptHandler.java:879)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.5.0) ].util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:298)
at java.lang.Thread.run(Thread.java:662)


Start an sqlplus session

SQL> conn apps/appspassword
Connected.
SQL> select fnd_web_sec.validate_login('GUEST','ORACLE') FROM DUAL;
select fnd_web_sec.validate_login('GUEST','ORACLE') FROM DUAL
*
ERROR at line 1:
ORA-03113: end-of-file on communication channel

Check the Database Alert Log
Tue May 31 15:50:51 2011
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x0] [PC:0x108250218, ioc_pin_shared_executable_object()+1240] [flags: 0x0, count: 1]
Errors in file /appsr1213/oracle/11.2.0/admin/vision_vision/diag/rdbms/vision/vision/trace/vision_ora_10944.trc (incident=121312):
ORA-07445: exception encountered: core dump [ioc_pin_shared_executable_object()+1240] [SIGSEGV] [ADDR:0x0] [PC:0x108250218] [Address not mapped to object] []
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Tue May 31 15:50:54 2011
Sweep [inc][121312]: completed

If you open two unix sessions side by side:

First session: connect to sqlplus as apps and execute
select fnd_web_sec.validate_login('GUEST','ORACLE') FROM DUAL;

Second session: tail Database alert log

You'll notice that whenever this sql is executed in first session, the alert log will immediately log an ORA-07445 error.

This is an 11.2.0.2 bug. The workaround is to bounce the database and listener. Once I learn more about this bug, I'll update this post.

Friday, May 13, 2011

Can't locate strict.pm in @INC

On running adcfgclone.pl on 11.2.0.2 RDBMS ORACLE_HOME, the following error occurred.

$ perl adcfgclone.pl dbTechStack
Can't locate strict.pm in @INC (@INC contains: /erpR1213/oracle/11.2.0/perl/lib/5.8.3 /erpR1213/oracle/11.2.0/perl/lib/site_perl/5.8.3 /erpR1213/oracle/11.2.0/appsutil/perl ../lib/5.10.0/sun4-solaris-thread-multi-64 ../lib/5.10.0 ../lib/site_perl/5.10.0/sun4-solaris-thread-multi-64 ../lib/site_perl/5.10.0 ../lib/5.10.0 ../lib/5.10.0/sun4-solaris-thread-multi-64 ../lib/site_perl .) at adcfgclone.pl line 27.
BEGIN failed--compilation aborted at adcfgclone.pl line 27.


This is a known issue and now documented in My Oracle Support Article ADPreclone Fails When Can't Find Strict.PM Following Database Upgrade to 11gR2 [ID 1139403.1].

11.2.0.2 ships with perl 5.10.0. So you need to change the values of context variables pointing to PERL5LIB and ADPERLPRG inside the context file.

vi $CONTEXT_FILE
Global replace 5.8.3 with 5.10.0
Save
cd $ORACLE_HOME/appsutil/template
vi adxdbctx.tmp
Global replace 5.8.3 with 5.10.0

Run adcfgclone.pl, or adconfig or adpreclone.pl and it will work now.

Wednesday, April 27, 2011

Files in which OTM stores IP address and port

During the recent upgrade of OTM v6.1.0 to OTM v6.2.1, I discovered that OTM stores IP address of its servers in the following files:

$GLOG_HOME/tomcat/conf/server.xml
$GLOG_HOME/weblogic/domains/otm/config/config.xml.fresh
$GLOG_HOME/apache/conf/httpd.conf
$GLOG_HOME/apache/conf/workers.properties


For Apache and Weblogic services, OTM listens on virtual IP of the instance, so there is no port conflict as the Virtual_IP:Port remains unique.

However, the Tomcat component listens on localhost on port 8007 by default. So if you have two OTM instances share the same physical box as application tier, you need to change the default port of 8007 to some other port which is available. This change is made in $GLOG_HOME/tomcat/conf/server.xml.

Tuesday, April 12, 2011

DBUA unable to recognize oracle user as unix owner when unix uid is 9 digits

While upgrading an Oracle instance from 11.2.0.1 to 11.2.0.2, through DBUA I discovered a bug. If the unix uid of the oracle user is 9 digit, DBUA refuses to recognize oracle as the unix owner of the software. This issue doesn't occur with older oracle ids which were 5 digit. We got a similar issue in past with runInstaller for which an SR was logged and Oracle opened bug 10205927. For the current issue with DBUA, I have logged a new SR

Till this bug is fixed by Oracle, DBUA will fail for any instance whose oracle user has unix uid of 9 digit. All upgrades to 11.2.0.2 on such instances will have to use the manual upgrade process.

I'll update this post with more details, after Oracle responds on the SR.