Blog dedicated to Oracle Applications (E-Business Suite) Technology; covers Apps Architecture, Administration and third party bolt-ons to Apps

Tuesday, March 31, 2009

^M character

Vinu asked me about ^M character in his scripts. ^M denotes carriage return. 

You can verify this by pressing the keys Ctrl and M together on your windows command prompt or unix prompt.

I told him that it is a well known issue. Whenever an ASCII file is transferred in binary mode to unix, ^M character is appended after every line. To convert it to correct unix format, you can use the dos2unix utility and the following command:

dos2unix original.file withoutM.file

In a loop (originally referenced here) you can fix all files:

$ for files in `find . -print`;
> do
> mv $files $files.old
> dos2unix -437 $files.old $files
> rm -rf $files.old
> done

Some of the other characters from Ctrl+A to Ctrl+Z:

Oct Dec Hex Name
000 0 0x00 NUL
001 1 0x01 SOH, Control-A
002 2 0x02 STX, Control-B
003 3 0x03 ETX, Control-C
004 4 0x04 EOT, Control-D
005 5 0x05 ENQ, Control-E
006 6 0x06 ACK, Control-F
007 7 0x07 BEL, Control-G
010 8 0x08 BS, backspace, Control-H
011 9 0x09 HT, tab, Control-I
012 10 0x0a LF, line feed, newline, Control-J
013 11 0x0b VT, Control-K
014 12 0x0c FF, form feed, NP, Control-L
015 13 0x0d CR, carriage return, Control-M
016 14 0x0e SO, Control-N
017 15 0x0f SI, Control-O
020 16 0x10 DLE, Control-P
021 17 0x11 DC1, XON, Control-Q
022 18 0x12 DC2, Control-R
023 19 0x13 DC3, XOFF, Control-S
024 20 0x14 DC4, Control-T
025 21 0x15 NAK, Control-U
026 22 0x16 SYN, Control-V
027 23 0x17 ETB, Control-W
030 24 0x18 CAN, Control-X
031 25 0x19 EM, Control-Y
032 26 0x1a SUB, Control-Z

Friday, March 27, 2009

FRM-92050 Failed to connect to the server

Srinivas reported this error after converting an instance to shared application file system:

Forms interface was not launching with this error:

FRM-92050: Failed to connect to the Server: apps11i.justanexample.com:8000

Details:
Java Exception:
java.io.EOFException
at java.io.DataInputStream.readInt(Unknown Source)
at oracle.forms.net.SocketConnection.connect(Unknown Source)
at oracle.forms.engine.Runform.initConnection(Unknown Source)
at oracle.forms.engine.Runform.startRunform(Unknown Source)
at oracle.forms.engine.Main.createRunform(Unknown Source)
at oracle.forms.engine.Main.start(Unknown Source)
at sun.applet.AppletPanel.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

I tried doing a telnet apps11i.justanexample.com 8000. This is a good test to see if the service is running. The service was not running. It was started through adfrmctl.sh. When I tried again, it connected but on pressing enter the connection was broken, which should not happen.

I found a direct hit in Metalink Note: 438231.1

Forms Server is up (f60webmx process is shown as a result of "ps -ef | grep f60" command) but when starting the forms server with the following command instead of adfrmctl.sh:

f60srvm em=__QA port= log=forms_log.txt mode=socket exe=f60webmx 2>&1 | tee forms_out.txt &

It returns
Failed to exec runform f60webmx

Cause

The issue is caused by the following setup: $FND_TOP/bin/f60webmx is renamed (e.g. to $FND_TOP/bin/f60webmx.sav) or removed and thus is inaccessible.

A similar issue is described in Metalink Note 1079806.6: Failed to Exec Runform when Starting Forms Server.

To relink f60webmx, I executed:

adrelink.sh force=y ranlib=y "fnd f60webmx"

It errored out with:

make -f $APPL_TOP/admin/out/link_fnd_21892.mk $FND_TOP/bin/f60webmx

Starting link of fnd executable 'f60webmx' on Fri Mar 27 16:02:14 EDT 2009
make: Fatal error: Don't know how to make target `$FND_TOP/lib/xitiap.o'
Done with link of fnd executable 'f60webmx' on Fri Mar 27 16:02:14 EDT 2009

Metalink Note 178919.1 describes this problem and sites missing file as the cause.

When I checked for $FND_TOP/lib/xitiap.o, it was indeed missing.

So I compared the number of files in $FND_TOP/lib in this instance and another instance. Total of 4 files were missing in this instance:

afpwrr.o
fdrrun.o
xitfnd.o
xitiap.o

These 4 files were copied from a working instance, and Srini ran:

adrelink.sh force=y ranlib=y "fnd f60webmx"

f60webmx got created without any errors this time.

FRM-92050 error also disappeared as forms was able to locate f60webmx.

Thursday, March 26, 2009

java.lang.SecurityException: cannot verify signature block file META-INF/CUST

Anand called me at 1:30 AM tonight. After applying Jan 2009 CPU, they were getting weird errors. Java console gave these errors:

Oracle JInitiator: Version 1.3.1.29
Using JRE version 1.3.1.29-internal Java HotSpot(TM) Client VM
User home directory = C:\Documents and Settings

Proxy Configuration: Browser Proxy Configuration

JAR cache enabled
Location: C:\Documents and Settings\Oracle Jar Cache
Maximum size: 50 MB
Compression level: 0
----------------------------------------------------
c: clear console window
f: finalize objects on finalization queue
g: garbage collect
h: display this help message
l: dump classloader list
m: print memory usage
q: hide console
s: dump system properties
t: dump thread list
x: clear classloader cache
0-5: set trace level to
----------------------------------------------------

Downloading http://apps11i.justanexample.com/OA_JAVA/oracle/apps/fnd/jar/fndforms.jar to JAR cache

java.lang.SecurityException: cannot verify signature block file META-INF/CUST
at sun.security.util.SignatureFileVerifier.process(Unknown Source)
at java.util.jar.JarVerifier.processEntry(Unknown Source)
at java.util.jar.JarVerifier.update(Unknown Source)
at java.util.jar.JarFile.initializeVerifier(Unknown Source)
at java.util.jar.JarFile.getInputStream(Unknown Source)
at oracle.jre.protocol.jar.JarCache$CachedJarLoader.authenticate(Unknown Source)
at oracle.jre.protocol.jar.JarCache$CachedJarLoader.access$4100(Unknown Source)
at oracle.jre.protocol.jar.JarCache$10.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.jre.protocol.jar.JarCache.privileged(Unknown Source)
at oracle.jre.protocol.jar.JarCache.access$2800(Unknown Source)
at oracle.jre.protocol.jar.JarCache$CachedJarLoader.download(Unknown Source)
at oracle.jre.protocol.jar.JarCache$CachedJarLoader.load(Unknown Source)
at oracle.jre.protocol.jar.JarCache.get(Unknown Source)
at oracle.jre.protocol.jar.CachedJarURLConnection.connect(Unknown Source)
at oracle.jre.protocol.jar.CachedJarURLConnection.getJarFile(Unknown Source)
at sun.misc.URLClassPath$JarLoader.getJarFile(Unknown Source)
at sun.misc.URLClassPath$JarLoader.(Unknown Source)
at sun.misc.URLClassPath$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.misc.URLClassPath.getLoader(Unknown Source)
at sun.misc.URLClassPath.getLoader(Unknown Source)
at sun.misc.URLClassPath.getResource(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at sun.applet.AppletClassLoader.findClass(Unknown Source)
at sun.plugin.security.PluginClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.applet.AppletClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.applet.AppletClassLoader.loadCode(Unknown Source)
at sun.applet.AppletPanel.createApplet(Unknown Source)
at sun.plugin.AppletViewer.createApplet(Unknown Source)
at sun.applet.AppletPanel.runLoader(Unknown Source)
at sun.applet.AppletPanel.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)


This error appeared in apache log:

[Fri Mar 27 01:53:10 2009] [error] [client 192.168.4.70] client denied by server configuration: $OA_JAVA/oracle/apps/fnd/util/ParameterList.class

Most probably the certificate was corrupt.

So I created a new certificate by giving the command:

adjkey -initialize

After that we ran adadmin and force regenerated all jar files.

Problem was resolved after this.

Here's something I found by googling:

The Signed JAR File

When jarsigner is used to sign a JAR file, the output signed JAR file is exactly the same as the input JAR file, except that it has two additional files placed in the META-INF directory:
a signature file, with a .SF extension, and
a signature block file, with a .DSA extension.

The base file names for these two files come from the value of the -sigFile option. For example, if the option appears as
-sigFile MKSIGN
the files are named "MKSIGN.SF" and "MKSIGN.DSA".

If no -sigfile option appears on the command line, the base file name for the .SF and .DSA files will be the first 8 characters of the alias name specified on the command line, all converted to upper case. If the alias name has fewer than 8 characters, the full alias name is used. If the alias name contains any characters that are not legal in a signature file name, each such character is converted to an underscore ("_") character in forming the file name.

The Signature (.SF) File
A signature file (the .SF file) looks similar to the manifest file that is always included in a JAR file when jarsigner is used to sign the file. That is, for each source file included in the JAR file, the .SF file has three lines, just as in the manifest file, listing the following:
the file name,
the name of the digest algorithm used (SHA), and
a SHA digest value.
In the manifest file, the SHA digest value for each source file is the digest (hash) of the binary data in the source file. In the .SF file, on the other hand, the digest value for a given source file is the hash of the three lines in the manifest file for the source file.
The signature file also, by default, includes a header containing a hash of the whole manifest file. The presence of the header enables verification optimization, as described in

JAR File Verification.
The Signature Block (.DSA) File
The .SF file is signed and the signature is placed in the .DSA file. The .DSA file also contains, encoded inside it, the certificate or certificate chain from the keystore which authenticates the public key corresponding to the private key used for signing.

JAR File Verification
A successful JAR file verification occurs if the signature(s) are valid, and none of the files that were in the JAR file when the signatures were generated have been changed since then. JAR file verification involves the following steps:
Verify the signature of the .SF file itself.
That is, the verification ensures that the signature stored in each signature block (.DSA) file was in fact generated using the private key corresponding to the public key whose certificate (or certificate chain) also appears in the .DSA file. It also ensures that the signature is a valid signature of the corresponding signature (.SF) file, and thus the .SF file has not been tampered with.
Verify the digest listed in each entry in the .SF file with each corresponding section in the manifest.

The .SF file by default includes a header containing a hash of the entire manifest file. When the header is present, then the verification can check to see whether or not the hash in the header indeed matches the hash of the manifest file. If that is the case, verification proceeds to the next step.
If that is not the case, a less optimized verification is required to ensure that the hash in each source file information section in the .SF file equals the hash of its corresponding section in the manifest file (seeThe Signature (.SF) File).
One reason the hash of the manifest file that is stored in the .SF file header may not equal the hash of the current manifest file would be because one or more files were added to the JAR file (using the jar tool) after the signature (and thus the .SF file) was generated. When the jar tool is used to add files, the manifest file is changed (sections are added to it for the new files), but the .SF file is not. A verification is still considered successful if none of the files that were in the JAR file when the signature was generated have been changed since then, which is the case if the hashes in the non-header sections of the .SF file equal the hashes of the corresponding sections in the manifest file.
Read each file in the JAR file that has an entry in the .SF file. While reading, compute the file's digest, and then compare the result with the digest for this file in the manifest section. The digests should be the same, or verification fails.
If any serious verification failures occur during the verification process, the process is stopped and a security exception is thrown. It is caught and displayed by jarsigner.

So its possible that we had an issue with the certificate which caused this problem.

How to get the value of environment variables in PL/SQL code

A well coded program avoids hard-coding of paths and uses variables. If you want to get the value of unix environment variables in your programs, you can use the DBMS_SYSTEM.GET_ENV procedure. For applying it in E-Business Suite, you would need to make sure that the variables are defined in RDBMS $ORACLE_HOME/custom$CONTEXT_NAME.env

For example, I defined environment variable CUSTOM_TOP in a new file $ORACLE_HOME/custom$CONTEXT_NAME.env. This file is called automatically by your database environment file $CONTEXT_NAME.env.

To test, whether the value appears simply do this in an sql session:

SQL> set autoprint on
SQL> var CUSTOM_TOP varchar2(255)
SQL> exec dbms_system.get_env('CUSTOM_TOP',:CUSTOM_TOP);

PL/SQL procedure successfully completed.


CUSTOM_TOP
--------------------------------------------------------------------------------
/custom/apps11i

Here's another code snippet:

1 DECLARE
2 v_file UTL_FILE.FILE_TYPE;
3 V_directory VARCHAR2(255);
4 BEGIN
5 dbms_system.get_env('CUSTOM_TOP', v_directory);
6 v_directory := v_directory||'/out';
7 dbms_output.put_line(v_directory);
8 v_file := UTL_FILE.FOPEN(v_directory,'TEST.DAT','W');
9 UTL_FILE.PUT_LINE(v_file,'This is a test file on CUSTOM_TOP');
10 UTL_FILE.FCLOSE(v_file);
11 EXCEPTION
12 WHEN OTHERS
13 THEN
14 DBMS_OUTPUT.PUT_LINE(SQLERRM);
15* END;
SQL> /
/custom/apps11i

PL/SQL procedure successfully completed.

SQL> host ls -ltr /customnp/apps11i/TEST.DAT
-rw-r--r-- 1 oracle dba 13 Mar 23 16:52 /custom/apps11i/TEST.DAT

SQL> host cat /custom/apps11i/TEST.DAT
This is a test file on CUSTOM_TOP

Wednesday, March 25, 2009

Forms compilation errors parameter.G_query_find

Recently all custom forms compilation failed with errors like these:

FRM-18108: Failed to load the following objects.

Source Module:APPSTAND
Source Object: STANDARD_PC_AND_VA
Source Module:APPSTAND
Source Object: STANDARD_TOOLBAR
Source Module:APPSTAND
Source Object: STANDARD_CALENDAR
Compiling package specification APP_CUSTOM...
No compilation errors.

or

Compiling KEY-NXTBLK trigger on QUERY_FIND data block...
Compilation error on KEY-NXTBLK trigger on QUERY_FIND data block:
PL/SQL ERROR 49 at line 1, column 1
bad bind variable 'parameter.G_query_find'
PL/SQL ERROR 49 at line 1, column 1
bad bind variable 'parameter.G_query_find'
PL/SQL ERROR 49 at line 3, column 1
bad bind variable 'parameter.G_query_find'

It turned out that FORMS60_PATH was not set correctly. Whenever forms are compiled FORMS60_PATH is set to:

FORMS60_PATH=$FORMS60_PATH:$AU_TOP/forms/US:$AU_TOP/resource

In context file it was set as:

$AU_TOP/resource:$AU_TOP/resource/stub

After correcting this and compiling manually it worked. To make this permanent change the s_f60path to include $AU_TOP/forms/US:$AU_TOP/resource

Tuesday, March 24, 2009

FNDCPSPN fails to relink

We got this error after applying TXK Rollup S:

Starting link of fnd executable 'FNDCPSPN' on Tue Mar 24 23:14:10 EDT 2009
ld -dy $ORACLE_HOME/lib/SC4.2/crti.o
$ORACLE_HOME/lib/SC4.2/crt1.o
$ORACLE_HOME/lib/SC4.2/crtn.o -s -o
$FND_TOP/bin/FNDCPSPN
$FND_TOP/lib/afcpspn.o \
$FND_TOP/lib/libfnd.a -lsql -lclntsh
$ORACLE_HOME/lib/nautab.o $ORACLE_HOME/lib/naeet.o
$ORACLE_HOME/lib/naect.o $ORACLE_HOME/lib/naedhs.o `cat
$ORACLE_HOME/lib/naldflgs` -lnetv2 -lnttcp -lnetwork -lncr -lnetv2
-lnttcp -lnetwork -lclient -lvsn -lcommon -lgeneric -lmm -lnlsrtl3 -lcore4
-lnlsrtl3 -lcore4 -lnlsrtl3 -lnetv2 -lnttcp -lnetwork -lncr -lnetv2 -lnttcp
-lnetwork -lclient -lvsn -lcommon -lgeneric -lepc -lnlsrtl3 -lcore4
-lnlsrtl3 -lcore4 -lnlsrtl3 -lclient -lvsn -lcommon -lgeneric -lnlsrtl3
-lcore4 -lnlsrtl3 -lcore4 -lnlsrtl3 `cat
$ORACLE_HOME/lib/sysliblist` -R
/opt/SUNWcluster/lib:$ORACLE_HOME/lib -Y
P,$ORACLE_HOME/lib::$ORACLE_HOME/network/jre11/lib/sparc/n
ative_threads:$CZ_TOP/bin:/usr/dt/lib:/usr/openwin/lib
:/opt/SUNWcluster/lib:/usr/ccs/lib:/usr/lib -Qy -lc -laio -lm
$ORACLE_HOME/rdbms/lib/defopt.o
$ORACLE_HOME/rdbms/lib/ssbbded.o
Undefined first referenced
symbol in file
FDCPSPN
$FND_TOP/lib/afcpspn.o
ld: fatal: Symbol referencing errors. No output written to
$FND_TOP/bin/FNDCPSPN
*** Error code 1
make: Fatal error: Command failed for target
`$FND_TOP/bin/FNDCPSPN'
Done with link of fnd executable 'FNDCPSPN' on Tue Mar 24 23:14:10 EDT 2009

Relink of module "FNDCPSPN" failed.
See error messages above (also recorded in log file) for possible
reasons for the failure. Also, please check that the Unix userid
running adrelink has read, write, and execute permissions
on the directory $FND_TOP/bin,
and that there is sufficient space remaining on the disk partition
containing your Oracle Applications installation.

Metalink Note: 417535.1 has the solution:

Cause

$FND_TOP/lib/libfnd.a does not include fdcpspn.lc


It sounds like a problem with the environment being used, not a problem with the patch. Patch 4905678 does not bring in afcpspn.lc, but somehow the file already exists on the system, and fdcpspn.lc does not exist in $FND_TOP/lib/libfnd.a. Any patch that tries to relink with afcpspn.lc and fdcpspn.lc on this system will fail.

A similare issue is described in INTERNAL, not visible to customers, Bug 4115814 (92) CERT 4030688 CERTIFICATION OF AFMDMSG.LCT FAILED ON RELINKING FNDCPSPN
You have to check $FND_TOP/lib/libfnd.a does not include the fdcpspn.o

Solution

To implement the solution, please execute the following steps:

1. Please take a backup of $FND_TOP/lib/ directory

2. adrelink.sh force=y ranlib=y "fnd FNDCPSPN"
and check if the adrelink is OK (exiting with status 0)

3. If failure at step 2:

4. cd $FND_TOP/lob
5. ar rvl libfnd.a fdcpspn.o
6. adrelink.sh force=y ranlib=y "fnd FNDCPSPN"