adcfgclone.pl hangs

Raju pinged me today and told me that adcfgclone.pl appsTier command was hanging after displaying this:

First Creating a new context file for the cloned system.
The program is going to ask you for information about the new system

Doing a truss did not give any additional information.

I opened adcfgclone.pl in vi and put in print messages at various places to find out where it was stopping. After a few runs I concluded that it was stopping at this line:

my $cmd="$JAVA -classpath $CLASSPATH oracle.apps.ad.context.CloneContext -e $CTXORIG -validate -pairsfile $pairsfile -stage $clonestage -passwd $PWD $isMer
ge $isDbTechStack $gConfig{'perlBin'} -perl5lib $ENV{'PERL5LIB'}";

When $cmd was executed it stopped.

I tried executing this command manually but it would give errors like /tmp/adparsfile_2822 doesn't exist.

I searched further in adcfgclone.pl script to find out about pairsfile:

Create the pairsfile with defaults ctx values
# s_hostname, user, group
# my $pairsfile = "$tempdir/adpairsfile_$$.lst"; unlink ($pairsfile);
open(PAIRSF, ">$pairsfile");

However it wasn't creating the anything like /tmp/adpairsfile*

That function open in the line open(PAIRSF, ">$pairsfile"), made me think of open files and open file descriptors. I thought of checking the ulimit on file descriptors.

ulimit -n
2048

I increased it:

ulimit -n 8192

and executed the command:

adcfgclone.pl appsTier

Voila, adcfgclone.pl did not hang any more, it proceeded ahead and asked for instance name.

It was hanging because it had run out of open file descriptors. In other words 2048 files were already open, that's why it could not open a new file. Increasing the ulimit allowed it to open the adpairsfile as the 2049th file.