Automating Response File Creation

At the time of writing most sites are still using Oracle 11.2.0.3.0, usually with a more recent PSU or CPU. This version of Oracle is supplied with version 11.2.0.1.7 of the opatch utility. In order to apply newer patches, it is first necessary to upgrade opatch which can be downloaded as patch 6880880 from MOS.

One of the changes in recent versions of opatch are a couple of questions about the Oracle Configuration Manager (OCM) which allows sites to receive security updates by e-mail. I have not yet encountered a site that has wished to configure this feature when patching the database. However, by default it is necessary to answer these questions every time a patch is applied.

You can optionally skip the questions by creating a response file. This response file can be specified on the opatch command line using the -ocmrf parameter.

The OCM response file is generated using the emocmrsp utility which is located in $ORACLE_HOME/OPatch/ocm/bin. I use the following Korn shell script to invoke emocmrsp:

#!/usr/bin/ksh

EMOCMRSP=$ORACLE_HOME/OPatch/ocm/bin/emocmrsp

$EMOCMRSP

By default this utility reads from standard input and generates the following output:

OCM Installation Response Generator 10.3.4.0.0 - Production
Copyright (c) 2005, 2010, Oracle and/or its affiliates.  All rights reserved.

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
The OCM configuration response file (ocm.rsp) was successfully created.

The first question is the Email address/User name which is usually answered with a newline character. The second question is whether you wish to remain uninformed about security issues, which is usually answered with a Y character followed by a newline.

By default output is written to a file called ocm.rsp in the current working directory. The location of the output file can be controlled using the -output parameter. For example:

#!/usr/bin/ksh

EMOCMRSP=$ORACLE_HOME/OPatch/ocm/bin/emocmrsp

$EMOCMRSP -output /home/oracle/expect/ocm0.rsp

The banner can also be suppressed using the -no_banner option:

#!/usr/bin/ksh

EMOCMRSP=$ORACLE_HOME/OPatch/ocm/bin/emocmrsp

$EMOCMRSP -no_banner -output /home/oracle/expect/ocm0.rsp

This generates the following:

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
The OCM configuration response file (/home/oracle/expect/ocm0.rsp) was successfully created.

So far so good, but what if we want to automate generation of the response file. If you are attempting to perform a silent install of the Oracle Grid Infrastructure or RDBMS software that includes application of a CPU, PSU or one off patches, you will need to generate this response file. By default this requires user input which obviously conflicts with the use of silent options for the rest of the installation.

Unfortunately emocmrsp does not appear to support a silent option. Nor does it allow the user to specify command line parameters to answer the questions. Of course it would take Oracle Support a few seconds to remedy this situation, but apparently this is not a priority. Therefore, a more creative scripting solution is required.

An obvious choice is the HERE document which is a feature of most shell scripting languages (sh, bash and ksh at least).

So we could write the script as follows:

#!/usr/bin/ksh

EMOCMRSP=$ORACLE_HOME/OPatch/ocm/bin/emocmrsp
OCM_FILE=/home/oracle/expect/ocm2.rsp

$EMOCMRSP -no_banner -output $OCM_FILE <<EOF

Y
EOF

A here document redirects everything between the << symbol and the label specified immediately after that label. In the above example the first line after the << symbol contains a newline character and the second contains a 'Y" and an newline character. The here document is terminated by the closing EOF label.

This looks OK, but when executed it returns the following results:

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Invalid value specified.

Although the here document contains the same characters that we entered before, the utility fails to read them correctly and returns "Invalid value specified". The reasons for this are not entirely clear, but I have reproduced them in both AIX and OEL Linux.

I searched the internet to see if others had already solved this problem and found a suggestion on the OTN community https://community.oracle.com/message/11345171; to use the expect utility. Although the page claims that the question is not answered it is close enough.

Expect is an interesting utility that allows interactive sessions to be scripted. Unlike here documents, expect scripts can implement conditional scripts based on the output of the program which they are interacting with. I saw this utility used in a Dell whitepaper on SAN snapshot management and vowed to investigate it when I had the chance.

Expect was not installed by default in my OEL5U6 VM, but it was included in the software distribution, so I installed it using

rpm -ivh expect-5.43.0-5.1.x86_64.rpm

Expect depends on TCL and optionally on TK, but these were already installed in my VM as part of my default configuration.

This was my first attempt (t3.exp):

#!/usr/bin/expect -f
spawn /u01/app/oracle/product/11.2.0/dbhome_1/OPatch/ocm/bin/emocmrsp \\
-no_banner -output /home/oracle/expect/ocm3.rsp
expect {
  "Email address/User Name:"
  {
    send "\
"
    exp_continue
  }
  "Do you wish to remain uninformed of security issues*"
  {
    send "Y\
"
    exp_continue
  }
}

Expect is similar to a shell scripting language so a directive can be included in the first line to use /usr/bin/expect as shown above. In this case the -f flag indicates that the remainder of the file is the expect script.

The first action is to execute emocmrsp using the spawn directive. The rest of the script consists of an expect command. When emocmrsp prints the line "Email address/User Name:" expect will respond with a newline character. When emocmrsp prints the security issues question, expect will respond with a Y followed by a newline character.

Note that it is possible to use a backslash to escape a newline character in the spawn command. This was a lucky guess.

Running this script has the desired result:

[oracle@vm3]$ ./t3.exp
spawn /u01/app/oracle/product/11.2.0/dbhome_1/OPatch/ocm/bin/emocmrsp -no_banner -output /home/oracle/expect/ocm3.rsp
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
The OCM configuration response file (/home/oracle/expect/ocm3.rsp) was successfully created.

We have eliminated the error that occurred with the here document though I really have no idea how.

However, I am developing my automated scripts on AIX and have been mandated is to use the Korn shell. So how can I combine the expect script with a Korn shell script:

In first attempt t4.ksh contained the following:

#!/usr/bin/ksh

export EMOCMRSP=$ORACLE_HOME/OPatch/ocm/bin/emocmrsp
export OCM_FILE=/home/oracle/expect/ocm4.rsp

/usr/bin/expect -f t4.exp

The expect file t4.exp contained the following:

spawn $env(EMOCMRSP) -no_banner -output $env(OCM_FILE)
expect {
  "Email address/User Name:"
  {
    send "\
"
    exp_continue
  }
  "Do you wish to remain uninformed of security issues*"
  {
    send "Y\
"
    exp_continue
  }
}

A quick internet search revealed that expect scripts can access exported environment variables which means I can set them in the Korn shell script and access them in the expect script.

However, as I mentioned earlier, I am generated all of the scripts required for the automation process. The goal is to generate the entire configuration for an Maximum Availability Architecture (MAA) configuration with four nodes in each cluster, so I have hundreds of scripts to manage. Consequently I am keen to minimize the number of scripts that I need to generate in order to simplify maintenance and to reduce the amount of documentation required. I was interested therefore in combining the Korn shell and expect scripts.

Ironically the solution was to use both expect and a here document as shown below (t5.ksh):

#!/usr/bin/ksh

EMOCMRSP=$ORACLE_HOME/OPatch/ocm/bin/emocmrsp
OCM_FILE=/home/oracle/expect/ocm5.rsp

/usr/bin/expect - <<EOF
spawn $EMOCMRSP -no_banner -output $OCM_FILE
expect {
  "Email address/User Name:"
  {
    send "\
"
    exp_continue
  }
  "Do you wish to remain uninformed of security issues*"
  {
    send "Y\
"
    exp_continue
  }
}
EOF

The above Korn shell script solves all the issues I have been addressing:

However, the real reason I am so interested in expect is that it may allow me to script the configuration of SSH connectivity for RAC clusters. The OUI configures SSH connectivity when run in GUI mode. However, when executed in silent mode it requires SSH connectivity to have been manually configured as a prerequisite. For ten years I have been configuring SSH manually, but at last I think I can see the light at the end of the tunnel.