Oracle EBS: RW-50010: Error: - script has returned an error: 1

This one is fun. EBS installation failed at 29% after half a day of waiting.


Error

C:\\oracle\\dev\\fs2\\FMW_Home\webtier\OPatch ## Node=NodeId=1698 Type=24 TypeName=filesys_patch_action Name= RefId=901 State=init ConfigDoc=APPS_OHS_HOME ParentDoc=null Topology=R12 Action=os_cmd

    at oracle.apps.fnd.txk.config.FileSysPatchActionNode.doFileSysOSCmd(FileSysPatchActionNode.java:169)

    at oracle.apps.fnd.txk.config.FileSysPatchActionNode.processState(FileSysPatchActionNode.java:101)

    at oracle.apps.fnd.txk.config.PatchActionNode.processState(PatchActionNode.java:187)

    at oracle.apps.fnd.txk.config.PatchNode.processState(PatchNode.java:338)

    at oracle.apps.fnd.txk.config.PatchesNode.processState(PatchesNode.java:79)

    at oracle.apps.fnd.txk.config.InstallNode.processState(InstallNode.java:68)

    at oracle.apps.fnd.txk.config.TXKTopology.traverse(TXKTopology.java:594)

    at oracle.apps.fnd.txk.config.InstallService.doInvoke(InstallService.java:224)

    at oracle.apps.fnd.txk.config.InstallService.invoke(InstallService.java:237)

    at oracle.apps.fnd.txk.config.InstallService.main(InstallService.java:291)

C:\Users\ADMINI~1\AppData\Local\Temp\1>if 1 == 0 goto :INSTAS_OK

C:\Users\ADMINI~1\AppData\Local\Temp\1>echo Cannot install one-off patches

Cannot install one-off patches

RW-50010: Error: - script has returned an error:   1

RW-50004: Error code received when running external process.  Check log file for details.

Running APPL_TOP Install Driver for dev instance

This error is very generic. One should not stopped at this level of troubleshooting. I do not believed looking at this one can guess what is the root cause. Quite a few Oracle noteid look the same but having a different root cause at the end of the resolution.

User will need to look at the install log to look at which stage and line that failed. At 29%, it does half a dozen things particularly moving java files around. User can look at the entire installation in the  adrunfmw.cmd. 

Locate the install log. It usually reside in the EBS install folder under temp.  Example, oracle.apps.fnd.txk.install0.

Look for stdout and stderr or any exception errors. In my case, I encountered the following error.

  <message>Process Completed (3)  cmd /c rmdir /s /q C:\\oracle\\dev\\fs2\\FMW_Home\webtier\OPatch
Stdout:  See C:\oracle\dev\fs2\EBSapps\appl\admin\dev_ebsdev\tmp\T1456139825284_122.tmp
Stderr: The system cannot find the path specified.
</message>
</record>
<record>
  <date>2016-02-22T18:46:09</date>
  <millis>1456141569147</millis>
  <sequence>344</sequence>
  <logger>oracle.apps.fnd.txk.install</logger>
  <level>SEVERE</level>
  <class>oracle.apps.fnd.txk.config.InstallService</class>
  <method>printError</method>
  <thread>10</thread>
  <message>oracle.apps.fnd.txk.config.ProcessStateException: FileSys OS COMMAND Failed : Exit=3 See log for details. CMD= cmd /c rmdir /s /q C:\\oracle\\dev\\fs2\\FMW_Home\webtier\OPatch ## Node=NodeId=1698 Type=24 TypeName=filesys_patch_action Name= RefId=901 State=init ConfigDoc=APPS_OHS_HOME ParentDoc=null Topology=R12 Action=os_cmd
    at oracle.apps.fnd.txk.config.FileSysPatchActionNode.doFileSysOSCmd(FileSysPatchActionNode.java:169)
    at oracle.apps.fnd.txk.config.FileSysPatchActionNode.processState(FileSysPatchActionNode.java:101)
    at oracle.apps.fnd.txk.config.PatchActionNode.processState(PatchActionNode.java:187)
    at oracle.apps.fnd.txk.config.PatchNode.processState(PatchNode.java:338)
    at oracle.apps.fnd.txk.config.PatchesNode.processState(PatchesNode.java:79)
    at oracle.apps.fnd.txk.config.InstallNode.processState(InstallNode.java:68)
    at oracle.apps.fnd.txk.config.TXKTopology.traverse(TXKTopology.java:594)
    at oracle.apps.fnd.txk.config.InstallService.doInvoke(InstallService.java:224)
    at oracle.apps.fnd.txk.config.InstallService.invoke(InstallService.java:237)
    at oracle.apps.fnd.txk.config.InstallService.main(InstallService.java:291)

It is pretty self-explanatory. The required tmp file was not found. What I found though, not only the T1456139825284_122.tmp not found but the entire directory of C:\oracle\dev\fs2\EBSapps\appl\admin\dev_ebsdev\tmp\ was empty. The team have decided to get through this and re-visitng this missing file once the installation is successfully. Just to fast forward a little bit, there will be another missing file that I will point out in a bit.

Taking a look at the adrunfw.cmd command, I realized the script is hack-able. Since we are going to come back and revisit this issue if the installation ever completed, I decided to make a copy of the script and delete the script ending line so it break out of the loop if it failed and move on. Now - another issue found here, the  j11067592_fnd.zip was not residing in the right path where the script will be copying from later on. It was sitting one level outside of fnd directory. I decided to just copy and paste the file into the fnd folder.

As expected, a few hours into the installation, it did make it passed 29% and completed successfully. As of the "T1456139825284_122.tmp" and possibly other files, I have no idea how to address it. That file was no where to be found in any of the medias nor I know what they are or for. Perhaps, run a one-off patching might restore them since, the it failed at the One-Off stage. 

Please leave comments if you know what this T1456139825284_122.tmp is.