HI Exports,
Need Very urgent help, upgrade is in process and we are in Preprocessing Phase - MAIN_SHDIMP/SUBOD_SHD2_RUN/STOP_SHDI_SHD2 step.
normally this step should take more then 1 mins to shutdown Shadow instance but from past 12 hours the step is not completed.
When checked in SUM/Abap/log last log was generated 9 hours but there is no error reported in log of SUM tool.
Logs of STOPSHD1.LOG
tail -100 STOPSHDI.LOG
1 ETQ201 Entering upgrade-phase "MAIN_SHDIMP/SUBMOD_SHD2_RUN/STOP_SHDI_SHD2" ("20150721213615")
1 ETQ399 Removing function description 'SUBST_START_BATCHJOB' from RFC cache.
1 ETQ399 Removing function description 'SUBST_CHECK_BATCHJOB' from RFC cache.
1 ETQ399 Removing function description 'SUBST_START_REPORT_IN_BATCH' from RFC cache.
2 ETQ367 Connect variables are set for standard instance access
4 ETQ399 System-nr = '00', GwService = 'sapgw00' Client = '000'
4 ETQ399 Environment variables:
4 ETQ399 dbs_ora_schema=SAPR3
1 ETQ200 Executing actual phase 'MAIN_SHDIMP/SUBMOD_SHD2_RUN/STOP_SHDI_SHD2'.
1 ETQ399 Phase arguments:
2 ETQ399 Arg[0] = 'STOP_SERVICE'
1 ETQ399 SYSTEM MANAGER: Initializing.
1 ETQ399 SYSTEM MANAGER: STOPSAP on UNIX.
1 ETQ399 SYSTEM MANAGER: STOPSAP with SAPCONTROL.
1 ETQ399 SYSTEM MANAGER: SAPControl tries to control all instances.
1 ETQ399 SYSTEM MANAGER: Calling getInstances.
1 ETQ399 SYSTEM MANAGER: found instance for action STOP : Instance Number: 02
1 ETQ399 SYSTEM MANAGER: ControlInstance with SAPCONTOL action STOP for instance 02.
1 ETQ399 SYSTEM HEALTH MANAGER: running preCheck for instance 02 on host sapehx01.
2 ETQ399 SYSTEM HEALTH MANAGER: Checking instance number 2 for used ports on host 'sapehx01'.
3 ETQ399 SYSTEM HEALTH MANAGER: Port 3202 appears to be used.
1 ETQ399 SYSTEM HEALTH MANAGER: call test rfc.
4 ETQ399 Set RFC variables for shadow connect:
4 ETQ399 System-nr = '02', GwService = 'sapgw02' Client = '000'
1 ETQ359 RFC Login to: System="EHX", AsHost="sapehx01" Nr="02", GwHost="sapehx01", GwService="sapgw02"
2 ETQ232 RFC Login succeeded
4 ETQ010 Date & Time: 20150721213615
1 ETQ233 Calling function module "UPG_IS_SHADOW_SYSTEM" by RFC
2 ETQ373 parameter "IV_R3UP" = "X"
1 ETQ234 Call of function module "UPG_IS_SHADOW_SYSTEM" by RFC succeeded
4 ETQ010 Date & Time: 20150721213615
2 ETQ373 parameter "EV_SHADOW" = "X"
1 ETQ399 SYSTEM HEALTH MANAGER: check for instance processlist.
1 ETQ399 SAPCONTROL MANAGER: getProcessList with host: sapehx01 and instance: 02
3 ETQ120 20150721213616: PID 10813590 execute '/usr/sap/EHP7/SUM/abap/exe/sapcontrol -format script -prot NI_HTTP -host sapehx01 -nr 02 -function GetProcessList', output written to '/usr/sap/EHP7/SUM/abap/log/SAPup.ECO'.
3 ETQ122 20150721213616: PID 10813590 exited with status 3 (time: 0.000/0.060/0.112 real/usr/sys)
1 ETQ399 SYSTEM HEALTH MANAGER: System up and running, go on with stop action
1 ETQ399 SAPCONTROL MANAGER: stopWait with host: sapehx01 and instance: 02
3 ETQ120 20150721213616: PID 10813592 execute '/usr/sap/EHP7/SUM/abap/exe/sapcontrol -prot NI_HTTP -host sapehx01 -nr 02 -function StopWait 300 10', output written to '/usr/sap/EHP7/SUM/abap/log/SAPup.ECO'.
LOG of SAPup.ECO:
EXECUTING /usr/sap/EHP7/SUM/abap/exe/sapcontrol -format script -prot NI_HTTP -host sapeht01 -nr 02 -function GetProcessList
21.07.2015 21:36:16
GetProcessList
OK
0 name: msg_server
0 description: MessageServer
0 dispstatus: GREEN
0 textstatus: Running
0 starttime: 2015 07 21 15:16:43
0 elapsedtime: 6:19:33
0 pid: 7143444
1 name: enserver
1 description: EnqueueServer
1 dispstatus: GREEN
1 textstatus: Running
1 starttime: 2015 07 21 15:16:43
1 elapsedtime: 6:19:33
1 pid: 10748044
2 name: disp+work
2 description: Dispatcher
2 dispstatus: GREEN
2 textstatus: Running
2 starttime: 2015 07 21 15:16:43
2 elapsedtime: 6:19:33
2 pid: 15335606
3 name: gwrd
3 description: Gateway
3 dispstatus: GREEN
3 textstatus: Running
3 starttime: 2015 07 21 15:16:44
3 elapsedtime: 6:19:32
3 pid: 14287024
SAPup> Process with PID 10813590 terminated with status 3 at 20150721213616!
SAPup> Starting subprocess with PID 10813592 in phase 'STOP_SHDI_SHD2' at 20150721213616
ENV: DIR_LIBRARY=/usr/sap/EHP7/SUM/abap/exe
ENV: JAVA_HOME=/usr/java14_64
ENV: LIBPATH=/usr/sap/EHP7/SUM/abap/exe:/usr/sap/EHP7/SUM/jvm/jre/lib/ppc64/server:/usr/sap/EHP7/SUM/jvm/jre/lib/ppc64:/usr/sap/EHP7/SUM/jvm/jre/../lib/ppc64:/usr/sap/EHP7/SUM/jvm/jre/lib/ppc64/jli:/usr/sap/EHP7/SUM/jvm/jre/../lib/ppc64/jli:/usr/lib:/lib:/usr/sap/EHT/SYS/exe/run:/oracle/client/11x_64/instantclient
ENV: NLS_LANG=AMERICAN_AMERICA.WE8DEC
ENV: ORACLE_BASE=/oracle
ENV: ORACLE_SID=EHT
ENV: PATH=/usr/sap/EHP7/SUM/abap/exe:/usr/java14_64/bin:.:/export/home/ehtadm:/usr/sap/EHT/SYS/exe/run:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java5/jre/bin:/usr/java5/bin
ENV: SAPDATA_HOME=/oracle/EHT
ENV: SAPSYSTEMNAME=EHT
ENV: dbms_type=ORA
ENV: dbs_ora_schema=SAPR3
ENV: dbs_ora_tnsname=EHT
PWD: /usr/sap/EHP7/SUM/abap/tmp
EXECUTING /usr/sap/EHP7/SUM/abap/exe/sapcontrol -prot NI_HTTP -host sapeht01 -nr 02 -function StopWait 300 10
....................................................................................................................................................................................
So that where its hanging and not going forward and it also not erroring in out.
Any valuable suggession would be great as we need reach deadline.
Thanks