Kevin Kempf's Blog

July 28, 2010

Not Feasible to Fix

Filed under: 11i, Discoverer — kkempf @ 1:15 pm

Stopping Discoverer 10g

Anyone familiar with Discoverer 10g on AS 10g (10.1.2.3) has probably seen the following error message when trying to use opmnctl to stop Discoverer:

opmnctl: stopping opmn and all managed processes...
================================================================================
opmn id=hostname-01:6200
    6 of 7 processes stopped.

ias-instance id=BI10g_PROD.hostname-01.domain.com
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--------------------------------------------------------------------------------
ias-component/process-type/process-set:
    OC4J/OC4J_BI_Forms/default_island

Error
--> Process (pid=8331)
    time out while waiting for a managed process to stop
    Log:
    /u02/appprod/proddisco/opmn/logs/OC4J~OC4J_BI_Forms~default_island~1

For whatever reason, last maintenance windows I decided I was sick of waiting on this thing, and wanted to get rid of the ugly (but harmless) error message.   I was amused by what I found.

Bug 7357325

From My Oracle Support:

Solution
To implement the solution, please execute the following steps:

Discoverer development back-ported a fix to the second Discoverer cumulative patch on top of Oracle Application Server patchset 3 (10.1.2.3); however since then, per Bug 7357325, this workaround fix has been removed as it causes other regressions. Development has marked this as not feasible to fix in 10g. The message is harmless and can be avoided once all the Discoverer sessions exit cleanly (see above note).  We do recommend that you apply the latest cumulative patch, per Note 237607 ‘ALERT: Required and Recommended Patch Levels For All Discoverer Versions’

Goodness.  I hope they don’t cop that attitude with too many products.

Advertisements

July 26, 2010

For those still on RDBMS 10g

Filed under: 10g — kkempf @ 3:50 pm

Just a friendly reminder about 10.2.0.5

Oracle released the terminal RDBMS 10g patch this spring, and if you can apply it before the end of July it looks like it buys you a year of extended support fees as Steven Chan pointed out here.   It feels largely like an another upgrade for the sake of upgrading, and I have a few ash & trash databases which I need to upgrade this week.  You can find the download (8202632) here for most platforms; oddly Linux x86 is absent, unsure what the deal is with that.

July 23, 2010

Securing EM 11g with your own SSL cert

Filed under: Enterprise Manager, Oracle — kkempf @ 6:24 pm

A bit of a teaser

I recently had a miserable SR with Oracle regarding how to replace their “canned” certificate (part of the 11g install) with my own, real domain certificate.  In the end, I was told that unless I was using SSO, it was not supported.

This answer didn’t sit right with me, but I would never call SSL and certificates one of my strengths.  When my sysadmin/Linux counterpart heard this answer, he spent the day trying to prove Oracle wrong.. I sat shotgun for the ride.

The Short Answer

I’m sure it’s completely unsupported, but it’s not an external product; it’s only used within IT so it felt safe enough to “play” in.  In the end, he figured out how to get our certificate and key into a p12 wallet file via a pki tool and landed them on the server (.sso and .p12 file) where the existing (Oracle HTTP server) files were.  Yeah, that’s right.  EM 11g runs on Weblogic, which only serves as a container for the old Oracle HTTP Server.  In other words, it’s iAS running inside Weblogic.  The actual certificates are under an OHS directory.  Once replaced, the main website was using “our” certificate (https://hostname:1159/em).  Then the agents stopped communicating with the OMS; the temporary fix was to go do an emctl unsecure agent on all the servers.  Not ideal, we’ll work more on that later.  Consider this a “post in progress”.  Am I the only one who finds it ridiculous that Oracle “doesn’t support” using a real cert for EM?

The Messages I no longer see

Chrome:
The site’s security certificate is not trusted!
You attempted to reach hostname.com, but the server presented a certificate issued by an entity that is not trusted by your computer’s operating system. This may mean that the server has generated its own security credentials, which Google Chrome cannot rely on for identity information, or an attacker may be trying to intercept your communications. You should not proceed, especially if you have never seen this warning before for this site.

IE:
There is a problem with this website’s security certificate.

The security certificate presented by this website was not issued by a trusted certificate authority.

Security certificate problems may indicate an attempt to fool you or intercept any data you send to the server.
We recommend that you close this webpage and do not continue to this website.
Click here to close this webpage.
Continue to this website (not recommended).

Overview of the solution

If you can find a way to get your certificate, key and any intermediate certificates into a standard keystore, you’re in good shape.  We had a problem because apparently Oracle doesn’t like wildcards in the certificate name (*.domainname.com).  So the admin used the pki tool to create a p12, then imported it into Oracle Wallet Manager.  From here, you can select the Wallet menu option, then “Auto Logon”.  Now when you save it, you get the .sso file as well.  Land the cwallet.sso and the ewallet.p12 file in gc_inst/WebTierIH1/config/OHS/ohs1/keystores/upload and restart services.  Remember to unsecure your agents; I’ll update the post if we figure out what’s messing those up.

Follow Up

It appears that the fix which we stumbled upon can be mostly accomplished (except never in the case of wildcard certificates) from the command line with emctl.

To secure the console, use emctl secure console and point it to your own certificate wallet directory.

To secure the oms, use emctl secure oms and point it to your wallet and trusted cert location.  There is a catch to this, however.  You cannot use a *.domainname.com (wildcard) cert for this, as the agents pull this certificate and attempt to connect to *.domainname.com (instead of a real server name).  In our case, if we had a certificate with the exact server EM is running on, this would have worked as well.

Another oddity is the number of ports opened and said to be in use (which may or may not be the case).  In any case, I recommend typing emctl status oms -details to determine what ports are actually available for login.  They may or may not correspond to what WebLogic says.  In our case, https for our own certificate changed to 4444 by default.  We over-rode ssl.conf to change it to 8443, but it’s still a bit of conFUSION as to why Weblogic rarely agrees with EM on what is running where.

July 22, 2010

A really nasty 11i bug

Filed under: 11i, Bugs — kkempf @ 3:32 pm

When is a user not a user anymore?

In 11i, we tend to end-date users’ responsibilities, and assume that’s the end of their ability to run jobs in the Apps.  Not always the case, it turns out.

So there I was, minding my own business

I’d actually seen this bug before, but at the time, when I opened an SR, the analyst “stumbled over” the solution (stop concurrent managers, run cmclean.sql, kill all pending/no manager jobs, start concurrent manager), without identifying the root cause.  So when my 11i front end became near-unresponsive, and I saw this query running at 100% on every CPU, I at least recognized it:

.
SELECT  R.Conc_Login_Id, R.Request_Id, R.Phase_Code, R.Status_Code, P.Application_ID, P.Concurrent_Program_ID, P.Concurrent_Program_Name, R.Enable_Trace, R.Restart, DECODE(R.Increment_Dates, ‘Y’, ‘Y’, ‘N’), R.NLS_Compliant, R.OUTPUT_FILE_TYPE, E.Executable_Name, E.Execution_File_Name, A2.Basepath, DECODE(R.Stale, ‘Y’, ‘C’, P.Execution_Method_Code), P.Print_Flag, P.Execution_Options, DECODE(P.Srs_Flag, ‘Y’, ‘Y’, ‘Q’, ‘Y’, ‘N’), P.Argument_Method_Code, R.Print_Style, R.Argument_Input_Method_Code, R.Queue_Method_Code, R.Responsibility_ID, R.Responsibility_Application_ID, R.Requested_By, R.Number_Of_Copies, R.Save_Output_Flag, R.Printer, R.Print_Group, R.Priority, U.User_Name, O.Oracle_Username, O.Encrypted_Oracle_Password, R.Cd_Id, A.Basepath, A.Application_Short_Name, TO_CHAR(R.Requested_Start_Date,’YYYY/MM/DD HH24:MI:SS’), R.Nls_Language, R.Nls_Territory, DECODE(R.Parent_Request_ID, NULL, 0, R.Parent_Request_ID), R.Priority_Request_ID, R.Single_Thread_Flag, R.Has_Sub_Request, R.Is_Sub_Request, R.Req_Information, R.Description, R.Resubmit_Time, TO_CHAR(R.Resubmit_Interval), R.Resubmit_Interval_Type_Code, R.Resubmit_Interval_Unit_Code, TO_CHAR(R.Resubmit_End_Date,’YYYY/MM/DD HH24:MI:SS’), Decode(E.Execution_File_Name, NULL, ‘N’, Decode(E.Subroutine_Name, NULL, Decode(E.Execution_Method_Code, ‘I’, ‘Y’, ‘J’, ‘Y’, ‘N’), ‘Y’)), R.Argument1, R.Argument2, R.Argument3, R.Argument4, R.Argument5, R.Argument6, R.Argument7, R.Argument8, R.Argument9, R.Argument10, R.Argument11, R.Argument12, R.Argument13, R.Argument14, R.Argument15, R.Argument16, R.Argument17, R.Argument18, R.Argument19, R.Argument20, R.Argument21, R.Argument22, R.Argument23, R.Argument24, R.Argument25, X.Argument26, X.Argument27, X.Argument28, X.Argument29, X.Argument30, X.Argument31, X.Argument32, X.Argument33, X.Argument34, X.Argument35, X.Argument36, X.Argument37, X.Argument38, X.Argument39, X.Argument40, X.Argument41, X.Argument42, X.Argument43, X.Argument44, X.Argument45, X.Argument46, X.Argument47, X.Argument48, X.Argument49, X.Argument50, X.Argument51, X.Argument52, X.Argument53, X.Argument54, X.Argument55, X.Argument56, X.Argument57, X.Argument58, X.Argument59, X.Argument60, X.Argument61, X.Argument62, X.Argument63, X.Argument64, X.Argument65, X.Argument66, X.Argument67, X.Argument68, X.Argument69, X.Argument70, X.Argument71, X.Argument72, X.Argument73, X.Argument74, X.Argument75, X.Argument76, X.Argument77, X.Argument78, X.Argument79, X.Argument80, X.Argument81, X.Argument82, X.Argument83, X.Argument84, X.Argument85, X.Argument86, X.Argument87, X.Argument88, X.Argument89, X.Argument90, X.Argument91, X.Argument92, X.Argument93, X.Argument94, X.Argument95, X.Argument96, X.Argument97, X.Argument98, X.Argument99, X.Argument100, R.number_of_arguments, C.CD_Name, NVL(R.Security_Group_ID, 0)
FROM fnd_concurrent_requests R, fnd_concurrent_programs P, fnd_application A, fnd_user U, fnd_oracle_userid O, fnd_conflicts_domain C, fnd_concurrent_queues Q, fnd_application A2, fnd_executables E, fnd_conc_request_arguments X
WHERE R.Status_code = ‘I’ And ((R.OPS_INSTANCE is null) or (R.OPS_INSTANCE = -1) or (R.OPS_INSTANCE = decode(:dcp_on,1,FND_CONC_GLOBAL.OPS_INST_NUM,R.OPS_INSTANCE))) And R.Request_ID = X.Request_ID(+) And R.Program_Application_Id = P.Application_Id(+) And R.Concurrent_Program_Id = P.Concurrent_Program_Id(+) And R.Program_Application_Id = A.Application_Id(+) And P.Executable_Application_Id = E.Application_Id(+) And P.Executable_Id = E.Executable_Id(+) And P.Executable_Application_Id = A2.Application_Id(+) And R.Requested_By = U.User_Id(+) And R.Cd_Id = C.Cd_Id(+) And R.Oracle_Id = O.Oracle_Id(+) And Q.Application_Id = :q_applid And Q.Concurrent_Queue_Id = :queue_id And (P.Enabled_Flag is NULL OR P.Enabled_Flag = ‘Y’) And R.Hold_Flag = ‘N’ And R.Requested_Start_Date <= Sysdate And ( R.Enforce_Seriality_Flag = ‘N’ OR ( C.RunAlone_Flag = P.Run_Alone_Flag And (P.Run_Alone_Flag = ‘N’ OR Not Exists
(Select Null
From Fnd_Concurrent_Requests Sr
Where Sr.Status_Code In (‘R’, ‘T’) And Sr.Enforce_Seriality_Flag = ‘Y’ And Sr.CD_id = C.CD_Id)))) And Q.Running_Processes <= Q.Max_Processes And R.Rowid = :reqname And ((P.Execution_Method_Code != ‘S’ OR (R.PROGRAM_APPLICATION_ID,R.CONCURRENT_PROGRAM_ID) IN ((0,98),(0,100),(0,31721),(0,31722),(0,31757))) AND ((R.PROGRAM_APPLICATION_ID,R.CONCURRENT_PROGRAM_ID) NOT IN ((510,40112),(510,40113),(510,40910),(510,40911),(530,42260),(530,42261),(535,40893),(535,40894),(535,40895),(660,44069),(660,44100),(660,44808),(660,44849),(660,46421),(660,46422),(702,33152),(20004,44868),(20009,45000)))) FOR UPDATE OF R.status_code NoWait
SELECT  R.Conc_Login_Id, R.Request_Id, R.Phase_Code, R.Status_Code, P.Application_ID, P.Concurrent_Program_ID, P.Concurrent_Program_Name, R.Enable_Trace, R.Restart, DECODE(R.Increment_Dates, ‘Y’, ‘Y’, ‘N’), R.NLS_Compliant, R.OUTPUT_FILE_TYPE, E.Executable_Name, E.Execution_File_Name, A2.Basepath, DECODE(R.Stale, ‘Y’, ‘C’, P.Execution_Method_Code), P.Print_Flag, P.Execution_Options, DECODE(P.Srs_Flag, ‘Y’, ‘Y’, ‘Q’, ‘Y’, ‘N’), P.Argument_Method_Code, R.Print_Style, R.Argument_Input_Method_Code, R.Queue_Method_Code, R.Responsibility_ID, R.Responsibility_Application_ID, R.Requested_By, R.Number_Of_Copies, R.Save_Output_Flag, R.Printer, R.Print_Group, R.Priority, U.User_Name, O.Oracle_Username, O.Encrypted_Oracle_Password, R.Cd_Id, A.Basepath, A.Application_Short_Name, TO_CHAR(R.Requested_Start_Date,’YYYY/MM/DD HH24:MI:SS’), R.Nls_Language, R.Nls_Territory, DECODE(R.Parent_Request_ID, NULL, 0, R.Parent_Request_ID), R.Priority_Request_ID, R.Single_Thread_Flag, R.Has_Sub_Request, R.Is_Sub_Request, R.Req_Information, R.Description, R.Resubmit_Time, TO_CHAR(R.Resubmit_Interval), R.Resubmit_Interval_Type_Code, R.Resubmit_Interval_Unit_Code, TO_CHAR(R.Resubmit_End_Date,’YYYY/MM/DD HH24:MI:SS’), Decode(E.Execution_File_Name, NULL, ‘N’, Decode(E.Subroutine_Name, NULL, Decode(E.Execution_Method_Code, ‘I’, ‘Y’, ‘J’, ‘Y’, ‘N’), ‘Y’)), R.Argument1, R.Argument2, R.Argument3, R.Argument4, R.Argument5, R.Argument6, R.Argument7, R.Argument8, R.Argument9, R.Argument10, R.Argument11, R.Argument12, R.Argument13, R.Argument14, R.Argument15, R.Argument16, R.Argument17, R.Argument18, R.Argument19, R.Argument20, R.Argument21, R.Argument22, R.Argument23, R.Argument24, R.Argument25, X.Argument26, X.Argument27, X.Argument28, X.Argument29, X.Argument30, X.Argument31, X.Argument32, X.Argument33, X.Argument34, X.Argument35, X.Argument36, X.Argument37, X.Argument38, X.Argument39, X.Argument40, X.Argument41, X.Argument42, X.Argument43, X.Argument44, X.Argument45, X.Argument46, X.Argument47, X.Argument48, X.Argument49, X.Argument50, X.Argument51, X.Argument52, X.Argument53, X.Argument54, X.Argument55, X.Argument56, X.Argument57, X.Argument58, X.Argument59, X.Argument60, X.Argument61, X.Argument62, X.Argument63, X.Argument64, X.Argument65, X.Argument66, X.Argument67, X.Argument68, X.Argument69, X.Argument70, X.Argument71, X.Argument72, X.Argument73, X.Argument74, X.Argument75, X.Argument76, X.Argument77, X.Argument78, X.Argument79, X.Argument80, X.Argument81, X.Argument82, X.Argument83, X.Argument84, X.Argument85, X.Argument86, X.Argument87, X.Argument88, X.Argument89, X.Argument90, X.Argument91, X.Argument92, X.Argument93, X.Argument94, X.Argument95, X.Argument96, X.Argument97, X.Argument98, X.Argument99, X.Argument100, R.number_of_arguments, C.CD_Name, NVL(R.Security_Group_ID, 0)FROM fnd_concurrent_requests R, fnd_concurrent_programs P, fnd_application A, fnd_user U, fnd_oracle_userid O, fnd_conflicts_domain C, fnd_concurrent_queues Q, fnd_application A2, fnd_executables E, fnd_conc_request_arguments XWHERE R.Status_code = ‘I’ And ((R.OPS_INSTANCE is null) or (R.OPS_INSTANCE = -1) or (R.OPS_INSTANCE = decode(:dcp_on,1,FND_CONC_GLOBAL.OPS_INST_NUM,R.OPS_INSTANCE))) And R.Request_ID = X.Request_ID(+) And R.Program_Application_Id = P.Application_Id(+) And R.Concurrent_Program_Id = P.Concurrent_Program_Id(+) And R.Program_Application_Id = A.Application_Id(+) And P.Executable_Application_Id = E.Application_Id(+) And P.Executable_Id = E.Executable_Id(+) And P.Executable_Application_Id = A2.Application_Id(+) And R.Requested_By = U.User_Id(+) And R.Cd_Id = C.Cd_Id(+) And R.Oracle_Id = O.Oracle_Id(+) And Q.Application_Id = :q_applid And Q.Concurrent_Queue_Id = :queue_id And (P.Enabled_Flag is NULL OR P.Enabled_Flag = ‘Y’) And R.Hold_Flag = ‘N’ And R.Requested_Start_Date <= Sysdate And ( R.Enforce_Seriality_Flag = ‘N’ OR ( C.RunAlone_Flag = P.Run_Alone_Flag And (P.Run_Alone_Flag = ‘N’ OR Not Exists(Select NullFrom Fnd_Concurrent_Requests SrWhere Sr.Status_Code In (‘R’, ‘T’) And Sr.Enforce_Seriality_Flag = ‘Y’ And Sr.CD_id = C.CD_Id)))) And Q.Running_Processes <= Q.Max_Processes And R.Rowid = :reqname And ((P.Execution_Method_Code != ‘S’ OR (R.PROGRAM_APPLICATION_ID,R.CONCURRENT_PROGRAM_ID) IN ((0,98),(0,100),(0,31721),(0,31722),(0,31757))) AND ((R.PROGRAM_APPLICATION_ID,R.CONCURRENT_PROGRAM_ID) NOT IN ((510,40112),(510,40113),(510,40910),(510,40911),(530,42260),(530,42261),(535,40893),(535,40894),(535,40895),(660,44069),(660,44100),(660,44808),(660,44849),(660,46421),(660,46422),(702,33152),(20004,44868),(20009,45000)))) FOR UPDATE OF R.status_code NoWait

What Enterprise Manager was telling me

The database itself was reasonably happy.  I mean it was running a stupid query over and over, but it was doing its job.  It was the application server that was getting killed

As you can see, the application server got slammed about 11:15am.  It was running out of memory, the load was in the 20’s (8 CPUs available) and Linux was starting to kill processes in order to preserver itself.

Root Cause and Fix

If you ever see this query, or possibly this one:

update  fnd_concurrent_requests set phase_code = :phase ,status_code = :status, completion_text = :text where request_id = :reqid

you are likely encountering bug 9109247: SCHEDULED REQUESTS STILL RUN FOR END_DATED USERS 100% CPU.

The immediate fix is to shut down concurrent managers (adcmctl.sh) and look in the concurrent request queue to see who is trying to run what (start with the lowest request ID which is trying to run).  In my case, one user (HFAULLING) had submitted about 20 similar requests.  I cancelled these requests (in fact, all of this users’ requests), can cmclean.sql and restarted the managers.

Turns out, this user was end dated the day this happened.  In other words, they weren’t even a valid Oracle apps user anymore.  But this bug allowed their scheduled concurrent requests to run, get stuck in some crazy endless loop, and kill my system (since he had submitted more requests than I had standard managers and they all queued up at the same time).

In closing

I decided to post this information for two reasons.  One, I think it’s rather random and impossible to predict when you may hit this bug.  It’s my opinion that this should have received a lot more press from Oracle as a recommended patch!  Wouldn’t it be rather embarrassing if it got out that Oracle 11i not only allowed terminated employees to run concurrent requests, but by doing so they could effectively sabotage the system?  Second, it’s a bug which can have catastrophic consequences to the whole ERP.  The first time I hit this bug I extracted the (long) query (above) and submitted it to the analyst asking them what the h*ll it was.  At the time, they had no idea.  Seems like it should be in the various notes related to the bug as well.


July 21, 2010

Where are my 11i manager logfiles?

Filed under: 11i — kkempf @ 1:04 pm

I pieced together a nifty query today while troubleshooting my workflow notification mailer.  If you’ve ever wondered which log file is for the various managers running, this may be of use to you.

select
  concurrent_queue_name manager
  ,fcp.concurrent_process_id pid
  ,meaning
  ,fcp.logfile_name
from
  fnd_concurrent_queues    fcq
  ,fnd_concurrent_processes fcp
  ,fnd_lookups              flkup
where
  fcq.concurrent_queue_id = fcp.concurrent_queue_id
and
  fcq.application_id = fcp.queue_application_id
and
  flkup.lookup_code=fcp.process_status_code
and
  lookup_type ='CP_PROCESS_STATUS_CODE'
and
  meaning='Active'
order by
  manager
;

The output will look something like this:

MANAGER PID MEANING LOGFILE_NAME
CSTROLLUP 53985 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53985.mgr
FNDCPOPP 53912 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/FNDOPP53912.txt
FNDCRM 53915 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/c53915.mgr
FNDICM 53898 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/DEV_0714.mgr
FNDSCH 53916 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/s53916.mgr
FNDSM_ORADEV-02 53900 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/FNDSM53900.mgr
INVMGR 53984 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53984.mgr
INVMGR 53983 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53983.mgr
INVMGR 53982 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53982.mgr
INVMGR 53981 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53981.mgr
INVTMRPM 53903 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53903.mgr
INVTMRPM 53902 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53902.mgr
INVTMRPM 53899 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53899.mgr
INVTMRPM 53901 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53901.mgr
INVTMRPM 53904 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53904.mgr
OAMCOLMGR 53928 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53928.mgr
PODAMGR 53910 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53910.mgr
PODAMGR 53909 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53909.mgr
PODAMGR 53908 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53908.mgr
RCVOLTM 53905 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53905.mgr
RCVOLTM 53906 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53906.mgr
RCVOLTM 53907 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/t53907.mgr
STANDARD 53936 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53936.mgr
STANDARD 53935 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53935.mgr
STANDARD 53929 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53929.mgr
STANDARD 53930 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53930.mgr
STANDARD 53931 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53931.mgr
STANDARD 53932 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53932.mgr
STANDARD 53933 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53933.mgr
STANDARD 53934 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53934.mgr
WFALSNRSVC 53914 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/FNDCPGSC53914.txt
WFMLRSVC 53913 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/FNDCPGSC53913.txt
WFWSSVC 53911 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/FNDCPGSC53911.txt
DW_MANAGER 53979 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53979.mgr
DW_MANAGER 53980 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53980.mgr
LONG_REQUEST_MANAGER 53922 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53922.mgr
LONG_REQUEST_MANAGER 53921 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53921.mgr
LONG_REQUEST_MANAGER 53920 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53920.mgr
LONG_REQUEST_MANAGER 53919 Active /u02/appdevl/devcomn/admin/log/DEV_oradev-02/w53919.mgr

July 16, 2010

Starting and stopping Discoverer 11g

Filed under: 11i, Discoverer, Support — kkempf @ 9:20 am

I’ve learned a few things about Discoverer 11g in the past few weeks.

  • Apparently, the model for Discoverer 11g on Linux is: start it once from the installer and pray it never shuts down and that you never need to restart the server.
  • Weblogic people at support don’t have any idea what Discoverer is, but they’re happy to help you troubleshoot the weblogic pieces.  In other words, nobody seems willing to embrace the whole product.  conFusion has arrived!
  • The most difficult part of a Discoverer 11g upgrade/install is figuring out how to start and stop the server

Support Woes

Document 1099996.1 explains the gist of starting/stopping Discoverer 11g on linux (via command prompt).  Not only did the Weblogic analyst I had not know about Discoverer 11g, he was also unfamiliar with Linux, and was apparently helping me troubleshoot my problems via a Windows install of Weblogic.  Seems like not such a great fit.  He was further confounded by the fact that a web conference was unavailable, as my system (OS & browser) failed the checks on their site.

Guess I’ll figure it out myself

At some point in most SRs, it becomes clear that if I really want the solution, it won’t be coming from my analyst.  If I manage to figure it out, I’ll keep the SR open to see if my analyst ever does.  Usually I get annoyed with their irrelevant questions and close the SR as “Determined solution myself”.  Somehow, those SRs are always “Hard Closed” by the analyst as “Solution Provided”.   This is one of those cases.

The gist of the problem starting and stopping all the pieces of Discoverer 11g on Linux is that there’s no good documentation.  Through trial and error, I got this thing working.  There were major problems:

  • The scripts laid out in doc 1099996.1 are not like normal Oracle scripts which automatically run in the background.  They have to be run nohup (no hangup) and with & (background).  This isn’t a problem per se, except some of them are interactive.  Meaning some of these scripts ask for things like username and password.  This doesn’t work nohup’d and backgrounded.
  • There’s too many scripts.  4 scripts to start discoverer; in 10g there were 2 (opmnctl startall and emctl start iasconsole).  It’s a sorry step in the wrong direction.
  • For whatever reason, there’s a t3 in front of the URL argument passed to many of the scripts.  I have no idea why, but it’s required.
  • /yourpath/Middleware/user_projects/domains/ClassicDomain/bin/startManagedWebLogic.sh requires you to hard code values in their script.  Yes, you need to vi the file and put your username and password in the script under WLS_USER and WLS_PW or it will not start except in the foreground on your screen (interactively).  If you do this, you cannot ever close that screen, or Discoverer stops.
    • WLS_USER=”weblogic”
    • WLS_PW=”yourpassword
    • To add insult to injury, the script exports these values, meaning they’re available variables to the host.

Startdisco.sh

#!/bin/bash

export MIDDLEWARE_HOME=/u02/appdevl/devdisco/Middleware

export DOMAIN_HOME=$MIDDLEWARE_HOME/user_projects/domains/ClassicDomain

export WL_HOME=$MIDDLEWARE_HOME/wlserver_10.3

export ORACLE_HOME=/u02/appdevl/devdisco/Middleware/as_1

export ORACLE_INSTANCE=/u02/appdevl/devdisco/Middleware/asinst_1

rm -rf nohup.out

rm -rf /tmp/wls_start.log

rm -rf /tmp/start_nodemanager.log

rm -rf /tmp/start_mgdwls.log

echo “Ensure NO processes related to disco 11g are running or this will fail”

nohup $DOMAIN_HOME/bin/startWebLogic.sh -Dweblogic.management.username=weblogic -Dweblogic.management.password=disco11g > /tmp/wls_start.log &

nohup $WL_HOME/server/bin/startNodeManager.sh > /tmp/start_nodemanager.log &

echo “sleeping”

sleep 60

nohup $DOMAIN_HOME/bin/startManagedWebLogic.sh WLS_DISCO t3://(server.domain):7001 > /tmp/start_mgdwls.log &

echo “sleeping”

sleep 60

$ORACLE_INSTANCE/bin/opmnctl startall

$ORACLE_INSTANCE/bin/opmnctl status

echo “If Discoverer doesn’t start properly, login to http://(server.domain):7001/console”

echo “From the home page, click servers (Under Environment), then the control tab.  Check WLS_DISCO then click the start button below the checkbox”

Stopdisco.sh

#!/bin/bash
export MIDDLEWARE_HOME=/u02/appdevl/devdisco/Middleware
export DOMAIN_HOME=$MIDDLEWARE_HOME/user_projects/domains/ClassicDomain
export WL_HOME=$MIDDLEWARE_HOME/wlserver_10.3
export ORACLE_HOME=/u02/appdevl/devdisco/Middleware/as_1
export ORACLE_INSTANCE=/u02/appdevl/devdisco/Middleware/asinst_1
$ORACLE_INSTANCE/bin/opmnctl stopall
$ORACLE_HOME/opmn/bin/opmnctl status
# stop the discoverer server
$DOMAIN_HOME/bin/stopManagedWebLogic.sh WLS_DISCO t3://(server.domain):7001 weblogic disco11g
# stop the weblogic admin server
$DOMAIN_HOME/bin/stopWebLogic.sh
killall startNodeManager.sh
killall nmz
echo “There’s probably still 1 disco process running – ps -ef|grep jrockit and kill it (java)”
echo “It’s probably pid:”
ps -ef|grep jrockit|cut -c10-15

July 14, 2010

Tethering a Blackberry 9630 via Bluetooth to Ubuntu Lucid Lynx (10.04)

Filed under: Blackberry, Linux — kkempf @ 1:44 pm

Lucid Lynx

Aside from the name Lucid Lynx (which I find annoying to remember), I love Ubuntu as a work platform.  I’m proud to say I haven’t had to fire up my Windows VM for weeks.  One pleasant surprise when switching from CentOS (Red Hat) to Ubuntu was the ease in setting up a bluetooth-tethered Blackberry modem. Under CentOS, it was rather awkward and obscure; under Ubuntu it’s all available for you to click on in the task bar.

The Basics

It’s somewhat helpful to know that Ubuntu (and CentOS as I recall) treats a Blackberry modem as a Dial Up Networking modem; in other words, it’s reminiscent of the old dial up days, except you don’t have to hear the modem static.  I’m writing this entry post-install, and therefore it’s entirely possible I’m missing some minor nuances and screenshots.  If so, let me know what they are and I’ll add them.  In particular, I don’t recall how the network connection for Mobile Broadband (Blackberry) shows up; it may be automatic, it may take intervention.

Assumptions

  • You’re using Ubuntu 10.04 Lucid Lynx (LTS)
  • You have a Blackberry with the data plan enabled (mine is a 9630; this topic seems generic enough that it should work on any)
  • You have bluetooth available on your PC/Notebook

Pairing

Bluetooth likes to see things paired before use.  Unless you’ve somehow left the devices wide open, the PC and the device need to pair or handshake a first time, to confirm they aren’t being hijacked.

The first step is to figure out how to make your Blackberry discoverable:

  • Go to options
  • Select Bluetooth
  • Hit the Blackberry button
  • Select Options (again)
  • Ensure Discoverable is set to Yes
  • It may be handy to note the Device Name and Device Address (MAC Address)

Discover the Blackberry from Ubuntu

  • Ensure the Bluetooth Icon is in the bottom right corner of your screen, on the task bar.   If it’s not, possible causes include that the bluetooth service isn’t running, or perhaps you switched off the wireless switch on your notebook?

Set Up Dial Up Networking & Pair the Devices

Left click on the Bluetooth Icon in the bottom right corner of the taskbar, and click the search button.

Select the device you just paired, then right click and hit Add Device

Now it wants to pair; the Blackberry asks you for a PIN; type it in and hit enter.

It now wants that same PIN on your PC (type in the PIN you just entered on the Blackberry):

Now you can right click on the device and hit trust; this adds a shiny yellow star icon to the picture.

Finally, right click on the device and hit Dialup Networking.  I can’t remember if the first time I did it I had to put in the dial in number or it somehow figured it out.  Regardless, here’s what it looks like when it’s running; note that there’s a little green light on the bluetooth icon in the lower right, indicating a bluetooth connection, a desktop alert on the top right telling me that I’m connected to Verizon, and colored blinky lights in the Bluetooth Device window indicating received signal strength (orange), link quality (green) and transmit power level (blue).  Right click again on the device and select disconnect to hang up.

Worth noting, if you right click on the network manager in the bottom right and select edit connections, ensure you have “connect automatically” checked for the Blackberry connection.  To be honest, I don’t remember the details but this removes one extra mouse click when you dial in with the Blackberry.

I thought it may be useful to see what Bluetooth-related packages I’ve got installed via the Ubuntu Software Center

Blog at WordPress.com.