Kevin Kempf's Blog

February 22, 2011

Discoverer certified with 11i

Filed under: 11i, Discoverer — kkempf @ 10:48 am

Of Course

Seems like Discoverer is the only thing I’m posting about these days.  No sooner had I completed the upgrade to when Oracle certified with 11i.

I’ve noticed it only runs on WLS 10.3.4, so that’s a new install too.  It appears that there is some sort of “Smart Update” program to update Weblogic Server to 10.3.4 and the patch set installer will upgrade me from (or if I were using that) to   We’ll see.   At least Oracle has been consistent about incrementing their versioning:

  • Oracle Fusion Middleware is certified with WebLogic Server 10.3.1
  • Oracle Fusion Middleware is certified with WebLogic Server 10.3.2
  • Oracle Fusion Middleware is certified with WebLogic Server 10.3.3
  • Oracle Fusion Middleware is certified with WebLogic Server 10.3.4

Upgrade Complete

I’ve successfully upgraded my DEV environment to  It wasn’t as painful as to  Some parts were a pleasant surprise, others were what I would consider to be almost expected failure points now that I’ve run this installer a few times.

Upgrading Weblogic Server

After shutting everything down, you first upgrade Weblogic to 10.3.4.  This is pretty easy.  To find this patch,  go to My Oracle Support, Patches & Updates, Product or Family (Advanced Search).  Type in Oracle Weblogic Server as the Product, and WLS 10.3.4 as the Release.   Hit search and you get appropriate choices (appropriate if you consider PLACEHOLDER BUG FOR WEBLOGIC SERVER 11GR1 (10.3.4) UPGRADE INSTALLER a good name for a patch).  Note how it says upgrade installer.  Well pull this, invoke your environment, run the binary on your Discoverer server, and uncheck “Launch Quickstart”.  Poof!  You’re done.

Upgrading Discoverer

For me, the worst part of this upgrade was finding and downloading the applicable patch.  When I got to My Oracle Support, click on Patches & Updates,  and do a “Product or Family (Advanced Search)” and punch in Oracle Discoverer and version iAS I get 0 results under Linux x86, Linux x86-64, Windows 32-bit.  I know iAS isn’t really what it is (I believe it should read FMW or possibly WLS but given that this is the first version of Discoverer to be called, it seems the most logical.  I hate the new My Oracle Support.  It consistently fails, in almost every conceivable way.  In the end, I found the patch numbers (not available on OTN, by the way) in an obscure location in one of the readme’s about how to upgrade FMW, WLS or Discoverer (I forget which).  To save you the time, I’ve listed them here:

  • WLS Upgrade 10.3.4: 11060985
  • Discoverer Upgrade 11060983

In the end, just run the upgrader for Discoverer, and follow the bouncing ball answering the questions.  When I was completely done, two issues.  First, the webcache file permissions were hosed again ( Step 5 Number 2) and my passwords were no longer hard coded in the startup and shutdown scripts ( Step 5 Number 5)

What’s New

Beats me.  The patch says “The list of bugs fixed is not available. Consult the Read Me file.” You won’t find it in the readme.  It will point you to the Portal, Forms, Reports and Discoverer Release Documentation.  I click on release notes there, and select my platform, and it brings me to the Oracle Fusion Middleware Release Notes (11gR1 or 11.1.1) for Linux x86.  I gave up.  I’m going to go with Discoverer is awesome, that’s what’s new.

Edit 4/27/11

It turns out, I had to revert to because some of my reports broke in  This issue revolved around’s apparent inability to understand date parameters (in some workbooks) and I wasn’t about to re-write reports for the sake of this lousy upgrade.  All I can say is it was unexpected (I’d never had a problem with a minor skip release like that before) and I recommend you test all your workbooks before upgrading.

February 17, 2011

Resolved with Solution?

Filed under: Oracle, Support — kkempf @ 11:48 am

Show me the Money!

Lets talk about My Oracle Support

WARNING!  I’m about to complain about Oracle Support!  Don’t read further if you’re too close to the Kool-Aid bowl!  I don’t know about you, but we pay a fortune for Oracle support.  Enough to easily cover the cost of hiring an experienced second DBA.  It’s a very artificial and contrived reference point, I admit.  But often, I ask myself, if I had another DBA, would they provide more value than Oracle support?  I fear the answer, more often than not, is yes.  I know it’s not feasible to take yourself off support.  But it’s fun to think about…

Resolved with Solution?

Unless you’re very different than me, I challenge you to look at your recent SRs closed with “Resolved with Solution”.  How many of them were actually solved by Oracle?  I mean their statistics at Oracle must be through the roof, because they’re never wrong, or unable to resolve my problem.  And even when I fix it myself (whether I refuse to tell them the solution or not) they close it “Resolved with Solution”.  Is letting the customer figure it out a solution?  Must be.

  • 11i forms/web footprint grows unabated.  Resolved with solution: check some irrelevant kernel parameters (upon receiving this task I pretty much abandoned the SR)
  • Install 11.1 Agent on X86_64 without ssh.  Resolved with solution: in 11g due to someconstraintsthey have removed this method of installation.
  • ORA 3137 running generic EM11g.  No fault found: Despite the fact that I said this error occurs intermittently in RDBMS 11.2 running against EM 11g (their product), they need a reproducible test case.
  • How to use my own SSL certificate with EM 11g.  Resolved with solution: Unfortunately the current release only supports 3rd party certificates with SSO
  • Discoverer 11g login page unavailable.  Resolved with solution: Thank you for the great explanation Kevin. I am researching and “playing” in our test environment and will update the SR with the results. (No update received, I wrote my own startup scripts and asked the SR be closed)
  • java.lang.NullPointerException after clicking on description: Resolved with Solution: (Me) Thanks I just don’t have time to run a bunch of diagnostics and queries especially for known issues. You should just ask me what version of “X” I’m on. I found the solution myself

Surely a Survey would Fix this?

Why is it every time I get an invitation to do a survey after an SR, it’s one where I asked the world’s easiest or more salient question.   It’s never when I have a random ORA-0600 in my alert log.  It’s more like when I ask “how do I set a session timeout in Discoverer 11g?”.  I rarely respond to surveys, since I’ve largely decided the whole game is rigged.

Food for Thought

Take a second and consider, if you will, what the following DBA would be worth to your organization.  I’ll use he/she indiscriminately, as it doesn’t really matter:

When issues arise, he’s always asking irrelevant questions, and refuses to work problems until he’s satisfied with the answers.

She always insists on being on the latest version of every possible product, even if it came out a day ago.  She even pushes this to the applications the users see, not just the database, even though there is no business requirement to upgrade.

He refuses to work after hours unless you’re right there working the issue after hours with him.

She has random talent swings which vary from incompetent to well qualified, and no consistent expertise.

You get so frustrated with his slow or shoddy work, that you often decide to find a solution and do it yourself.  When you go to the status meeting about the project, he claims credit for having done the work.

If you ask her the same question twice, it’s likely you’ll get a different answer.  Communicating with her is a very difficult and trying process.

I challenge you to at least think about the value you’re getting from Oracle support, versus what you’re paying.  My curiosity was piqued when I came across a company called Rimini Street at OAUG last year.  They’re a 3rd party support provider for Oracle. Of course, they’re in all kinds of legal trouble with Oracle because Larry doesn’t let anyone steal from the table.

What if you could buy a reduced cost tier of Oracle support where you simply had access to the online documentation, and patches/updates, and pay sharply if you had to contact an analyst? I for one would definitely consider that.

Finally, I ran into a group called Oracle Advanced Customer Services at a regional conference a year ago.  For all I know they don’t exist anymore.  But as far as I could tell, they were like a “super support” structure.  The woman basically said “You know, for when support can’t give you the answer”.  What?  So if I don’t like the bad support I’m paying a fortune for, I can pay even more and get good support?  Or are they some flavor of Oracle Consulting?  I didn’t understand.  Maybe I’d seen one too many sales pitch.

February 15, 2011

Discoverer 11g: Soup to Nuts install of

Filed under: 11g, Discoverer — kkempf @ 11:43 am

Why Soup to Nuts?

I was looking at my statistics on my most visited pages, and the entries on Discoverer 11g installation and Enterprise Manager 11g installation are by far tops.  I was using my own entries as a guide to re-install Discoverer 11g the other day (it needed to be relocated to a new server) and I realized it was a bit confusing, with important information in several different posts.

What’s New?

Details regarding how to change the default Discoverer 11g client timeout (60 mins), how to install Discoverer on a 64-bit platform, and applying the January 2011 CPU.  Other than that, I’m mostly just smoothing wrinkles in the walk through, and putting it all in one place.  I’m not going to detail as it’s just a subset of and you can find the information elsewhere on the blog.

Environment Variables/Paths I reference

MIDDLEWARE_HOME=Middleware Install Location (xxx/xxx/xxx/Middleware)





Step 1: Weblogic

The first step is to determine, download and install the appropriate version of WebLogic server.  According to my Oracle technology salesperson (as opposed to the 11i Apps salesperson), this is free as we are paying for Discoverer.  I encourage you to check your license terms before you assume you’re entitled to this product, as much has changed since the “old days” of 4i.
You can find the downloads at  Note, my experience is only installing on Linux, but I’ve done both 32-bit and 64-bit installs so I can go through those.  This is by far the easiest part of the whole install.

Ensure that your WebLogic version lines up with your planned Discoverer install

Oracle Fusion Middleware is certified (only) with WebLogic Server 10.3.3 and Apps 11i

If you’re running the 32-bit installer, you just execute the downloaded .bin file; it contains java already and is an executable binary.  If you’re installing the 64-bit version, you’re getting a jar file which isn’t platform specific.  You need to start the installer by using syntax: java -jar wls1033_generic.jar or the like.

There’s really nothing to the install.  I went with defaults, although I’m not sure I needed Oracle Coherence, I figured it wasn’t worth the effort of researching it much further.  There’s only one last item of note here: uncheck and don’t run the quickstart when it’s done laying down the files.

Finally, if you screw up, don’t like the install location, or change your mind, there’s an oui uninstaller buried in there somewhere.  That still leaves a mess on the filesystem (and inventory, in my experience) so the other option is to rm -rf the whole install directory and clean up the oracle inventory ($INVENTORY_HOME/oraInventory/ContentsXML/inventory.xml) to remove references to the install you just did.

That’s it.  Check the box, you’re done with the WLS install for now.  Once all the pieces are installed, there’s a few critical files you may wish to manually edit (regarding starting services) but I’ll go over that later.

Step 2: Discoverer Fusion Installation

Once again, you need to pull the install for the appropriate version of Discoverer.  You can find the downloads at towards the bottom of the page.

Since I’m only concerned with, pull Portal, Forms, Reports and Discoverer ( and Portal, Forms, Reports and Discoverer (  The installer is only an upgrade from, so you have to run both of them.  Note that there’s also the Windows-only Discoverer Desktop and Discoverer Administrator ( which you will probably need, as well as the Repository Creation Utility (RCU)

Start with the Fusion Middleware (FMW) install.  Simply run the installer and follow the bouncing ball, but ensure you select software only install.  There’s no sense trying to configure, you’re not going to use it.   When it’s done, run the installer.

I don’t recall which installer ( or asks for what, but here’s what’s of note during the questions it asks.

  • It will want to know where your Fusion Middleware home is.  Point it to the WLS 10.3.3 install home.
  • Remember the password you provide for the weblogic user, it’s important
  • Stick with the defaults, unless you’re an experienced Weblogic user, regarding Oracle Home name (as_1), Oracle Instance name (asinst_1), and the domain name (ClassicDomain).
  • Under components to install, uncheck everything except Oracle Discoverer and Web Cache. Note there’s a checkbox for Clustered there also.  I unchecked it, have no idea what checking it leads to but it wasn’t applicable to me.

At this point, both installers should be done, but nothing should be configured.

Step 3: Repository Creation Utility (RCU)

The RCU just creates a few tablespaces, new users, and objects in the database.  I don’t know for certain if it uses your existing repository, but it seems that it does.  To launch the RCU, unzip the file, and run the executable called rcu which is in the bin directory.

Of note to the questions it asks:

  • The only thing you need checked on the entire Select Components screen is Discoverer.  (Under Portal and BI-> Discoverer)
  • Remember your new schema user and passwords, they will be required during the configuration of Discoverer and Weblogic.  I recommend SID_DISCOVERER (PROD_DISCOVERER)

Step 4: Configuration

Now you will need to run $MIDDLEWARE_HOME/as_1/bin/ to complete the Discoverer setup.

Most of this should be easy to answer by now, but of note are a few things:

  • Under Specify Schema, use the SID_DISCOVERER (PROD_DISCOVERER) schema you set up with the RCU
  • Unless you’re using OID, uncheck Use Application Level Identity Store.  I missed this once (I don’t use OID) and troubleshooting the startup was impossible.  I had to reinstall the whole stack.
  • I recommend using Auto Port configuration if possible
  • At the end of the GUI, it displays all ports and URLS.  I recommend you save this file to the OS as it’s a handy reference for how to get to the WLS server and Discoverer login page.
  • Follow note 1073963 which details some profile option values which you need to change in 11i to accommodate Discoverer 11g.

Step 5: Fixes

1.  Fix the webcache bug (doc 1113163.1) “Failed to assign port 8090: Address family not supported by protocol”if you hit it

  • Edit the file $MIDDLEWARE_HOME/asinst_1/config/WebCache/webcache1/webcache.xml
  • Add <IPV6 ENABLED=”NO”/> tag below the </MULTIPORT> tag

2.  I also hit a common webcache problem this time, where the webcache wouldn’t start because webcached permissions hadn’t been set right:

  • Navigate to $MIDDLEWARE_HOME/as_1/webcache/bin
  • As root, run setroot (username) where username is the user who installed the webcache
  • Permissions should look like this when you’re done: -rwsr-x— 1 root    dba 6796777 Dec  3 11:32 webcached

3.  Copy/link the approriate (8.0.6 home for 11i installs) tnsnames. ora file to 2 locations with a command like this (substitute cp for ln if you want to simply copy the file, obviously)

  • cd $MIDDLEWARE_HOME/asinst_1/config
  • ln -s $8.0.6_HOME/network/admin/SID_host/tnsnames.ora
  • cd $MIDDLEWARE_HOME/as_1/network/admin4
  • ln -s $8.0.6_HOME/network/admin/SID_host/tnsnames.ora

4.  Change the Discoverer default timeout

  • Edit the attribution Timeout in ORACLE_INSTANCE/config/PreferenceServer/Discoverer_instance-name/pref.txt
  • Run ORACLE_INSTANCE/Discoverer/Discoverer_instance-name/util/

5. Hard code your password in the startup/shutdown scripts

  • Edit $DOMAIN_HOME/bin/ and
  • add entries for:
    • WLS_USER=”weblogic”
    • export WLS_USER
    • WLS_PW=”password”
    • export WLS_PW

Step 6: Starting/Stopping

These scripts works great for me.  You may have to tweak them a bit, but it’s tested and proven.  Note the sleeps are in there because it takes time to crank some of the services, and unless they’re running the next step fails.  60 seconds is generous.


export MIDDLEWARE_HOME=/u02/appdevl/devdisco/Middleware

export DOMAIN_HOME=$MIDDLEWARE_HOME/user_projects/domains/ClassicDomain

export WL_HOME=$MIDDLEWARE_HOME/wlserver_10.3

export ORACLE_HOME=/u02/appdevl/devdisco/Middleware/as_1

xport ORACLE_INSTANCE=/u02/appdevl/devdisco/Middleware/asinst_1

rm -rf nohup.out

rm -rf /tmp/wls_start.log

rm -rf /tmp/start_nodemanager.log

rm -rf /tmp/start_mgdwls.log

echo “Ensure NO processes related to disco 11g are running or this will fail”

nohup $DOMAIN_HOME/bin/ > /tmp/wls_start.log &

nohup $WL_HOME/server/bin/ > /tmp/start_nodemanager.log &

echo “sleeping”

sleep 60

nohup $DOMAIN_HOME/bin/ WLS_DISCO t3://(server.domain):7001 > /tmp/start_mgdwls.log &

echo “sleeping”

sleep 60

$ORACLE_INSTANCE/bin/opmnctl startall

$ORACLE_INSTANCE/bin/opmnctl status

echo “If Discoverer doesn’t start properly, login to http://(server.domain):7001/console”

echo “From the home page, click servers (Under Environment), then the control tab.  Check WLS_DISCO then click the start button below the checkbox”

# Shutdown


export MIDDLEWARE_HOME=/u02/appdevl/devdisco/Middleware

export DOMAIN_HOME=$MIDDLEWARE_HOME/user_projects/domains/ClassicDomain

export WL_HOME=$MIDDLEWARE_HOME/wlserver_10.3

export ORACLE_HOME=/u02/appdevl/devdisco/Middleware/as_1

export ORACLE_INSTANCE=/u02/appdevl/devdisco/Middleware/asinst_1

$ORACLE_INSTANCE/bin/opmnctl stopall

$ORACLE_HOME/opmn/bin/opmnctl status

# stop the discoverer server

$DOMAIN_HOME/bin/ WLS_DISCO t3://(server.domain):7001 weblogic password

# stop the weblogic admin server



killall nmz

echo “There’s probably still 1 disco process running – ps -ef|grep jrockit and kill it (java)”echo “It’s probably pid:”ps -ef|grep jrockit|cut -c10-15

Step 7 CPU (optional)

I had no problem applying CPU0111 (patch 10233533), though my ORACLE_HOME required a more recent Opatch (patch 6880880) version than the installer provided.

Step 8 Navigation Reference (Default)

Discoverer Plus: http://hostname:8090/discoverer/plusDiscoverer

Viewer: http://hostname:8090/discoverer/viewer

Weblogic Server Administration: http://hostname:7001/console

Weblogic Marketing Propoganda: http://hostname:7001

Blog at