BI Blogs

Eddie Awad asks: How come no one has mentioned “testing”?! A very important part of developing an application (any application) is to test it again and again until the tests produce the desired results.
Of course Eddie is right, but sadly, like good design, adequate testing often takes a backseat role to getting the job done. Reduced rigour in testing is, however, not always a deliberate ploy by

Yesterday just prior to our AMIS Query on ADF Faces I was talking to Toon Koppelaars about quite something else: the end_date column in tables that contain contiguous records with BEGIN_DATE and END_DATE where there is no overlap and there are no gaps. Every BEGIN_DATE is the day after the …

Multiforce.com is more a development environment than an application.

Today is the first day that I have had over 330 (and still over 2 hours to go) hits/day on this blog - no doubt thanks to a link from Tom Kyte’s Blog… and my post was a link to his Blog! - The strange thing here is that his post has over 20 comments and I got, er, one (well at the time of posting) not that I am jealous.

Spent the day in the ICC in Birmingham at an Oracle conference - no, I did not go a month too early for UKOUG. Today was an event for UK Oracle partners, and as I am looking after my company’s Oracle partnership arrangements for the time being, I thought I had better go along. It was a good event - no heavy technology sessions and the chance to meet a lot of helpful people. Despite being in Birmingham I did not see Tim Hall.

For anyone who’s not at Open World (must be a few people, even if there are
35,000 here) I thought I’d put a list of all the BI & Data Warehousing
presentations that you can download. When you get prompted for user credentials,
use username/password cboracle/oraclec6.

• Scalable Data Warehousing on Dell and EMC 
  
  
• Oracle Real Application Clusters for Data Warehousing: Unleash the Power
  
  
• Data Warehousing Architectures
  
  
• Transforming Your SAP Data into Oracle Information, Using Oracle
  Business Intelligence Warehouse Builder 10g Release 2
  
  
  
  
• Expressway to Data Movement
  
  
  
  
• Building a High-Performance, Adaptive Business Intelligence
  Infrastructure with HP and Oracle
  
  
• Agile Methods and Data Warehousing
  
  
  
  
• Oracle Warehouse Builder 10g and OLAP: What’s New?
  
  
  
  
• An Overview of Oracle Business Intelligence Warehouse Builder 10g
  Release 2 ("Paris")
  
  
  
  
• Oracle Data Mining and Predictive Analytics: Overview, 10g Release 2 New
  Features and Demo
  
  
• Enabling Real-Time Data Warehousing with Oracle Database 10g Release 2
  and Oracle Business Intelligence Warehouse Builder 10g Release 2
  
  
• Bringing It All Together: Building an Interactive Analytic Dashboard
  with Oracle Discoverer and Oracle Portal
  
  
  
  
• Rethinking OLAP: Turbocharge Your Star Schema with Analytic Workspaces
  
  
  
  
• Enterprise Reporting Basics: Introduction to Oracle Reports Developer
  and Its New Features in Oracle 10g Release 2
  
  
  
  
• Data Quality: Transforming Data into Quality Information
  
  
  
  
• Adding Data Mining to Extend Your OLAP BI Solution
  
  
  
  
  
  
• Oracle’s Advanced Analytics in the Database: Statistics, Data Mining,
  Text Mining, SQL Analytics, and More
  
  
• Oracle Business Intelligence Discoverer 10g Best Practices
  
  
  
  
  
  
  
  
• Keynote: Oracle Business Intelligence and Data Warehousing Overview
  
  
  
  
• Enabling Spatial Intelligence in Your Business Intelligence Solution
  with Oracle Warehouse Builder, Oracle Discoverer, and Oracle MapViewer
  
  
  
  

EXECUTIVE SUMMARY Too often an enterprise’s information is an unmanaged asset businesses can’t find consistent information and IT struggles under the resulting weight of project delays and ballooning storage costs. Information architecture (IA) helps IT address business needs by providing a framework to map and describe an enterprise’s information assets and their relationship to processes and systems. But IA programs fail if not approached the right way. To be successful build iteratively, focus on pain points, and execute top-down.

Must stop composing this stuff in Word and pasting it in! - Last night’s posting fell over in Feedburner becuase of an ‘odd’ character in the text!

Trying to make myself useful by answering some of the questions on the OTN SQL and PL/SQL Forum, I came across this question: Group Above Report problem…not able to display absent rows in date range. The question could be translated to: I am trying to aggregate records by date and …

An anonymous poster to Tom Kyte’s Blog shared a tale of woe about data warehouse performance. Well, I felt for that poster – how can a developer put their hand on their heart (or wallet, if they are an external consultant) and say in effect “design does not matter, the database will be able to handle it” Of course design matters; and getting it right is important for the success of any

With slightly more than a month to go until launch, it looks as if SQL Server 2005 is finally a done deal?and that?s a big deal for BI pros.

PerkinElmer tapped a BI solution from Business Objects SA to consolidate its disparate BI holdings

In the wake of Oracle?s stunning Siebel gambit, is there a future for a Teradata-friendly Siebel Analytics suite? Teradata officials say so.

Oracle to
Support IBM WebSphere with Project Fusion Apps: "A week ago, as IBMers in
the iSeries Division were preparing, like myself, to head down to Orlando, Fla.,
for the COMMON iSeries user group meeting, other IBMers from Software Group were
heading out to Silicon Valley to do a legal dance in advance of the OracleWorld
user group meeting for Oracle customers, which was held last week as well. What
came out of those meetings was an agreement by Oracle to support IBM’s WebSphere
application server on the forthcoming "Project Fusion" application suite, which
will be a whole new software suite due in 2008 or so that will be upgradeable
from current Oracle, JDE, and PeopleSoft suites (and probably Siebel and Retek
as well).

Certainly the most momentous news that came out of Open World last week was
the rapprochement between IBM and Oracle and the subsequent deal to certify
e-Business Suite and the forthcoming Fusion applications on IBM’s middleware
technology. This could well lead to a situation where Oracle support Fusion on a
whole range of competitors middleware and databases, and you could even imagine
a situation where WebSphere becomes the preferred application server over
Oracle’s Application Server 10g. All very interesting compared to a couple of
years ago, when Oracle’s aquisition of PeopleSoft was seen as a way of driving
Oracle database and app server sales; I imagine Oracle’s desire to lead the apps
market - where it presumably sees the bulk of it’s revenues in the future -
means that it now needs to follow it’s customers’ requirements to support all
the major databases and J2EE servers on the market, not just Oracle’s, which is
why there was all the talk last week about standards-compliance and "hot
swappable" middleware components.

Earlier this year I was asked to take a look at
how we
might use 10046 trace files to help carry out OWB performance tuning, the
idea being that by automatically generating trace files, we could identify why
certain mappings and

process flows were taking too long to run. A reader was
interested in the technique at the time and tried it out on an OWB implementation he was
working on. The other day he sent me some feedback on how the approach went. I
thought it’d be interesting as a bit of "real-world experience" on carrying out
OWB performance tuning.

"Bit of follow up that I’ve been meaning to write re the statistics
gathering/event tracing in mappings that Mark wrote up on his blog a few
months back.

Further consideration has shown that this was not the correct approach for
this project, reasons as follows. We undertook a strategy to run a daily
smoke test (http://www.stevemcconnell.com/bp04.htm)
off the full process based around different test packs. This enabled us to
test volumes, specific cases etc etc. With daily event 10046 stats turned on
(the full process executed 300 sql scripts and ~100 mappings) we found the
following:

(1) There was too much information to review.

(2) We kept running out of disk space because of the Gb’s of log files -
thus actually stopping the jobs running as the database errored.

The approach now taken is the take timings against volumes for each of the
jobs run, and the extrapolate the estimated run time for full volume. We
then have a daily/weekly meeting to review the top 10 cases and on the basis
of this focus any performance and tuning activity as required. I believe
this to be a more practical approach, and also follows to a degree the
approach describe by C Millsap et al in the optimising Oracle book, i.e.
focus your tuning efforts to provide maximum benefit from the resource
(developer time) you assign to it. Depending of the mapping to be tuned we
will then determine if event 10046 tuning is required, it may be a case of
logic rewrite or peer review may solve the issue.

I do not mean to devalue the work we did earlier in the year, but like
everything theory and reality are often different, and sometimes the old
fashioned look at the query/explain plan and realise no indexes are being
used is enough.

The greatest asset has been the daily smoke tests this has give us a
historic record of timings, if we do get unexpected variances in runtimes we
can use these figures to determine if its related to database config and
utilisation, or whether it relates to code releases (all releases are logged
and versioned).

Mark - may be worth updating the blog - I can write something if you like,
or just quote the above if you think it’s appropriate; I think it would be
useful to share though."

Thanks for the feedback. I think what he is saying here is that you shouldn’t really by
default trace all mappings, as the hit on performance and on disk usage is too
great. If as he says though, you concentrate tracing on just those mappings
that are affecting "response time" - as he says, determined through
the smoke test approach he used - then it can still be a useful approach,
although you shouldn’t forget the simple approach of just generating an
intermediate mapping and then running an explain plan on it.

Telecom Business Performance Management Metrics: “Over time, these metrics have changed from subscriber additions to average revenue per user (ARPU) to churn. A recent Informa Telecoms & Media Study shows how most of the ‘old’ standbys are still valid performance metrics. These standbys include:

Subscriber data
Pre and post paid ARPU
Pre and post paid minutes of use (MOU)
Churn