BI Blogs

Doing some work at the moment that requires me to provide rough sizing for a new SAS hardware environment.

One of the things you may not know is that there is a group based in Carey that are responsible for doing sizing estimations for the various SAS teams around the world. They have a pretty hefty sizing questionnaire but once you have filled it out they are able to give you recommendations of what sort of hardware you will need. You will need to go through your SAS Account manager to access this service.

One of the great things this team has decided to do is to publish each sizing they do for a customer as an anonymous document. So the SAS teams are able to browse through say sizings for 4-cpu windows servers supporting SAS Enterprise Business Intelligence Server and get an idea of how many users they typically support. Great if you don’t really have a good idea of what user profile you will end up with (but obviously a little bit dangerous as well).

Again this is only accessible by SAS staff, so ask your friendly SAS Presales (Sales Consultant/Engineer depending on which part of the world you reside) to see if they can help.

There are some publicly available/publish benchmark documents for various configurations available, so I have added an article to the sasInct.com site where I can list them as I find them.

SAS Server Sizing reference Papers

Operational BI is becoming quite the buzz today. There are countless articles, tips, vendors with operational BI offerings, case studies, etc., available today. Yet, I still get asked the very basic question of how do you get started. What are the first steps? Well, here are my thoughts on how to get started…

If your ETL process fails, for whatever reason, you have two options: to restart the process from the beginning, or to resume from the last successfully completed task.

Restarting from beginning

It is reasonable easy to restart the process in OWB, you would have to wait until your top level process flow has completed and then if your target tables have loading type of TRUNCATE/INSERT, or you use the WB_TRUNCATE public transformation to achieve the same effect, then you can restart. This restart is however from the beginning of the process flow and so may involve you re-running hours of ETL that has already ran correctly. If the duration of your ETL process is under an hour this could be acceptable, however if your ETL process is more like 6 hours, and only has an 8 hour window this is more of a problem.

Resuming the process

OWB has no built in functionality to resume process flows. One approach we have used is to use a transformation to test whether a mapping has already executed for a run before executing it. This is done in a process flow by using a transformation to test whether the mapping should be executed. Hence each mapping needs the following activities:

• A Transformation to test the mapping
• The mapping itself
• An OR activity to collect the results

Each mapping looks like the one below.

The result of the transformation should be tested in the transition and the required execution path chosen. In the example above, the transformation returns 0 (zero) if the mapping is to be run, or any positive integer if it is not. Any other value is treated as an error and will cause the process flow to fail. This can be implemented by using a complex condition in the transition.

The OR activity is then required to allow execution to continue, whichever execution path reaches it first will cause execution to continue, as the transformation uses mutually exclusive conditions this will mean in reality that only one execution path can reach it.

Controlling the execution with an ID

In the example above the transformation can only determine if the the mapping has already run by supplementing the logging that is provided by OWB. The transformation must be aware of the logical execution that is being run. In a real world example the logical run would relate to one data load into the Data Warehouse. The execution ID identifies a logical run and is set at the beginning of a run and can be passed between each process flow, sub process flow and/or mapping as a parameter.

In the example above the each mapping has been configured to log to a table its start time, the execution ID and, if the mapping completes successfully, the end time. This can be achieved by pre and post mapping process. An example of this is shown below.

Using this technique transformation activity can always establish whether a mapping has been executed successfully for this particular run, this is helped by setting the post mapping process to only run ‘ON SUCCESS’ which can be configured from within a mapping, hence you could ensure the end time would only be populated if the mapping completed successfully.

Number of errors

If you are executing your mapping in set-based mode the mapping activity parameter MAX_NO_OF_ERRORS is set to 0 (zero). If you leave it with the default of 50, it seems that if the mapping errors then this only counts as one error, so the post mapping process executes as one is less than 50, and for that reason it thinks the ON SUCCESS condition has been met. Set the parameter as follows:

Forcing mappings to execute

There may be certain instances where you want a mapping to execute regardless, or you want to re-execute the mappings during the course of a run. In the first instance mappings can you added to process flows without the transformation and OR activity and then they will just execute as per normal. In the second instance the you can add another parameter to the transformation to force the following mapping to execute, regardless if it already has or not.

Automation using OMB Plus

Finally it is worth looking at the overhead this adds to the development process, you now have to add three activities to a process flow and configure transitions, whereas before you simply had to add a single mapping. One way round this we have used is to use OMB Plus scripts to either generate the process flows for a list of mappings, or to add all three activities to a process flow.

A

Oracle BIWA Summit - December
2nd & 3rd
Please pass this
information to customers and partners

LEARN MORE
BIWA
Summit 2008 Website
Request
for Presentation Website
OracleBIWA.org
Watch for Summit Registration details coming soon.

About BIWA: 
BIWA
is the Oracle Business Intelligence, Warehousing, and Analytics Special
Interest Group, part of the Independent Oracle User Group (IOUG). BIWA
is a worldwide association of persons seeking the successful deployment
of Oracle Database-centric business intelligence, data warehousing, and
analytical products, features and Options.

What does it really mean to be a global company? From a communications perspective, it certainly demands that we all speak the same language, whether its in English, Chinese, French, Danish or Bahasa Malayu. For that matter, whether it’s US English, UK English, Aussie, Kiwi or South African. Challenge? You bet.

It was that challenge that carried me recently to Malaysia, my first foray beyond the western hemisphere. As the guests of SAS Malaysia , we – my colleague Bart Queen of C3 Communications and I – conducted two media training workshops with a group representing several SAS operations in southeast Asia.

Ah, media training, you say! Do you mean, spin and subterfuge? Not at all.

Have you ever listened to an interview with a brilliant, knowledgeable, experienced expert who – despite a barrelful of content that was insightful, intriguing and all those in-words – had you drooling in your blackberry before two minutes was up? Or who distracted you with tics, ums and repetitive air-chopping or spider pushups (picture a spider doing mirror image dancing) so you were riveted by his fingers instead of his facts?

Even the most polished presenter (mind you, I said “presenter,” not “communicator” for a reason) gets zero marks when the shine glosses over garble. I’m talking about knowing exactly how to say what you want to say, and practicing it. Over and over.

And surprise – it doesn’t vary from country to country. The same skills and preparation work, no matter where or to whom we speak. This kind of capacity is life-changing – the ability to communicate effectively and powerfully can open up new worlds in both personal and professional settings.

For SAS, it means that what we started eight years ago with our first brand campaign – The Power to Know(tm) – has changed who we are as a company because it changes how we say who we are, and thus how we are perceived. In every language.

And there’s more work to do to hone the message (remember, message and not spin!). We must communicate effectively who we are, what we do, and how we benefit customers – not just those who use our software, but those who enjoy the products and services our customers provide.

The best news is that it works. That it is universal. And that working across cultures to develop a universally accepted and understood voice is as rewarding in the process as in the product.

More on the message later!

The Boston Chapter of TDWI had its quarterly meeting yesterday with a theme on Master Data Management (MDM). We had a keynote presentation titled “Master Data Management (MDM): Data Salvation or Your Next Data Silo?” We also had a terrific panel discussion on MDM with representatives from IBM (IBM), Business Objects/SAP (SAP), Oracle (ORCL) and Kalido.  The panel consisted of an interesting mix including sales management, a CTO and two representing field organizations.

In keeping with the general nature of TDWI events we limited the hype and sales pitches. I moderated and have to compliment the panelists on keeping the focus on providing the attendees with the successes of MDM experiences along with the cautions and dangers that one may encounter.

A few key points:

MDM needs to be a business-driven endeavor. The business must feel the need and participate for success. IT cannot build MDM without business.

The MDM effort needs to be focused on delivering real business value and solve specific business problems (top-line improvement or bottom-line cost reductions.)  Justifying a MDM program with a declaration of achieving a "single version of the truth" or "360 degree view of the customer" just won’t do anymore. This “Field of Dreams” scenario is too nebulous.

As one panelist said “MDM is for life, not just for Christmas.” MDM is not a onetime project but an ongoing program. And the successful programs just keep getting more to do.

The best candidates for MDM are those companies that have felt the pain and are already educated about how difficult it is to get master data defined and blessed by the business. In analytical MDM efforts, the best candidates are those that have been doing enterprise data warehousing for awhile and have already started some form of data governance even if it is only in its infancy.

MDM is about people, politics and processes rather than technology. Actually, I said that not the panelists, but they agreed with me.

Although it was a little tough to get them to admit it, they all agreed that MDM were not small projects in terms of time, people or budgets. They kept telling us that you could start on a small scale but it probably did involve millions of dollars. The early adopters they mentioned though generally spent a good deal on their initial efforts.

It was a very interesting and informative afternoon.

I just received the final evaluation from the DIG conference and the feedback was great. Across the board we had positive scores and commentary from attendees. Here are a few snippets that I wanted to share:

What valuable insights are you walking away with?