Five Minute Data Integration Podcasts
Source: Clickstream [link]
Podcasts we did on defining data integration technologies just went up at SearchDataManagement today. Each describes a particular technology, what it’s used for and when its appropriate. We did four of them on ETL, EII, EDR and EAI, with plans to add several more on topics like master data management and data profiling. It’s always strange to hear yourself as other people hear you.
Upgrading to 10g - OK, I’m a bit behind
Source: Pete-s random notes [link]
At long last, my final data warehouse customer on Oracle 9 is moving on to 10g. It has been a somewhat protracted move as for reasons of vendor support certification we had to adopt a specific order of refreshing their whole technology stack. That’s not to say we have no Oracle 8i customers out there in […]
PIP *.*;*/RM
Source: Pete-s random notes [link]
Doug Burns’ feelings about the quirky nature of OpenVMS reminded me of the fun I used to have with files on PDP-11 and DEC System 10s. PIP was a great program, and no Marko, PIP is not on my resume (and nor is DATATRIEVE, my first database, for that matter)
And to prove that I […]
Necessary Shifts in the Industry
Source: Blog: Dan E. Linstedt [link]
For too long the industry has preached: load quality data into your warehouse, cleanse the data, manipulate it, and then load it to the warehouse. The mantra has been “never release bad data to the end users.” There are hundreds of articles written about this, and probably quite a few groups carrying this mantra forward. But I have to ask, where did this mantra come from? How on earth did it get “written in stone?” What really, is the true value here? When we implement this kind of paradigm do we RISK getting the warehouse “wrong?” Are we integrating away problems which are causing the business to hemorrhage money?
TodoBI.com Widget Reader
Source: Todo BI: Business Intelligence, Data Warehouse, CRM y mucho mas... [link]
Si eres un apasionado del Businesss Intelligence, que incluso tienes tu propio blog o portal sobre Business Intelligence o Data Warehouse, te mostramos una herramienta que seguro te va a encantar. Se trata de un Widget que permite leer las noticias de tus Blogs Favoritos. Nosotros te lo hemos configurado para que estes al tanto de las novedades de TodoBI.com, pero podrás añadir todas aquellas fuentes que quieras tener siempre en primera plana.
Por cierto, este widget y otros muchos espectaculares los podréis obtener en: Spring Widgets
Learning more about BI Publisher - OBE
Source: Oracle Business Intelligence Blog [link]
If you want to quickly learn about or get a feel for how BI Publisher works, and this may be especially the case for those who come more from the Discoverer side of things, I would suggest you spend a few minutes going over the excellent OBEs (Oracle By Example) series available on BI Publisher on OTN.
The Oracle by Example (OBE) series provides hands-on, step-by-step instructions on how to
SCOPE and calculated members
Source: Chris Webb's BI Blog [link]
This post on the MSDN forum:
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2064643&SiteID=1
…reminded me of something interesting I found out a few months ago. It seems that whereas you can’t mix regular and calculated measures in a set used in the SCOPE statement, you can rewrite the assignment to avoid using SCOPE and do a direct assignment instead. So, for example, if you were trying to assign to a regular measure and a calculated measure using a SCOPE statement like this:
SCOPE({[Measures].[RegularMeasure],[Measures].[CalculatedMeasure]});
this=1;
END SCOPE;
You would get the following error:
A set has been encountered that cannot contain calculated members.
MdxScript(Cube1) (line, col) A set has been encountered that cannot contain calculated members.
The END SCOPE statement does not match the opening SCOPE statement.
MdxScript(Cube1) (line, col) The END SCOPE statement does not match the opening SCOPE statement.
You could rewrite the assignment as follows using two SCOPEs:
SCOPE([Measures].[RegularMeasure]);
this=1;
END SCOPE;
SCOPE([Measures].[CalculatedMeasure]);
this=1;
END SCOPE;
…but this is clearly a pain, as you’re duplicating your assignment logic. What you can do instead is this:
({[Measures].[RegularMeasure],[Measures].[CalculatedMeasure]})=1;
Not as easy to read as using SCOPE, I know, especially if you’re doing something complex, but at least it works! Now I wonder why SCOPE has this problem? Probably something worth opening an issue on Connect about…
BI Publisher Discoverer Integration - 4 - getting the software
Source: Oracle Business Intelligence Blog [link]
After the grand success of my first three posts on BI Publisher and Discoverer integration (BI Publisher Discoverer integration, BI Publisher Discoverer integration 2, and BI Publisher Discoverer Integration - 3) I considered retiring and moving to a life of book tours and guest lectures. Then I thought of something better - why not a fourth post. Well, why not? And mind you the thought was
Unstructured Information in BI - Implementation Practicalities with Tacit Data
Source: Cyril on Business Intelligence [link]
Designing an unstructured information based BI system must take account of the explicit and tacit distinction. There is consensus for this, in blog-speak that means “me and my mate in the next office agree”! Feedback on my last two posts does, however, unanimously support this contention. The issue remains, however, so what do we do about it? Here’s what I propose.
Most businesses have a reasonably adequate process for collecting explicit unstructured information, the documents, news, emails, reports, etc. And if your’s doesn’t, the corporate portal experience awaits your attention. For heavy hitters the UIMA approach with its multi vendor retinue is available, and willing, for a substantial sum.
I have opined in the last posts here and here that explicit unstructured information is not where BI relevance is at. It can be a start, but the real value lies in the qualification that the executive and professional mind-space can give to seeds of BI, both explicit and tacit. The tacit realm is the goldmine; it is where the current, relevant, actionable, validated business intelligence lies.
How, then Dear Reader, do you capitalize on your tacit resources?
It’s a 9 step process, as I see it
- Encourage contributions from everyone, everywhere, based on credible rumor, opinion, assessment, etc.
- Scour the corporate world for knowledge building seeds, explicit and tacit - web crawlers, internal and external portals, news feeds, etc.
- Selectively disseminate raw data seeds to subject specialists - formally appointed for preference
- Encourage comments on those seeds by the specialists - acts, sources, cross-references, importance, time criticality - with discussion threads escalating in importance as appropriate
- Selectively disseminate comments - dynamic audience creation, so that more people, and more senior executives, are aware of more important issues
- Encourage issue identification by executives and professionals - implications, assessments, importance value adjustments, criticality adjustments
- Selectively disseminate the discussion - dynamic audience modification as business significance becomes clearer, possibly creating closed group discussions if the issue becomes strategic
- Propagate decisions made to the appropriate staff
- Store knowledge created - with time stamp, sunset clause if appropriate, to help avoid multiple solutions to the same problem
Obviously this must be an explicit process, where the tacit input is first encouraged, then amplified, assessed, amplified again until either the issue dies, is resolved, or mutates into another issue. But make no mistake; it’s the tacit input that drives the successful implementation.
Essentially we are making explicit that which was tacit; but on a selective basis, right time, right people, right place.
There is downside, however. Creating a workable tacit unstructured information BI system with the above features is non-trivial. I have done it many times, and it was never easy.
Caveats and Dependencies
Cultural Crevasses
Culture of collaboration is the all important enabler. If the people related barriers to sharing the knowledge creation process are not addressed, the venture will fail. No question about it. I have made an earlier post on the cultural issues and how they can be managed, but, briefly, the most critical barriers are, in my experience:
- There’s no reward mechanism for contributing intelligence, and it’s a lot of work for no personal benefit
- You don’t know who to tell, and it’s a lot of effort to find out
- You don’t know if this BI snippet you have come across is accurate, you don’t want to bother someone else unnecessarily and someone else must know it anyway
- There’s no important person around to hear what you have to say; so keep this intelligence to yourself until there is the right audience – the more valuable it is, the longer you’ll wait.
- Tall poppies lose their heads, so keep your head down, and messengers get shot
- You don’t want to embarrass your boss, or peer group, so keep it quiet
Source Validation
The source of intelligence is most people’s key to determining apparent accuracy of any tacit input. If you get a stock market tip, you will always check where it came from before acting. It’s the same for a rumor on a competitor’s product recall.
Audience Creation
Dissemination is completely dependent on adequate categorization. If a document, email, news item, etc. is not classified it cannot be circulated to the right audience. And everyone in the business must use the same terms for categorization, or they will miss relevant documents.
Crucial Taxonomies
This implies a standard comprehensive corporate vocabulary or taxonomy. Setting this up is not trivial either.
Automatic Categorization is Oversold
It’s not sufficient to classify documents by internal content references. The real, useful keywords for document that is relevant to BI may not even appear in the text. In spite of the tremendous advances in text analysis, the personal categorization by a subject expert still wins the classification stakes, in my opinion. By all means use the automated technique to get the item to a subject expert, but he/she will always be the best determinant of cross-references, importance and time criticality.
Finally
I believe that an important principle BI analysts need to fully understand is “the strategic and most valuable information in your business is in the minds of the managers and professionals” as first enunciated by Henry Minzberg. Turning this tacit unstructured information into explicit useful stuff is universally a high priority task. Done well, it creates the difference between learning and non-learning enterprises.
The end of Business Intelligence takeovers?
Source: Data Doghouse - performance management, business intelligence, and data warehousing [link]
Has the
recent stock market downturn and credit crunch (sub-prime fiasco) stopped or at
least postponed the mergers and acquisitions (M&A) in the BI, CPM and DW
markets? Let’s break down M&A activity into two categories: private equity
or LBOs (leveraged buyouts) and strategic acquisitions.
equity buyouts started looking like NASDAQ 5000 redux. The deals kept getting bigger and the
business logic for the deals kept getting thinner. As long as cheap debt (VERY
cheap debt) was available, someone could make money out of taking a very sound
business that was generating a lot of cash, i.e. successful software firm, and
piling on the debt.
raised as a
England fiscally conservative person (cheap Yankee), so even though I understood
leverage I kind of cringed when I saw it applied so liberally.
seen recently, and in the NASDAQ crash several years ago, leverage is great
when it works and a disaster when it doesn’t. Something about living by the
sword and dying by the sword comes to mind.
equity deals are definitely slowed down, if not forestalled for a while. And
when they do come back they should be more reasonable for the foreseeable
future. (Until the next round of excess.)
news/bad news for the BI/CPM/DW industry is that most of the potential deals
are not the mega-deals that helped crash this party, but are much more
manageable market-cap size to complete.
addition, our industry’s software companies, in general, have little debt, great
cash flows and solid revenue and earnings growth. That means the math would
still work for one or more of our firms to be bought by private equity.
there are a few candidates where not only the financial math would work but
being private would empower them to attempt a transition that they might need.
A BI or CPM firm might be able to remake itself while private, where if it was
public quarterly scrutiny might tie its hands.
titans – Oracle, SAP, Microsoft, IBM and maybe someday HP – all have been buying
software firms to build out their capabilities. It may be necessary for the
neo-titans or wannabes to go private or off the public financial radar to
radically revamp their product and services line-up to compete.
transition to SaaS (software as a service), SOA (service-oriented architecture)
or maybe
OSS (open source software) might be necessary but would “destroy” profits
during the transition. Being private might be the only way that management
could enact those changes.
suggesting that this scenario will happen, but some software companies may
become irrelevant or be destined for the software graveyard (bought by a Titan
and turned into
equity may be slowed down but there is a lot of money to be made in BI/CPM/DW
software. If there is a will they will be a way.
Or maybe you have some software firms as private equity candidates in mind?
Feedback welcomed.
BI appliance Open Source (Ingres + Jasper)
Source: Todo BI: Business Intelligence, Data Warehouse, CRM y mucho mas... [link]
Ya esta aquí. El que se define como primer Appliance Business Intelligence Open Source. Respecto a como tradicionalmente se entiende un Appliance, aquí no se incluye el hardware, pero tiene interesantes aportaciones al mundo del BI Open Source.
Las principales características que tiene son:
- Toda la capacidad Business Intelligence la aporta Jasper.
- Tiene una única instalación y la configuración es mas sencilla.
- La Base de datos utilizada es Ingres Open Source.
- La Base de datos esta previamente configurada para este propósito.
- El mantenimiento esta validado para incluir futuros updates.
- Servicios de Gestión remota.
Descargar presentación de la solución.
Ver arquitectura:
Más Informacion:
Ingres Icebreaker BI Appliance solution sheet
JasperSoft and Ingres datasheet
Optwize and Ingres datasheet
Ingres BI Solution Appliance webinar
B-EYE Network webinar
DM Review webinar
Icebreaker Database
Product Availability Matrix
Ingres Support
Licensing
Events, Webcasts, & Podcasts
FAQs
Downloads
Ingres Icebreaker BI Appliance
Certified Products
Community Products
Community Developer Tools
Connectivity Tools & Drivers
Documentation
White paper on resolving query performance problems
Source: Chris Webb's BI Blog [link]
When is ELT the Right Choice?
Source: Data Doghouse - performance management, business intelligence, and data warehousing [link]
When I teach data integration concepts in my classes at various corporations or to my students at Northeastern University, one of the topics I address is the differences between engine-based and database-based data integration tools.
You may be more familiar with their other names:
• ETL (extract, transform and load) or engine-based
• ELT (extract, load and transform) or database-based tools
Inevitably, someone asks me which one is best. I have answer on two
levels. The first answer is that neither is the best under all
circumstances but depends entirely on the data integration scenario
that is being addressed. Evolving industry trends are causing
enterprises to often need both alternatives in order to best meet their
full range of data integration needs.
Read the complete post When is ELT the Right Choice? for the rest of my answer.
BI Publisher Discoverer Integration - 3
Source: Oracle Business Intelligence Blog [link]
So far I have blabbed blogged two posts (BI Publisher Discoverer integration and BI Publisher Discoverer integration 2) on the BI Publisher and Discoverer integration in the BI EE 10.1.3.3 release.
In the second post I took a Discoverer worksheet and showed a barebones BI Publisher report created using that worksheet as the data source. No layout, no template, but showed glimpses of what is
Consejos de Einstein
Source: Todo BI: Business Intelligence, Data Warehouse, CRM y mucho mas... [link]
Hoy contamos con un colaborador de lujo (Albert Einstein), que nos da unos consejos que no debemos despreciar:
Dado que se apróxima el final de las vacaciones para muchos, os ponemos una pequeña gota de humor.
* Podéis escribir vuestros propios mensajes de Einstein, desde este enlace:
http://www.hetemeel.com/einsteinform.php
BI as a Services (BIaaS) - Will Google Move In On This Opportunity?
Source: Blog: Mike Ferguson [link]
Most of you by now have probably found it difficult to avoid the hype around Software as a Service (SaaS). For many of us today this is already a reality in our business. You only have to look at the huge uptake of Saleforce.com by small and medium size businesses (SMBs) to realize that there is certainly a place for this in many companies. With respect to the BI market there is no doubt that there is also considerable growth in BI as a Service (BIaaS)and it would appear that many BI vendors are eagerly setting out the stall on the net to jump into this market of hosted BI Servces. Given that many BI products are already service enabled and also that many BI vendors have BI portal products there is no doubt that they are technically ready. They are however missing one thing - data, your data. Either they point their tools at you databases and access them over the net or they will need a ready supply of data from any BIaaS subscriber. If you already use SalesForce.com you can bet that all BI vendors entering this market will do so with an ETL adapter for SalesForce to get at your data on your behalf.
Of course SalesForce.com itself is no doubt keen on the BIaaS market and is already active in offering added value in terms of BI to existing clients.
Nevertheless, while simplicity, ‘point your browser and go’ and cool pre-built reports and graphs are the obvious attraction, there are implications when adopting BI as a Service in any business. The most important of these is that companies may need to supply their data to BI SaaS providers for upload to BIaaS sites so that ‘instant BI’ can be made available back to them via hosted web enabled BI tools and pre-built reports. There are also privacy regulations that have to be adhered to in this kind of situation not least the UK Data Protection Act. Companies need assurances on data protection as well as data security and should consider the implications of this in terms of giving BIaaS providers their precius operational data to be managed off-site. This is after all, the crown jewels of any business and there is no doubt that BIaaS providers would jump at the chance to know much more about their clients and would be sitting on a potential old mine with all that data. Reliability of such a service is also paramount so that BI is available when you need it. Companies considering this option should also think about what happens if they need their data back in-house and how easy is it to get it back from a BI SaaS provider. It would be madness to subscribe to such as service and overlook this requirement.
I have wondered about the potential size of the BIaaS market but it was not until I was looking at iGoogle a while back that I realised the real potential of BIaaS. Google have been steadily adding more and more services to their portfolio and are now making these services available over the internet for you to personalise with your own portal via iGoogle. All you have to do is click “Add Stuff” on iGoogle to see the huge number of instant services you can add to your portal. So what am I driving at here?
My question is this. How long before Google enters the enterprise SaaS market with avengence? Both SaleForce itself and a BI vendor could easily be a target to this Internet search giant. If you could get at hosted enterprise services in a SaaS offering just by using iGoogle “Add Stuff” to add it to your portal then how many SMBs would do it? That is a huge question. My guess however is that if it is as easy as Google are making it to add stuff on iGoogle today then the uptake by SMBs could be enormous. All Google have to do is solve the data upload problem and deliver vertical data marts for the industry of your choosing and they would no doubt get the attention of SMBs. So for those watching the BI market for mergers and acquisitions, I would not exclude Google from the mix. We may well see a very big splash if Google decides to move on the BI market to open up its stall as a BIaaS provider to SMBs. An iGoogle for Business offering would certainly do it. Who knows - I’m certainly watching with interest.
Do you have .Net 2.0?
Source: Oracle Business Intelligence Blog [link]
The Oracle Business Intelligence Add-in for Microsoft Office (Excel and PowerPoint) requires the
.Net 2.0 Framework. The client installable comes prepackaged with the x86 32-bit .Net 2.0 redistributable, which adds approximately 22MB to the download size of the BI Office Add-in.
My question is: do you already have the .Net 2.0 Framework installed on your Windows PCs? Please take a few seconds to
OOW 2007 News
Source: Oracle Business Intelligence Blog [link]
Oracle OpenWorld 2007 (San Francisco, Nov 11-15) is just a couple of months away.
The Session Tracks | Oracle OpenWorld 2007 page lists four tracks:
ApplicationsTechnologyIndustriesServices and you can view a listing of all the tracks and content at this page: Oracle OpenWorld - Content Catalog
Some sessions pertinent to BI:How-to: Oracle Business Intelligence Enterprise Edition and
EUL Horror Story - From Michael’s Blog
Source: Oracle Business Intelligence Blog [link]
An EUL horror story - from Michael’s blog. Don’t read that while drinking coffee or anything hot 
Hilarious.
What is new in OLAP engine in Katmai CTP
Source: Mosha Pasumansky [link]
After writing my previous blog post about dynamic named sets, I received feedback from multiple people saying the same thing: “How come nobody knew about this great new feature for the last 3 months”. Now, one of the constant complains about AS2005 was inadequate documentation. Almost two years after AS2005 release, the documentation improved greatly with several BOL Web releases. In the upcoming AS2008 release, all the new features are properly documented in CTP BOL. However, the problem seems to be with the “What’s new” documents. They are written at pretty high level and don’t list every single small feature. And since there is no easy way to get a diff between BOLs for different releases. Additionally, after July CTP was released, the “What’s new in June CTP” document disappeared, or at least I wasn’t able to locate it, there is only “What’s new in July CTP” document. Therefore there is simply no way for somebody to get list of all changes from AS2005 to Katmai. Therefore, I compiled the list of changes specifically in the OLAP engine only, i.e. I omit both Data Mining and Tools (UI/AMO). Of course, my list is likely to be incomplete, but until there is a real one - better something than nothing.
Analysis Services Personalization Extensions
This feature includes:
- Plug-in extensions to the AS stored procedures
- Events such as SessionOpened, SessionClosing, CubeOpened, CubeClosing
- Additions to the server ADOMD.NET object model to enable altering of session state (such as adding calculated members, named sets, KPIs etc)
BOL: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10md_3devsamp/html/a3e9bc0d-d56d-42a1-9c09-aa93e4f49bb2.htm
MDX: Dynamic sets
BOL for CREATE SET: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10md_6mdxassl/html/eff51eeb-5e7e-4706-b861-c57b6f3f89f0.htm
BOL for MDSCHEMA_SETS: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10md_9trfmisc/html/abb00dc0-2b83-48d6-b2ba-6615c1488d06.htm
Mosha’s blog: http://sqlblog.com/blogs/mosha/archive/2007/08/25/mdx-in-katmai-dynamic-named-sets.aspx
MDX: CREATE/DROP KPIs
BOL for CREATE KPI: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10md_6mdxassl/html/87618fef-95e5-4dd0-a650-aeb60ccbddcb.htm
BOL for DROP KPI: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10md_6mdxassl/html/d19c6809-b8a6-459d-8554-b41854f7cc45.htm
MDX: UPDATE MEMBER statement
This statement gives ability to modify calculated members on the fly without changing their solve order/pass/calculation precedence. It enables some interesting scenarios.
BOL: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10md_6mdxassl/html/07ab708d-d165-4fb1-a9f9-fb8197ff0dab.htm
MDX: CAPTION, DISPLAY_FOLDER, ASSOCIATED_MEASURE_GROUP calculation properties
Previously it was possible to specify these only through AMO or through XMLA ASSL. Now it is possible to use these inside CREATE MEMBER statement.
BOL: ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10md_6mdxassl/html/49379217-be2c-4139-a206-1168078b9b76.htm
Backup changes
In Katmai Backup changed dramatically to feature new storage implementation, improved performance, better scalability and removing limitations around the size of the database and individual files inside database.
Vidas’s blog: http://www.ssas-info.com/VidasMatelisBlog/28_ssas-2008-katmai-upcoming-changes-in-the-next-ctp
MDX: CREATE CACHE statement
Technically this is not a new thing in Katmai, because support for CREATE CACHE/WITH CACHE was introduced in SP2 (previously even though the syntax was accepted, these commands did nothing). However, when I looked at the list of changes in AS2005 SP2, I noticed that this feature was missing there.
P.S. I have a feeling that I forgot something, if I will remember, I will update this entry.