I attended a breakfast forum this morning sponsored be Vista Ventures, an early stage venture capital firm here in Boulder, CO. The speaker was Ike Nassi, EVP for SAP Research. During his talk, Ike discussed “Green IT”. He kicked off a whole series of thoughts in my head about how we can use BI to reduce our carbon footprints. So here goes:

According to AMI Research, “about 26% of small businesses, and about 33% of medium businesses currently use and/or plan to use SaaS.”

This week they published a SaaS research report that features LucidEra and 5 other vendors who “are bringing new value to SMBs by concentrating on making SaaS solutions more affordable, accessible and flexible.”

LucidEra’s 48 Hour Pipeline Healthcheck for salesforce.com customers is recognized for providing customers with “actionable information in the pre-sales phase,” thereby “speeding time to solution value,” and “reducing friction throughout the solution lifecycle.”

You can download a preview of the report here.

Evidence came in yesterday afternoon that, so far, BI is doing well in the new economy. Greg Turman, director of sales for the Western Region at Corda, phoned to say, “I’m going through the roof,” by which he means sales are good. “I’m seeing tremendous startup activity. People are saying, ‘I can no longer guess. I’ve got to know what’s happening so in case things get tight I’ll survive.’”

What’s behind the surge? He figures that people have finally understood the value of analytics and visual analysis.

I was working with Jason Garforth today on creating a TPC-H benchmark script which we can run on our warehouse to initially get a baseline of performance, and then, from time to time, rerun it to ensure things are still running with a comparable performance level.

This activity was on our new warehouse platform of an IBM Power 6 p570 with 8 dual core 4.7GHz processors, 128GB RAM and a 1.6GB/Sec SAN.

Jason created a script to run the QGEN utility to generate the twenty two queries that make up the TPC-H benchmark and also a “run script” to then run those queries against the target schema I had created using some load scripts I talked about previously.

The whole process seemed to be running smoothly with queries running through in a matter of seconds, until query twenty went off the scale and started taking ages. Excluding the 20th query, everything else went through in about three to four minutes, but query twenty was going on for hours, with no sign of completing.

We grabbed the actual execution plan and noticed that all the tables involved had no stats gathered. In such circumstances, Oracle (10.2.0.4 in this instance) uses dynamic sampling to take a quick sample of the table in order to come up with an optimal plan for each query executed.

The database was running with the default value of 2 for optimizer_dynamic_sampling.

After reading the TPC-H specification, it doesn’t say that stats should or should not be gathered, but obviously in gathering them, there would be a cost to doing so and, depending on the method of gathering and the volume of the database, the cost could be considerable. It would be interesting to hear from someone who actually runs audited TPC-H benchmarks to know whether they gather table stats or whether they use dynamic sampling…

We decided we would gather the stats, just to see if the plan changed and the query executed any faster…it did, on both counts, with the query finishing very quickly, inline with the other twenty one queries in the suite.

So, our options then appeared to include, amongst other things:

1. Gather the table stats. We’d proved this worked.
2. Change the optimizer_dynamic_sampling level to a higher value and see if it made a difference.
3. Manually, work out why the plan for the query was wrong, by analysis of the individual plan steps in further detail and then use hints or profiles to force the optimizer to “do the right thing”.

We decided to read a Full Disclosure report of a TPC-H benchmark for a similar system to see what they did. The FDR included a full listing of the init.ora of the database in that test. The listing showed that the system in question had set optimizer_dyamic_sampling to 3 instead of the default 2…we decided to try that approach and it worked perfectly.

In the end, given we’re not producing actual audited benchmarks then we’re free to wait for the gathering of optimizer stats, so we’ll go with that method, but it was interesting to see that option 2 above worked as well and illustrates the point that there is a lot of useful information to be gleaned from reading the FDRs of audited benchmarks - whilst, of course, being careful to read them with a pinch of salt, since they are not trying to run your system.

Another thing of interest was that in order to get the DBGEN utility to work on AIX 6.1 using the gcc compiler, we had to set an environment variable as follows otherwise we got an error when running DBGEN (also applies to QGEN too):

Set this:

export LDR_CNTRL=MAXDATA=0×80000000@LARGE_PAGE_DATA=Y

otherwise you may get this:

exec(): 0509-036 Cannot load program dbgen because of the following errors:
0509-026 System error: There is not enough memory available now.

Hello again and welcome to the fifth installment of our continuing video blog series Grounds for Conversation: Practical Insights for Customer-Focused Marketing.

Today’s topic is one that can obviously make a big impact on your marketing ROI and organization’s revenue: Engaging high potential customers.

To discuss this topic we’re pleased to have with us Michele Eggers, from SAS’ Customer Intelligence Global Practice. Michele shares her insight with us about how organizations can integrate their customers, products and offers into a complete, multifaceted marketing campaign by exploring new media types and getting smarter and more interactive in their customer communications.

Have a look:

Download the audio from this video in MP3 format.

 Subscribe to podcast feed&nbsp &nbsp

iTunes Podcast

I hope you enjoy the fifth installment of our series. Would you like to see Michelle back discussing other customer intelligence topics? - Let us know!

Be sure to tune-in next Thursday for episode #6, to learn how to establish key customer oriented metrics and incorporate accountability into your marketing process. Stay tuned!

Continue reading “Grounds for Conversation: Tracking Your Most Profitable Customers [episode #5]”

Today we announced that Mark Sellers, author of the book The Funnel Principle, and LucidEra’s founder Ken Rudin will deliver an interactive presentation at Dreamforce ’08 focused on sharing the secrets of using sales analytics and funnel management to succeed in doing more with less in a Sales 2.0 world. You can read the entire press release here.

“Breakthrough SalesPerformance is a sales consulting company with a singular purpose – to help sales executives use the sales funnel to achieve quota and other important sales objectives. The collaboration between Breakthrough and LucidEra is an obvious benefit to customers. Breakthrough’s proprietary funnel system increases the integrity of the sales pipeline data, while LucidEra’s technology allows sales VPs to gain insight from that data.”

- Mark Sellers, CEO of Breakthrough SalesPerformance.

“More than ever‚ sales organizations are seeking new ways to accelerate revenue growth without increasing resources. Cloud computing and the software-as-a-service (SaaS) model are helping to make business intelligence solutions more broadly available, but this is just the first step. Partnering with Breakthrough SalesPerformance allows us to continue to deliver built-in best practices to our customers and to take advantage of the years of sales pipeline expertise and knowledge.”

- Rob Reid, CEO of LucidEra.

LucidEra and Breakthrough SalesPerformance will be presenting at the Salesforce.com Dreamforce ’08 conference on Monday November 3rd at 2:15 pm. The session is geared towards sales management and sales operations and is called, “Got New Game? The Secrets of Funnel Management in a 2.0 World.“ LucidEra is a Silver sponsor of the Dreamforce ‘08 conference. More information on the Sales Exective sessions is available here. We’ll be at Booth 426 - please drop by and say hello.

The deadlines for submitting presentations to the Kaleidoscope and Collaborate conferences are rapidly approaching. Collaborate is being held in May of 2009 in Orlando (yet another conference in Orlando, yippee…) and their abstract submission deadline is Friday, October 31. Kaleidoscope is held in June of 2009 in Monterey, CA (hey, someplace I’ve never been to a conference!) and their deadline is Monday, November 3.

People involved
in business intelligence (BI) and data warehousing (DW) projects are
very familiar with terms such as facts, dimensions, attributes,
surrogate keys and slowly changing dimensions. But knowing the terms is
not enough. Increasingly, there appears to be a significant disconnect
between how BI or DW systems are actually built versus the best
practices described in books and articles. Clients point out that the
concepts in some books and articles are esoteric because they are not
connected to real-life situations. They teach the how but not the why.
Two best practices that people think they are implementing but often
fall short of are dimensional modeling and the hub-and-spoke
architecture.

When you don’t know why you should design a DW a certain way,
it’s easy to make the common mistake of designing DW and BI systems to
look like your source systems.

>>> Read the rest of my article Don’t Stop at How, Learn Why on the DM Review website.

In defense of the lively, Mark Madsen observes the nature of resistance to open source BI tools. An excerpt:

Overcoming someone’s resistance to open source in your organization means that you probably need to educate them, given that they use open source every day without thinking about it. It’s in everything from cars to cell phones, as well as almost all the commercial BI tools shipping today. More likely, they are resistant because they (a) are threatened in some way by the change you propose, (b) face organizational obstacles like educating the legal department about licenses or (c) face political consequences you aren’t aware of. It’s often their personal situation that is the biggest factor, given that most objections are easily refuted as myths.

DM Radio editor Eric Kavanagh puts on a scary mask for a special Halloween show this afternoon: “Scary Stories of Information Management.” Scaring you will be quite a trick after a year of cadaveric prose in BI articles and blogs. But there’s probably more where that came from. He wants your stories of fright and demons. Details here.

It’s hard to see through the smoke as our financial house burns down, I know. But what I’ve noticed is more interesting: the first signs of rebuilding.

This month, three experts I read—visual analytics expert Stephen Few, Competing on Analytics author Tom Davenport and digital-media economy specialist Umair Haque—all seem to have knit recent blog posts with the same thread: honest value in business and the economy.

“The macro crisis isn’t really a financial crisis, an economic crisis, or a solvency crisis,” writes Haque. “It’s an institutional crisis: the economic institutions of capitalism are in shock.”

Perhaps it’s the shock of having been conned. One of the three levels of transformation he suggests is the return of authentic value.

Authentic, long-run value isn’t created through arbitrage or gamesmanship — what we too often confuse strategy for. Games of off-balance sheet accounting, currency hedging, capital structuring, so-called labor arbitrage — where corporations simply shift to the lowest-cost, or most poorly regulated, sources of manpower — don’t create value. They just shift it around. Corporations who play this game of economic musical chairs are in for a rude awakening - because the music just stopped. And so they must rediscover the simple fact that value creation flows from making economic activities not just profitable in the short- run — but meaningful over the long-run.

Davenport lets some air out of “fluffy” social networking. He asks, “How can we really be producing value if we’re all sitting around blogging and Facebook-friending each other?”

All this fluffiness will be hard to maintain in our next period. He writes that “it wouldn’t be a bad outcome if the current crisis led to a more diligent, industrious economic climate.”

Few takes us down to products. He argues for honest value in business intelligence software. He often confronts badly designed dashboards—adorned with eye candy and other silliness—and urges those who listen to educate customers, not pander to them. The response is too often that, yes, we know, but it sells so there’s nothing we can do. Ah, promiscuity as a business strategy!

Any business that measures its success by current sales revenues or profits without regard for the effectiveness of their products will go for the silly stuff every time. I could argue that this is a poor business model because it’s short-sighted and doomed to fail, eventually resulting in declining revenues, but what’s the point? Businesses built on this model lack the foresight to appreciate the greater intelligence of long-term planning around products and services that effectively address the real needs of people. I believe the root problem that belies such business practices is not strategic short-sightedness or a myopic focus on sales—these are symptoms of a deeper, more fundamental problem. I believe that it’s wrong to build a business on self-interest alone.

Enough with this promiscuity. He adds, “Things that don’t work should not be sold—period. That’s good business.”

If the next president and Congress take Davenport’s advice and nurture big importers, high tech would surely be one of the first picks. How devastating, though, if Indian and Chinese entrepreneurs took advantage of an easy opening and overtook the U.S. even there.

I happened to read Few, Haque and Davenport. But I’ll bet they’re among many others these days arguing for a return to honest value and good business.

A research note from the 451 Group “Open source is not a business model” discussed that selling open source is a tactic rather than a business model.  They then went on to discuss the various business strategies of 114 open-source vendors.

I’ve got a couple of thoughts on this, but I’d also really like to hear your take on this in your comments.

First, there are other technologies that are also tactics, not business models. These include on-demand or software-as-a-service (SaaS) software and cloud computing. There are many people who hope that all three will change how software is distributed and used in our industry. All claim to have a lower TCO (total cost of ownership), but whether that is true depends on the specific business strategies employed by each vendor.

The advocates of these technologies proclaim that they are the wave of the future.  This may indeed be true, but just as PCs, client server computing and the web took years to become “mainstream,” it may take a while for these technologies too. Keep in mind that the leaders in these technologies, especially if they are small vendors, may not be the leaders (or even exist) by the time these technologies “cross the chasm.”

Second, and more important, is the underlying fact that it is the business use of these technologies that will drive their adoption and spread not the tool itself. There has to be a business use for the software before it is adopted. If a business does not need the required application then it does not matter whether it is offered on-demand versus on-premise, proprietary versus open source, or just hanging out on a cloud.

We have been experiencing a bear market and a recession lately. In general the stocks in our On-Demand Software and Business Intelligence indexes have fared poorly. When these stocks and companies rebound, besides the usual stock fundamentals, one of the key ingredients to success both in the marketplace and the stock market will be whether the vendors have the business applications that corporations are willing to pay for. Companies that are laying off people and have hiring/expense freezes are probably not going to adopt salary or expense management software regardless of the cost. Software targeted at new automobile or real estate sales will also get a lukewarm reception. Software oriented towards health care, particularly helping health care providers operate more efficiently, are likely to sell well.

Selling is always about meeting someone’s needs. Technologists too often think it is about the tools. Early adopters, often other technologists or companies with deep pockets, reinforce the Technology is King mindset. Sometimes it takes a bear market and a recession to highlight that it is not about the tool, it’s about how it’s used.

But very few (if any) actually execute on the vision that I am laying out here. This is a very short entry, but basically re-iterates some of the points of Dynamic Data Warehousing that I believe to be necessary before it (software/appliance/database) can be labled as being something like this.

Hoy me gustaria presentaros dos nuevos proyectos que empiezan estos dias sus singladuras por la web y que tienen pinta de que van a dar mucho de que hablar en los próximos meses.

El primero es Caribis, Inteligencia de negocio y pensamiento sistemico, el blog que ha comenzado mi amigo Carlos Luis y que espero que se lance a compartir todo el conocimiento que posee con todos nosotros.

El segundo me llega a traves Xavier Albadalejo, se trata de un proyecto de mas recorrido, no tanto un blog como un portal de conocimiento basado y centrado en enseñar y aprender SCRUM. Se trata de Proyectos ágiles una web que acaba de nacer y que creo que puede ser una referencia de las metodologías ágiles en España y Latinoamerica.

A ambas iniciativas les deseo mucha mucha suerte.

The next step after ‘Object Consolidation’ is Impact Assessment.

What is Impact Assessment? The process in which, we try to determine the variations or gaps in the existing objects/reports by comparing against the target platform.

The gaps or variations in an existing environment could be due to an existing function being replaced by a new function or an existing function not being supported with an equivalent function in the new platform.

Steps Involved in Impact Assessment

We try to do this comparison against the target platform ‘Impact Assessment’ in an automated way by leveraging the underlying metadata of the existing environment.

1.       Take the ‘object metadata’ collected as part of the Object Consolidation

2.      Collect details of the possible issues faced during the upgrade or migration process. The source of details would be from

a.       prior experience in executing similar projects

b.       the manuals and release notes provided by the product vendor

c.        the pilot project executed with the subset of objects from the existing setup

3.       Convert the ‘issues’ identified into a relational structure table.

a.       An issue could be an observation like the function ‘sum’ has been now changed in the newer version of the product to the word ‘sumif’

b.      One way of converting this issue to a relational structure is to have two column names  ‘issue_case’ and ‘issue_type’ where in issue_case will carry the value ‘sum’  and issue_type will carry the value ‘aggregate’.

c.       Converting to a relational structure enables using SQL queries for automated search of the impacted objects by joining the ‘object metadata’ table with the ‘issue’ table

4.      Run SQL queries joining the ‘issue’ table with the ‘object metadata’ table to determine the impacted existing objects

5.      Classify each object by the degree of impact(number of impact points) and decide on the strategy of upgrade/migration for each these groups

Benefits of Impact Assessment

1.       Foresee the challenges and being well prepared for system upgrade or migration

2.       Ability to estimate and plan the execution & testing phases very effectively

3.       Enable building comprehensive test cases

4.       Minimize surprises and provide confidence to the execution team

5.     Helps in making decisions of whether we should consider a object to be built from scratch or upgrade/migrate

Impact Assessment Challenges

First gathering the knowledge of issues; the options are talk to a person who has done an upgrade project to collect the issue details or perform a quick pilot with an appropriate sample from the set of objects to determine the issues.

Second conversion of issues logs into relational structure and running of queries to determine the impacts; both of these would require a good understanding of the underlying metadata structure, so explore the metadata structure and understand them to the fullest from the point of analysis.

Next time let us discuss on one other key task in an upgrade project…