BI Blogs

There’s been a lot of talk lately about how the US economy is crawling out of recession. You may have heard terms like “bottom” and “trough”, seen graphs of GDP growth, and read articles referencing NBER.

You may be highly skeptical and unwilling to believe that the latest recession is a thing of the past; after all, you likely know someone still looking for a job and you likely know someone else who recently lost a home. Or, you may be very optimistic, and knowing how the business cycle revolves, you are beginning to invest and think about growth. Understanding the business cycle will help you make better decisions — individually and in business.

Off the bat, I should state that economics is not an exact science (but you likely knew that already). And while there are a few axioms (supply/demand, money supply) and indicators (GDP, unemployment) that can help paint a realistic picture, most of what you see and read is based on analyst insight and experience, backed up with historical data and predictive models. There are faults all along the way. When you consider the complexity of the global economy, it should be clear that economics is more of an art than a science. The Business Cycle, though, seems to be something you can count on.

What is the Business Cycle?

Plainly put, the Business Cycle represents 4 phases of aggregate economic movement, ranging from periods of high growth to recession and back again. John Maynard Keynes (d. 1946) referred to these cycles as “waves of optimism and pessimism”, or to put it another way, waves of expansion and contraction. These phases were first written about more than 50 years ago by Arthur Burns in his book “Measuring Business Cycles”.

The four phases

1. Peak - The highest point of economic output just before a downturn
2. Recession - When the economy actually shrinks, or contracts
3. Trough - The “bottom”
4. Recovery - The economy has stopped shrinking is growing once more

Last 5 US Business Cycles

Data Source: NBER

Notice that peaks and troughs are represented by month and year, while the other two phases are measured over a number of months. You’ll also notice that recession and recovery periods can vary greatly.

Who determines when each phase begins and ends?

In the US, the task is managed by the National Bureau of Economic Research (NBER). NBER is a private, nonprofit, and nonpartisan research organization (stocked with Nobel Prize winning economists). They work on many economics projects and work closely with businesses and universities. They self-proclaim their dedication to promote “a greater understanding of how the economy works”.

For example, NBER most recently concluded that “the last [US economic] expansion ended in December 2007″. We know that after such expansion, according to the Business Cycle, will come a period of recession, followed by a bottom, leading to a new period of expansion.

Does everyone agree?

In short, nope. Milton Friedman, to name one prominent example, believed that the economy fluctuates rather than cycles. The new classical framework states that the economy is much more flexible than that which is implied by the business cycle framework.

There’s also an issue of market equilibrium. Having a somewhat predictable business cycle implies that the markets will be out of sync quite a lot — allowing some speculators and investors to take advantage of price differences at different phases of the cycle (this is called arbitrage, take a look into Rational Expectations Theory as well). Other differing methodologies include the credit/debt cycle, political cycles, and Marxian cycles.

Final thoughts

Even against dissenter argument and alternate viewpoints, the Business Cycle framework still works and is easily observable. Economists at NBER continue to assign dates to peaks and troughs. Individuals, businesses and organizations still base many of their purchasing and hiring decisions on what phase we’re currently in. This likely won’t change any time soon.

The only problem with relying on NBER is that they lag reality. For example, we have very likely reached the bottom of the current cycle and are now in a phase of recovery. NBER might make it official at some point this year, maybe next year. Investors waiting for official announcements will find that they’re missing the boat.

It took me a while, but I tracked down the online links and filled out the survey to get access to the CTP2 release of Gemini known as ‘SQL Server 2008 R2 August CTP and “Gemini” Add-in’.  I also signed up for the CTP3 Gemini Preview.

Here are the screenshots of the install and a quick demo of using the Excel add-in:

Notice the new Add-ins being loaded when Excel 2010 is launched	Message that is displayed to request permissions to install the new Add-in	Notice the new Gemini section in the toolbar and that the circular Office button in the top left has been replaced with a new rectangular tab
Gemini items available in the Ribbon	Version information and feedback options	New Windows opens up when you are going to load & prepare the data
Gemini Client Window Home Ribbon options – notice the new icon in the top left	Gemini Client Window Column Tools Ribbon options	Import wizard to get data from database
Setup the connection to database server and database (using Project REAL Warehouse Sample database)	Select the source tables and you can chose the option to select related tables and preview/filter the data also	Once you are done you will get the import summary (selected the date, store, item dimension tables, and the fact store inventory table)
Import progress, notice that the fact table is already displaying more than 2 million rows!	Each source table is pulled into separate worksheets in the Gemini Client Window – almost 4 million fact records!	Create relationships between the tables once they are imported
Progress of the relationship being created	Ability to manage the relationships (view them, add new ones, and delete them)	Now you are ready to start working with the data and create a PivotTable or PivotChart within the Excel Workbook that you initially created

That is a quick overview of the install process and a walkthrough of getting the data established with the new Gemini Add-in.  The data that I used for this example was from the Project REAL Reference Implementation.

For additional information in regards to Gemini take a look at some of these areas:

• Gemini Team Blog (RSS Feed)
• Microsoft BI Blog – Gemini (in particular Preview of Project Gemini)
• Interesting Gemini links (from Chris Webb)
• SQL Server 2008 R2 | Self-Service Business Intelligence

The release of the first Madison CTP was announced on the Data Platform Insider blog earlier this week:
http://blogs.technet.com/dataplatforminsider/archive/2009/08/24/microsoft-ships-the-first-technology-preview-for-project-code-named-madison.aspx

It didn’t cause much interest in the blogosphere – probably because the CTP isn’t publicly available, and even if the likes of me could download it it’s not the kind of thing I could install on a VM on my laptop…

Anyway, the question I’ve always had about Madison was whether integration with Analysis Services in ROLAP mode was going to be a priority for the first release. I now have it on good authority that it will be, and it should mean that with SSAS + ROLAP + Madison we’ll be able to create multi-terabyte (or larger!) ROLAP cubes that are super-fast. The new in-memory (IMBI) storage mode that Gemini features, and which we can assume will be appearing in the next full release of Analysis Services, will certainly increase the scalability of Analysis Services beyond the terabyte or so of data that’s currently feasible but apparently Madison will be the answer for the really large volumes. I wonder if HOLAP is an option here too? MS will certainly need to do something about the rubbish SQL that SSAS generates for its ROLAP queries before it can get the best out of Madison though.

Oh, and if anyone out there is on the Madison CTP and wants someone to help out testing it with SSAS, I’d be happy to help!

IT Business Edge writer Michael Stevens recently published a document called the “Data Warehousing ETL Checklist” (link to article with document). To get the file you’ll need to register with ITBusinessEdge (if you don’t already have an account).

The document isn’t a development checklist, but rather “a high-level checklist of the important topics”. Stevens has made a good effort to distill the various data integration topics and issues you’ll need to consider before you start your project.

Highlights and feedback

Mind scope and do not “try to boil the ocean”. Good advice. But I disagree with Stevens on how to approach one aspect of the scoping issue. He suggests that “in some cases, it may be better for the business as a whole to leave certain data sources untouched”. As I wrote here, I don’t think that scoping to the data source is a great approach. But you can decide.

I would also add that instead of looking at “Target System Content”, you should focus specifically on Business Processes — and this is regardless of the type of system your integrating into (MDM, DW, OLAP, etc.).

Data Profiling is a very important topic, and I’m ecstatic to see it on the list. But if it were up to me, I would have made it second or third. Almost every single failed or over budget ETL project can be linked to data source woes (quality, data availability, inadequate documentation, etc.). Most of these woes can be relieved by conducting good data profiles — a process that can begin during the initial requirement gathering phase of the lifecycle.

The data profile will reveal which source systems are useful. There may be redundancy, data quality, and data access issues compelling you to exclude a particular source. This isn’t a scoping issue, though. Go back to the project sponsor and to the board; report your findings. You may have other issues in your organization to resolve first.

In addition to Data profiling, I’m very happy to see that Naming Conventions made the list. Tied in closely with naming is the concept of conformity. I’ve written about this before, so please check here. In general, a label and its data must conform and be consistent throughout the entire system. And as Stevens points out, they must be agreed upon by the users.

For metadata, I also consider a third type: Process Metadata. This is technical data about all the processes (ETL, jobs, automated processes, reports, etc.). This data is then utilized directly by business users and administrators to monitor and audit the data warehouse. Is my report ready? consult the process metadata. Did that job finish? consult the process metadata.

Lastly, don’t forget to think about auditing, compliance, and lineage. If you’re making decisions based on the data retrieved from your data warehouse, you’ll need ways to trace data lineage. You’ll need to know when data was loaded, what version of the ETL system and business logic was used, and even why a particular data item might have been changed in transit.

Conclusion

This is a good high level checklist that you can use before you begin to dig deeper. It gives you a snapshot of some of the high-level decisions you’ll need to make before you get started with your next data warehousing project.

Perhaps Stevens will incorporate some of my feedback into a version 2!

Make sure that all the Panorama developers has the same version of Panorama NovaView
Desktop and that it’s the same version of the Panorama server. In addition, check
that all the Panorama servers (meaning development server, production server, etc.)
also have the same Panorama version. It’s very unpleasant when you copy the views
from the development server to the production server while installing your new or updated
BI project just to find out that some of the views don’t show because of that.
That’s right - it won’t cause all of the views to malfunction, it will only affect
some. That’s the dangerous thing and this is why you need to double check it immiedetly.

in my last
post I recommended you to learn about the batch mode feature of Panorama. There’s
one more thing you’ve got to know: Check you’re code flow very well because if you’ll
use batch mode within batch mode, meaning that you’ll call the EnterBatchCommandMode
function after you did it previously, the view will get stuck. Be sure that you don’t
have such a case.

This afternoon during lunch I presented the webinar Business Dashboards – Developing a Culture of Intelligent Execution.  I think we experienced some audio issues, so I apologize for that.  I hope everyone got something out of the webinar and I really enjoyed all of the great questions that were asked.

I have gone ahead and uploaded my presentation to my SkyDrive account here (includes screenshots of demos and all resource links)

and the recording should be available on the Chicago Technology Executive Club website next week to watch here in the Webinars section and you can apply an additional filter to get at the Business Intelligence Track.  They also have a great section with additional resource information with a bunch of white papers in their library within the Business Intelligence section.

I want to thank everyone that attended today and please feel free to leave me comments or send me a message through this posting or blog site if you have any additional follow-up questions.  You can also contact Magenic at info@magenic.com to schedule a follow-up talk to discuss any potential projects or assessments that you would like to get started on within your own organizations.

Thanks again.

Where to focus your decision making during a recession? Here’s ten areas to consider:

1. Evaluate your business model: Is it time to move on?
2. Is it possible to cut costs by increasing productivity?
3. Review your customer base: Are you addressing the entire potential market? Is it the right market for your product? Is your product right for the market?
4. Consider how to provide even more value to customers and clients.
5. Evaluate your marketing: Are you getting the right messages across to your markets in the right way?
6. Don’t lose knowledgeable people with valuable skills: Re-deploy them.
7. Invest in skills, especially among decision-makers and sales staff: Prepare them thoroughly for what has to be done.
8. Evaluate competitors, distributors, channels to market to find innovative ideas.
9. Assess the evolution of your industry and prepare to work with different partners and suppliers.
10. Identify new products and services to invest in.

Read more in aking better decisions, a report from the Economist Intelligence Unit, sponsored by SAS.

I was intrigued by Ken King’s recent post regarding North Carolina telecom executives supporting Dr. Goodnight’s call for education reform. In fact, it brought to mind an education customer we’ve been working with lately. The school is in a rural community that has implemented a 1:1 laptop initiative and is using SAS Curriculum Pathways. An interesting point I’ve heard the teachers make is that the kids now love working with the online content within SAS Curriculum Pathways, but at first it was a cultural shift for them.

I was very surprised by this since when I look at the product, I would have loved to have used it in school. So, we asked the teacher why this was the case. She mentioned that now the students HAVE to be engaged in class, and that using SAS Curriculum Pathways MAKES the students think. It seems that the students were used to the 20th century curriculum of being lectured to. This was something that would not have occurred to me. However, it perfectly fits in with Dr. Goodnight’s comments about keeping children engaged in class and using technology that students are used to in their personal lives. It also supports Bruce Friend’s recent post, Students’ speak up on Technology in the classroom. Students do want to use technology and consider it invaluable and inseparable part of their life. It would be interesting to me to apply that same survey to students at schools like the one I mentioned above. I think the results would show the progress that has been made to marry technology and curriculum resulting in student engagement and success.

Also in Ken’s post, he mentions that communication service providers, such as AT&T and others, are taking interest in helping schools with funding for 1:1 laptop initiatives. I think this is a wonderful idea with one caveat. It is not just the technology that will make the schools (and thus the students) successful. It is the marriage of three things: the hardware, online curriculum and professional development. Having met recently with several successful schools, they all emphasize these three areas. In the case of the school I mentioned above, they took teachers out of class for 3 days for intensive training on both the hardware and on SAS Curriculum Pathways. At first the teachers were apprehensive about being out of their classes so long; however, after the training they all expressed that it was very important for them and made a true difference in educating their students.

Most people are apprehensive about change but often come to appreciate change once it occurs. The students and the teachers at this inspirational school were both unsure about the new technology in their classroom. But now that they’ve implemented it, everyone is enjoying the benefits. In fact, the teachers were impressed by the content in SAS Curriculum Pathways. They mentioned that without SAS Curriculum Pathways, they would have to personally go search and find all that content themselves which would be 20 times more time consuming- time that they don’t have in their already busy schedules and time that could be better utilized in helping students learn.

Like any good SAS employee, I monitor the social Web for conversations about analytics. Not that I’m an analytics geek – far from it. As a lifelong writer and marcomms veteran, the quants view me as about as comprehensible (and as substantial) as navel lint.

It’s for precisely that reason that I look for articles and conversations that address analytics, particularly business analytics, in ways that everyday people can understand. Whether you’re a consumer, business owner, bureaucrat, board member or investor, analytics affects your success.

So I giggled with glee when I stumbled on this new blog from down under. Oz Analytics , by Business Intelligence expert Steve Bennett (note to Steve – I had a very tough time finding your bio and business info on the blog. Would love to know more), is one of those gems. Steve writes in very down-to-earth terms about such lofty concerns as data quality (how to convince the big guys that it matters), web analytics, social networking, information as an asset (and a living one, at that), all with lots of graphs, grams and visualizations. I actually understand what he says.

But here’s my favorite – because it gets at what undermines effective business practice at every size, every level, every industry. The post, 10 Signs that You Need Analytics pinpoints the insider pains (we’re not even talking about the money dumped on poorly targeted campaigns or misguided loan policies – just the day-to-day little cuts) caused by lack of effective analytics. I guess it’s the flip side of SAS’ “This is your life” video from last year. Here are Steve’s top 10:

1. You have to wait longer than a day for either IT or your business intelligence department to make/change a report for you.
2. Across the organisation there are more than 100 requests pending for reporting /dashboard /scorecard changes waiting for a specialist to deliver them.
3. When you attend meetings, there are multiple numbers being quoted for the same thing – and you don’t know which of them is correct.
4. When you talk about fundamental things like transaction, account, balance or available stock – and you discover that the person you are talking to is using the same words but means something different to what you mean.
5. You can’t get an instantly understanding when glancing at a report/dashboard/scorecard and what it is telling you.
6. The commentary is larger than the automatically generated report.
7. The report is not generated automatically but is a handcrafted labour of love by either yourself or one of your staff, or you spend hours trying to locate the right data and then have to consolidate it manually into Excel.
8. It takes longer than 5 minutes to view a new report.
9. You can’t access the report when and where you most need it.
10. There are hundreds of reports available to you but you don’t trust them and you spend time trying to manually validate key numbers.

Anything look familiar to you? Go read Steve’s entire post, which includes his definition of analytics.

Cynthia Marshall, president of AT&T North Carolina introduced Dr. Goodnight to an audience of telecom executives at the North Carolina Telecommunications Industry Association’s 78^th annual convention. AT&T has joined Dr. Goodnight in supporting his 1-to-1 computing initiative and Ms. Marshall asked the audience members to also join this worthy cause. Dr. Goodnight presented discouraging statistics about the drop-out rates among our high-school students and the negative consequences for our economy. He attributes much of this problem to schools that do not engage students because schools are not using today’s technology, but still offer a 20^th century curriculum. The telecom executives responded enthusiastically to Dr. Goodnight’s demonstration of SAS Curriculum Pathways, both because they want to help improve our schools but also to benefit their business.

Not only is The United States losing ground to our global competitors in education, we are also losing ground in broadband. In 2001 the United States ranked 4th in the world in per capita broadband lines. Today the US ranks 19th, with only 56 lines per 100 people, compared to 89 in Japan, 86 in Sweden, 81 in The Netherlands, and 79 in Italy. Not only does the US have fewer lines, but also slower access speeds.

The problem is not a lack of broadband infrastructure. More than 90 percent of North Carolina’s households can get broadband from at least one provider. The problem is low adoption rates among middle and low-income households, particularly in rural areas. Many consumers don’t see the value in home broadband, particularly in today’s difficult economic climate. Telecom companies are searching for applications that will drive higher adoption rates so they can recoup their investments and fund new investments.

Another problem for telecom companies is that the economic recession is accelerating the number of American households that are abandoning their home land line phone. The process has been underway for some time, but now 700,000 American households a month are cutting the cord. Smaller CSPs, those serving rural areas and small towns need another revenue generating option. Not only is their business under threat, their communities have been hard-hit by jobs in manufacturing, agriculture, and textiles.

The American Recovery and Reinvestment Act allocates $7.2 billion for broadband related projects. Private investment is only about $4B annually so the stimulus is very significant. There is general agreement that broadband providers would bypass rural areas for several years without the government incentives. Most of the companies present at the conference have or soon will be applying for broadband grants. The stimulus directs most of the funds to unserved and underserved rural areas. To qualify, applicants must address how they are providing service to schools and libraries.

North Carolina’s communications service providers now have another good reason to expand broadband service, to lower the costs, and increase speeds. Giving our kids access to broadband technology, computers and SAS Curriculum Pathways is a win-win for everyone concerned. It keeps kids interested in schools, and prepares them to join into the 21^st century economy upon graduation.