Why all the fuss around BI? - Data Ownership

I wanted to write today, and address a question that I think needs to be well understood. That is, Why all the fuss around BI? I mean, what is the big deal about business intelligence anyway? To start, there is a real need, for true business & enterprise intelligence that empowers users in their jobs. This goes from the Day-to-Day roles, through middle management & all the way to C-level executives.



To many people however, equate business intelligence to a tool-set, when in reality business intelligence is all about putting your data to work for you! The tool-set is not as important, is understanding the data model, understanding the business, and creating things like learning loops based on lag & lead data that can help drive companies through to their corporate strategy.



There are two big problems that customers face today with most ERP implementations, and relating to business intelligence. First, perceived overload of data. Second, the right tool for the right representation of data in order to glean intelligence from. With that said, there are a number of other specific issues & myths that cloud the path to true business insight - all which should be addressed when thinking about creating a BI story for a company. Ultimately however, these two questions seem to pop-up whenever thinking in terms of BI, from big data, to the personal self-service landscape.

These issues exist for companies small and large, and further what adds dimensions - excuse the pun - to these issues are how companies & their culture perceive BI. So in order to help address some of this, we need to understand a few fundamentals.

1. Data Ownership & Stewardship

If you don't think this topic is the most important to grasp, then your not understanding the goal of business intelligence. It's important for those users who need business insights, to understand who really own's said data, and who are stewards of that data. To many times, customers believe that IT owns the data, which is not reality at all. In fact your IT department, at most classify's as stewards of data. IT is not a cost center, blocking you from your data needs. Instead your IT department is meant to empower you, the user, with secure access to one version of the truth, for your reporting needs.

Users are the true owners of data, with different levels of responsibilities for that ownership. Further some users are data stewards, just like your IT, but with different roles and focus. This must be clearly understood, to help start down the path of addressing the perceive overload of data issue I mention above. The way to address this, is having users feel like they do own their data. This is, of course from the processes that create it, but also from the power in reporting on it. Giving them true Personal BI tools, that allow them access to help form the questions that lead to business insights.

Other myths, and issues that help lead to this confusion - and over worked, stretched IT budgets on report creation and never ending change request is that BI is a separate tool, than that of your ERP system. With out-dated ERP design's this use to be the case, however now exists only as a myth for those customers of Microsoft Dynamics AX.



With Microsoft Dynamics AX, and the stack that it lives on, full BI capabilities exist from the Personal, on to Team and straight through to Organizational BI needs. To help get people talking the same vocabulary, and language, we have the BI Semantic Model or BISM, that helps put some semantics in how we reference and talk about BI artifacts.

Going back to the point now, we have the ability to give users the power to fully grasp their roles as owners of their data - and understand the point of IT and IS departments as being the stewards of the systems, and data that lives within them. Having this clearly understood, means that Users can be turned into fans and a System of Engagement can start to form. Ultimately helping address one of the major issues we see with all this BI fuss, that of perceived data overload. It starts with owning the data, and therefore owning the processes that feed such data. Further, this means understanding the impacts of things like Master Data Management concepts for correct modeling of global data attributes, organization wide.

In order for users to own, and becomes fan's of their data, solid functional modeling & design must exist during the implementation process. Finally, understanding that real business insights, that create things like learning loops, starts with data ownership.

That's all for now, but check back soon as I continue this series and we can turn this BI fuss, into value added business & enterprise insights. Next we will continue our focus, with looking into wading through the onslaught of choices, and help end with classification of BI artifacts for those Usersfans of your organization or customers! Till Next Time!

Follow Me @:
       

"Visit the Dynamics AX Community Page today!"

Dynamics AX 2009 - Reporting Possibilites

So we have talked about BI in the past few months. Went over several Microsoft tehnologies that make up the Microsoft BI platform. This includes SQL Server, Reporting Services, Analysis Services, Integration Services.

This is all the BI platform from Microsoft. And we now have started to look at how this BI platform fits in with and can be used inside Dynamics AX 2009.

The first dive into this was a pratical use for creating custom SSRS reports for a Dynamics AX 2009 instance, and looking into OLAP use for Dynamics AX 2009. (Link: Dynamics AX 2009 BI: Practical use with SSRS and looking at OLAP)

So with that we saw with Dynamics AX 2009, there is a new Visual Studio 2008, Project Template for Dynamics AX 2009 SSRS custom reports.

With showing this, and coming back around to all that we have covered with this so far, I think we are at a point to where we should look at the higher level of reporting options inside Dynamics AX, and get some higher level pros and cons about them.

So with that said, lets look at the reporting options for a Dynamics AX 2009 instance.

The point of reporting is to gain access to data that lives inside a System and it's database. This is the point of reporting with Dynamics AX 2009. To give users access to the different data they may need to perform functions, processes and help make business decisions.

So the reporting options with Dynamics AX addresses a spectrum of reporting. You can look at this as having Procedural based reporting on one end of the spectrum and Analytical reproting at the other end.

On this spectrum, there is a few different ways that reports can be created that span the above mention spectrum. The following is a high level list of these reporting options.:

Standard MorphX reports
The standard MorphX reports are the base reports that come inside Dynamics AX. Most of these are very procedural based in their nature. They are meant to delivery things like invoice journal data, Sales Order Confirmation, Invoices, etc. They serve a purpose to help fulfill a business process.

These kind of reports have the most flexibility for working with business logic, and Dynamics AX objects. They are also the most difficult to develop and modify at times, and are the older way of reporting inside Dynamics AX. (They have been around since the early Axapta days.)

You use X++, and MorphX to create, modify and work with these reports. End users can make use of custom queries and filter possibilities that can stay with that user for as long as their usage data is kept.

So this offers the most flexibility of any of the reporting options for getting at Data inside Dynamics AX. These are very developer driven reports, and super users would not be creating these.

Though this is mostly procedural based reporting, some analyical reports exists in this form, though not many.

Custom SQL Server Reporting Services (SSRS) reports
This reporting option, though could be done to some degree in earlier versions, is really just now a true offering in Dynamics AX 2009.

This makes use of a easier, Visual Studio driven development model, that .Net developer would find familiar to work with. This process creates report definition files, that are used and connect through the .Net Business Conector to work with Dynamics AX data and business logic.

The business logic can be access through using the AxaptaWrapper .Net class, that comes from making use of the .Net Business Connector and the Dynamics AX .Net Reporting Framework.

This is not as complex to create, makes use of a report designer that is easy to use, drag and drop possibility, and real flexible in design and layout possibilities.

This also executes on a SSRS server, which can offload some of the report generation from the AOS itself.

You don't have as much flexibility with this option as a standard report, for example the custom Qeuries for filtering a report, that are user driven can not be used. The developer of these custom SSRS reports must give access to filtering through the report design.

This option offers the ability to create procedural based or analyical based reports in nature. OLAP cubes, KPI's, etc. can be access and added as sources of information for building reports here.

Report Builder / Ad-Hoc Report Creation
This option first came to exists in DAX 4.0, and has been improved in Dynamcis AX 2009.

This makes use of the perspectives node inside the AOT, to expose report models with to the Report Server. This option is targeted at super users who want to get at report data quickly, and in an ad-hoc nature.

This makes use of the SQL Server Reporting Services - Report Builder tool, and based on your instance and the deployed perspectives, will govern what report models your super users can access to look and report on data.

This is very much a drag and drop ability, and meant to be a design so the super users don't have to know the data model.

This is the most limited of the reporting option, and that understandable as this is Ad-hoc, and not meant to give all access to everything. The biggest limit is access to the X++ calculated fields inside Dynamics AX. Those fields that are not just a single table field, but calculated depending on a number of different variables.

The point of this though is to get super user the ability to gain access to around 80% of the reporting data needed. If more is needed, then they can get with thier IT staff to deliver a custom SSRS report for the full amount of data.

This reporting option, like the one before, can offer Procedural and Analytical in nature reports. Access to Cube and Procedural perspectives allow access to different data, and the different data and relations that might exists.

SQL Server BI Development Studio (BIDS)
This is the out of the box tool that most standard Microsoft BI projects use to create reports with. This is also used to create and manage OLAP cubes and OLAP database, and create and managed SSIS packages and logic.

For the report creation this is direct access to the SQL Server database, and now interaction with Dynamics AX itself. So that is the biggest Con with using this to create reports with. No access to those, sometimes much needed X++ calculated business logic fields.

If you choose this option, and need those fields, you would have to 're-create the wheel' and duplicate that logic through T-SQL or SQL Scalar Functions.

This can be very Procedural or Analytical, depending on what your doing with BIDS. Also, BIDS is used, even if not for report creation, to manage a Dynamics AX 2009 instance OLAP cubes. To look at them, modify them, trouble shoot them, etc.


Well that's all for now. I think this is a good start to the reporting options inside Dynamics AX 20009 instance. Moving forward we will dive deeper into looking at each of these, getting some screent shots and real world examples.

See you soon!





"Visit the Dynamics AX Community Page today!"

SSRS Performance and scaling

Now that we have dove deeper and deeper into the BI world, one of the Technical things that needs to be understood by someone who would be implementing and administarting a companies BI solution, is proper planning of hardware, and also scaling-out for performance.

That means, that someone would need the ability to understand SSRS, how to scale, when to scale, differences in version between SQL Server 2005 and 2008 for SSRS, etc.

So lets start with an older article from MS TechNet. Planning for Scalability and Performance with Reporting Services

This article was wrote in 2005, focusing on SSRS for SQL Server 2005. Still the knowledge that can be gained from this article is a good bit, and a great place to start.

This document covers scaling up vs. scaling out, the technologies used with SSRS and getting into specifics.



Like some of the basics for SSRS performance would be spliting out the Report Catalog (SQL Serve Databases) from the Report Server (IIS and SSRS Service). That would be an example actually of a scale out.

More true to the term of a scale out would be though, taking the Report Server, and having a few different boxes that run the Report Server role, and load balance them for execution. These all would point to the same Report Catalog, and therefore could connect to Dynamics AX.

You can see this from the following:
"The remote configuration eliminates contention for machine resources between the Report Server and the SQL Server hosting the catalog. However, you must provide adequate network bandwidth between the Report Server and the catalog server.

Scale-Up and Scale-Out

After you split the catalog to another system, you can choose to either scale up the Report Server, by adding processors, or scale out by adding machines. Figure 4 illustrates a scale-out configuration that uses multiple Report Servers to access a single catalog."


After you get a grasp at this level, the next step would be to go a little further with the when to scale out, and there is another article found here: Reporting Services Scale-Out Architecture



That one goes into more detail about the scale out architecture, and also covers the max number of concurrent users SSRS can handle, and how this increases when your scale out.

After getting this under your belt, the next level would be to understand the comparison between SQL Server 2005 and SQL Server 2008 SSRS. For that, a very detailed analysis was done over at SQLCat.com. The article for that can be found here: Scaling Up Reporting Services 2008 vs. Reporting Services 2005: Lessons Learned

From the post:
"Http Errors vs. Request Time-Out Errors

To better understand the hardware limits we hit with Reporting Services 2008 compared to Reporting Services 2005, we compared the HTTP 503 errors we saw with Reporting Services 2005 with the Reporting Services time-out errors that we saw with Reporting Services 2008. We discovered that while the failure rate for reports with Reporting Services 2005 was directly correlated with HTTP 503 errors, the failure rate for reports with Reporting Services 2008 was directly correlated with time-out errors. These time-out errors, combined with our analysis of the disk queuing on the data source, indicate that the data source was unable to keep up with the request for data from the front-end servers."


So This is all very important to understand, or have someone udnerstand who will be helping you get the best out of your SSRS part of your overall Dynamics AX 2009 solution.

Well check back soon, performance tunning Dynamics AX 2009 will be getting a lot of focus from my post. I will also be adding another Dynamics AX BI pratical post, focusing this time on making use of OLAP Cubes, and creating a custom cube for use within your Dynamics AX 2009 instance.

See you then!




"Visit the Dynamics AX Community Page today!"

Back again!

Well that was a nice long weekend. A good break, to take and get refocused. Sometimes we need breaks, in order to deliver our best work. So I feel refreshed this morning, ready to get back to things.

I have several things to focus on. For this blog, I will continue down the BI road for Dynamics AX, again going into more pratical uses of SSRS and OLAP with today's technology that you find in Dynamics AX 2009.

I will have post coming out about Role Centers as well, and also I will be starting a series on Performance Tunning your instance of Dynamics AX 2009. Creating a plan, and doing this to get the most out of your instance.

So there are plenty of great post coming. I also have plans for this summer to have more Microsoft interviews, including some with the product team about AX 6.0.

So check back soon and often!




"Visit the Dynamics AX Community Page today!"

The Future of Microsoft BI: Project 'Gemini'

So these past few weeks we have had a chance to take a dive into BI. What BI is from SQL Server, and how that is used as a based for the BI - SSRS, OLAP Cubes, KPI, etc. - inside Dynamics AX.

-What is SQL Server BI anyway?
-What is SQL Server BI anyway? - Part II

Now that we have a sense for what BI is in these terms, let take a step forward in the not to distant future, and look at what Microsoft has been, and still is calling, project 'Gemini'.

Project Gemini is the next step for the Microsoft BI solution, as is meant to be a real step forward into the theme of 'BI for the masses' (Dynamics BI: BI for the masses)

This BI for the masses theme, is Microsoft focus in making use of technologies companies already invest in, in order to offer up a true BI solution with. These technologies are: SQL Server, SharePoint & Office.

Project Gemini will allow information workers, to take and select from the millions of lines of data, from a datawarehouse, and use that to create reports, publish them on SharePoint, and use SharePoint to collaborate on them.

These reports can be created in Excel, driven by Excel, and super users making use of the latest Office technology will have the ability to create the reports they need, and have Excel Services update the reports as designed, and for delivery through SharePoint.

Also IT dept. will still have control over the data, and will be able to use the security and roles inside Excel Services, SharePoint and the SQL level to control access data. So this will not be a free for all, 70 versions of the truth problem. But an actual solution.

The Key points to project Gemini are:

- To ship First half of 2010

- Brand new technology, that will enable management of millions of lines of data called: ‘In Memory Business Intellegence Engine’

- This IMBIE (m-bye as I now call it) will reside with the excel environment. (based on current Microsoft information about it.)

- Excel services are then use to refresh the data, once published, and also used to delivery the data via SharePoint to other end users. Also Excel services can be used by IT pro’s to limit and control the access of data.

So this IMBIE technology, which is brand new, will be the key for allowing access and processing of massive amounts of data. This is the enabling technology, and look for more coverage on this, here on, in the future.

Also, TechEd, which is happening next week from May 11th - 15th will have a full demo of project Gemini from end to end, being driven by the Microsoft BI team. If your going, I highly recommend you check this out and the other BI sessions that will be going on there.

One last resource is a link to a nice presenatation that gives some insight into the bigger Microsoft BI picture, including working with data in the cloud.

BI Presentation from Microsoft

Well that's all for now. Check back soon, as more great post are coming out, and we will continue some BI as well, as we dive into actual, real world, Dynamics AX BI reports, technologies, and use with today's releases of technologies.

See you then!




"Visit the Dynamics AX Community Page today!"

What is SQL Server BI Anyway? - Part II

Recently I post this article: 'What is SQL Server BI Anyway'

In that article I took and addressed, at a high level, what SQL Server BI is. What technologies from SQL Server can and are a part of a SQL Server BI Solution.

We answered with that post: 'So what is BI in terms of SQL Server?' We talked about the 3 main teachnologies that make this up: SSRS, SSAS & SSIS.

Now having that knowledge level of SQL Server BI Solution, and technologies, that make it up, we can move forward with the next question posed during that post.

Why is this important to understand?

The reason why this is most important to understand, for someone wanting to know about BI in terms of Dynamics AX, is that the BI offered in Dynamics AX is based on the technologies that SQL Server BI Solutions are made up of.

On top of that, the BI solutions inside Dynamics AX add another part to the dimension of reporting, with KPI offered up by PerformancePoint Services for SharePoint. Also SharePoint itself plays another critical role in offering these reports, Cube, KPI point, etc. via Role Centers, targeted to information worker roles inside the Dynamics AX instance.

Once you understand the BI solution from SQL Server point of view, then you can truly incorporate that into Dynamics AX 2009 (and beyond) with how those technologies, were meant to be used.

For example, SSRS reports are meant to be used within Dynamics AX, and Dynamics AX role centers for reports. There is a new AOT node inside Dynamics AX 2009 called Datasets. These Datasets are part of what enables datasources for SSRS reports to be built with.

You also have OLAP cubes, which Dynamics AX comes with standard cubes out of the box. These can be built, processed, and then viewed from within Dynamics AX. For analytics, this is where your data comes from. Looking at different dimensions of data, across a slice of time, customers, vendors, inventory, etc. etc.

Then PerformancePoint comes in, which will not be delivered by SharePoint, and be a part of SharePoint. This offers KPI and other Analytical dashboards, etc. for use with your transactional data.

This could also be made use, of say you make use of SSIS to build a data mining database or several that come into a Datawarehouse. Well in turn the KPI, Dashboards and reporting inside PerformancePoint Services for SharePoint could be ran not just on the Dynamics AX parts of the information, but on all the data in your Datawarehouse for cross sections of information from all the systems you make use of.

There is a white paper on Microsoft's site, that is right at a year old. Actually one year old this month, to be exact. Even though this is a year old white paper, this does a great job of giving even more details into what I am talking about here. The direct link to that whitepaper can be found here: Microsoft Dynamics AX - BI White Paper (2008)

The only thing you need to really make note of in this, is that PerformancePoint Services is now part of SharePoint. That knowledge was not known yet when this white paper was wrote.

Also feel free to take a look at the BI home page of Microsoft: Microsoft BI Home page.

With the knowledge that you have gained, or can gain from these several post that have been done, you can start to really understand BI from Microsoft's point of view. The solution, how they are used and built from SQL through Dynamcis AX.

One other thing that can come into play is Office. I did not mention that, but Excel is a favorite of everyone. That very much can play into your BI solution, where Excel can be the front end for deliverying BI reports and Analytical information to your information workers.

Well check back soon. More great post are coming out. If you have further questions about BI, getting BI working for you at the SQL or Dynamics AX level and want to get into more details. Feel free to contact the experts at Sunrise Technologies, Inc..

See you soon!




"Visit the Dynamics AX Community Page today!"