What's next?

Had a lot of fun in this course, as it was one of the most challenging ones, in which I really had to sit down and think like a business analyst. Even though I've worked as a business analyst before, this was pretty different, considering that I wasn't building a straightforward reporting system, but a business intelligence prototype, with a lot of annoying real life like issues in the data, and data structure.

The readings provided were also thought provoking, and gave a deep insight on how much effort has been put into this field, and how there is still so much more to do, since there hasn't been much innovation done other than marketing innovation, reselling the same technologies, or engineering innovation, improving speed of the servers.

Now that gives me a lot of motivation to research in this field, as I love innovation, thinking outside the box, and bringing new things to the table.

For me, the next step is to complete the BI specialization with HDs in the next semester, by taking FIT5094 IT for management decision making and FIT5095 Data warehousing.

And yes, I have to beat or at least equal Sindy's 98% in Data warehousing!!! :D

After that I'll apply for the DSS Lab next year, and get me some precious experience. :)

Overall, I'm pretty set on my career path now, means I can fine tune my goals now.

Other than that, vacations are upon us! Time to make some goals for this long wonderful break. I've already printed out my unit guides for the next semester, and will be going to the library tomorrow to borrow the relevant books, especially the following:

• The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling - Ralph Kimball
• The Data Warehouse Lifecycle Toolkit - Ralph Kimball
• Now You See It: Simple Visualization Techniques for Quantitative Analysis  - Stephen Few
• Show Me the Numbers: Designing Tables and Graphs to Enlighten - Stephen Few

Thanks Steve and Sindy for a great semester, and all the knowledge you guys have given me. I really feel like I've grown to a new level as an analyst. :)

Now its time to enjoy the holidays, and learn new things!!

Week 12 Notes

Rise of Business Analytics

Business Analytics is the very similar to BI, and the technology has already been available to us for quite some time.

The difference is that BI provides more of a rear-view mirror approach, focusing on the data, what has been done and what is currently being done

Business Analytics focuses more on what can be done, like predictions, forecasts, how the data can be used usefully.

Even though the technology was available the organizations were not mature enough to ustilize this, plus it used to be a specialized feature, being pretty expensive, however, now the prices are getting much cheaper.

BI from ERP Systems

ERP companies are basically buying out BI companys, and bundling the ERP and solutions together, which is actually not too bad considering that the user gets a cheaper solution off the shelf.

However, the problem is that the application are based on the ERP design standards and requirements, which are quite different from BI ones

ERP requires speed of data entry, update and delete functionality, however, BI requires quick and intuitive access to data, with complex calculations, and attractive reports from various data sources.

BI on your mobile

The issues are:

• lack of bandwidth
• Desktop UI was basically squeezed onto a phone screen, not so attractive and useful
• Most of the current usage is more towards operational levels than strategic.

Innovation

There is very little innovation, more like marketing innovation, reselling the same idea, and some engieering evolutions, making systems and servers faster, however, not much has been done on teh BI itself or the UI.

Overall BI is still profitable, even in an economic crisis, because of the perception of companies that it is an essential source of technology which allows them to do more with less.

Week 11 Notes

Lecture Objectives:

• Explain the different types of numbers used in BI application user interfaces.
• Correctly organize and display numbers into tables that suit how the numbers will be used.
• Correctly organize and display numbers into charts (graphs) that suit how the numbers will be used.
• Correctly organize tables and charts into reports and dashboard to effectively communication information to a business user.

Numbers

Can be divided mainly into three major groups:

• Quantities and categories
• Relationships within Categories
• Nominal
• Ordinal
• Interval
• Hierarchical
• Relationships between Categories
• Ranking
• Correlation
• Ratio
• Numbers that summarise
• Measure of Average
• Measures of Distribution
• Measures of money
• due to Inflation
• Currency conversion

Tables

Can be unidirectional or bidirectional.

Are used when we are presenting data in which need to look-up specific numbers, and there is more than a single unit of measure.

For good standardized table we follow the six Ehrenberg rules:

1. Round the numbers to two significant figures.
2. Use column and row averages to provide a focus point to the reader
3. Use columns to allows users to compare figures.
4. Order your columns and rows, either ascending or descending
5. You can use space to separate columns, to increase space between things which are not being compared
6. In case the numbers are important use a table, otherwise if shape is needed then a chart

Charts

A chart is used the shape of the data is important, and we want to show relationships between the  is the message not the numbers themselves. For example when we are trying to show the trend line

A few design rules are:

• Avoid chart junk, by increasing the data–ink ratio.
• Use small multiples to highlight comparisons and assess change
• Use complex graphs to portray complex patterns
• Relate graph size to information content
• Use graphical forms that promote comparisons
• Integrate graphs and text
• Know the audience

Pie Charts

Should be avoided because:

1. Humans are not good at comparing areas and angles.

2. The segment and clock position are perceived differently, causing misinterpretation.

Reports and Dashboards

Reports are meant to be more static and less interactive, whereas Dashboards are dynamic and meant to be very interactive.

Standard headers, footers, fonts, number and currency formats should be used for reports.

Dashboards are more focused on providing monitoring and measuring information to the user at a glance, hence, we need to take take advantage of pre-attentive processing, to allow better intuitiveness.

Week 10 Notes

Lecture Objectives

• Explain the role and importance of the user interface in BI applications
• Explain impact individual user difference can have on the effectiveness of a BI application
• Explain key principles of good interface usability and data visualization.

Intro

We have to understand that vendors try to sell very weird ideas, like UIs need to be very standardized so that the users don't need to "think" at all. This is so wrong, since we can never fully contemplate what the user wants, hence, the requirements need to come from the user through information gathering techniques like CSF / BSF.

Growing Problems
The problems that the industry is facing are:

• more licenses have been sold that active users, meaning the BI apps are not being used
• Excel and Access which gives users another way to access the data, bypassing the BI system
• BI Analysts need to understand that they have to focus more on the UI, than just the technical back-end, like data modelling and integration.

The Nature of DSSBI systems are meant to be small, personalised focused system, which are able to perform urgent tasks quickly, at the speed of thought.

BI's integral theme is evolutionary, where the user uses the system, they gain knowledge and  understand it better, and along the way their perspective changes, and then the system needs to adapt otherwise it becomes irrelevant to the business is dropped.

Implications on a BI Analyst

• they need to understand the decision making process and also how executive think
• need powerful and flexible tools that allow us to quickly and easily make changes to the system based on user requests.
• need a good infrastructure, both data warehouse and also very skilled and experienced staff members to efficiently work with the system.
• need to win the trust of the users, in order to make them trust the data and the system itself.Decision makers will not use the system if they can't trust it. Trust means that the users can easily verify the output that they see.

Requirements of a good UI

• Speed of interaction - user should be able perform operations at the speed of though, and not click and wait.
• Interface overload - the UI should not be getting in the way of the analysis, and hence should be kept to a minimum.
• Ease and convenience - users should not need to use manuals, the usage should be intuitive to allow users to quickly adapt.
• Memorability - users should be able to recall easily how to use the system after being away from the system for sometime.

Week 6 notes

Thompson Diagrams:

Used for logical modelling of multi-dimensional structures in an OLAP system

Basics:

• Each dimension is represented by a vertical line
• Each dimension is described independently
• Every member of a dimension is represented by a unit interval on the line
• A multi-dimensional model is built by combining the ‘lines’ for the involved dimensions.

However, it's very difficult to show multiple hierarchies, which is why we use ADAPT diagrams.

ADAPT Diagrams:

ADAPT = Application Design for Analytical Processing Technologies

As discussed, it allows for multiple hierarchies, explicit modelling of attributes and also some complex calculations.

Very good from technical perspective, but maybe too complex and technical for business people, who may prefer Thompson's diagram.

COGNOS 24 Ways Approach:

This was pretty useful for me in the assignment, in sorting out which functional areas I can include, plus what kind of business process and KPIs do those functional areas follow.

The main assumption by Cognos is that organisations have a lot in common, and many dimensions are shared across functional and application boundaries. Given that data is available and is of good quality, attention more towards the business that the technical IT side.

Week 5 Notes

Main OLAP rules discussed:

OLAP Rule 1: Multidimensional Conceptual View

Multi-dimensional data models enable more straightforward and intuitive manipulation of data.

OLAP Rule 6: Generic Dimensionality

Every data dimension should be equivalent in its structure and operational capabilities

OLAP Rule 9: Unrestricted Cross-Dimensional Operations

Computational facilities must allow calculation and data manipulation across any number of data dimensions, and must not restrict any relationship between data cells.

OLAP Rule 10:Intuitive Data Manipulation

Data manipulation inherent in the consolidation path, such as drilling down or zooming out (i.e. drilling up), should be accomplished via direct action on the analytical model's cells, and not require use of a menu or multiple trips across the user interface.

Design Contraints:

There are two major constraint when designing your multi-dimensional database::

Business - the report requirements, and the representation of those reports, and moving from a spreadsheet to a multidimensional view

Data - the format, quality, source, and availability of data.

Important Definitions: (Microsoft BI: Making better decisions faster)

business intelligence (BI) An approach to management that allows an organization to define what information is useful and relevant to its corporate decision making. Business intelligence is a multifaceted concept that empowers organizations to make better decisions faster, convert data into information, and use a rational approach to management.

Dimensions: 

A dimension is a structural attribute of a cube that is a list of members, all of which are of a similar type in the user's perception of the data.

Measures: 

Measures are the numeric values that can be aggregated to give meaning to your dimensions.

calculated measure A measure that is calculated or derived from a combination of base measures.

Grain: 

The lowest level of drill-down is called 'grain', and is usually at the transaction level.

cube A multidimensional data structure that represents the intersections of each unique combination of dimensions. At each intersection there is a cell that contains a data value.

database A collection of related data that is organized in a useful manner for easy retrieval. There are different applications of databases depending on the type of data to be stored and how the data is to be used.


data warehouse A repository for data. Many experts define the data warehouse as a centralized data store that feeds data into a series of subject-specific data stores—called data marts. Others accept a broader definition of the data warehouse as a collection of integrated data marts.


desktop online analytical processing (DOLAP) An OLAP storage mode that keeps data on a client's machine and provides local multidimensional analysis.



hierarchy The organization of levels within a dimension that (1) reflects how data is aggregated from detailed levels to summarized levels and (2) serves as the drill-down path for top-down business analysis.


ragged hierarchy A hierarchy that has an inconsistent number of drill-down levels.



hybrid online analytical processing (HOLAP) An OLAP tool that can store data in both multidimensional databases and relational databases.


multidimensional online analytical processing (MOLAP) An OLAP storage mode in which data is placed into special structures that are stored on a central server(s).



online analytical processing (OLAP) Multidimensional analysis that is supported by interface tools and database structures that allow instantaneous access and easy user manipulation. Online analytical processing got its name because this name contrasts well with OLTP, a term that was already in widespread use when the term OLAP was created. There are fundamental differences between transaction processing and analytical processing. OLAP systems support multidimensional analysis at the speed of thought. OLAP typically follows the client/server paradigm, where an OLAP database server is accessed by many users who use multidimensional client tools to analyze data.


online transaction processing (OLTP) A data processing system designed to record all the business transactions of an organization as they occur. OLTP systems are structured for the purposes of running the day-to-day raw data of business, which requires efficiency and minute processing of transactions at the lowest level of detail. An OLTP system processes a transaction, performs all the elements of the transaction in real time, and processes many transactions on a continuous basis. OLTP systems usually offer little or no analytical capabilities.

relational online analytical processing (ROLAP) An OLAP storage mode where data is stored in relational databases.

uniform aggregation A method of summarizing data from its lowest level of detail to its highest level of detail, where data can be aggregated the same way across all dimensions.



semiadditive aggregation A method of summarizing data from its lowest level of detail to its highest level of detail in which measures are not aggregated uniformly across all dimensions.



slice and dice Two complementary methods for interacting with data. Slicing means isolating a specific member of a dimension for analysis. Dicing means breaking a data set into smaller pieces by examining how measures intersect with multiple dimensions.


***Memorize all of these!

Week 3 Notes

Lecture Objectives:

• Understand the key issues related to five case studies of Australia BI system.
• Be able to identify some of the key factors that can lead to the failure or the success of a BI system.

Keen's Adaptive Development Model

Overall Key Issues and Conclusions of Cases

• Cost/Benefit Analysis - this needs to be done for the BI system that is going to be implement to see the feasibility
• Evolution - the development strategy needs to be evolutionary, inline with the evolutionary application and function nature of BI applications.
• Data and Data Management - the data and the way it is managed is critical to the success of the BI system, and the infrastructure needs to be flexible.
• Profit Impact - the BI system can make a significant difference, however it depends on the organization how much profit the system can provide them. BI systems have very short spans of being relevant, after which the highly variant business/organizations change, and the BI system need to be adapted along the way.
• Cost - need to consider the transaction costs, more than the operational cost of the BI system, considering its not about fixing problems and bugs, but more about a living breathing system, which has both functional and application evolution going on, as per the demands of the users.
• Project Manager - needs to have experience in handling BI related projects, and also have very good technical, business, creative, and interpersonal skills, and should know how executives think and work.
• Development team - the team needs to be good in development/programming/tools in general, however, its very important for them to be experienced in the BI environment, plus they should have good business sense, and data organization skills.
• Politics  - need to be put aside for actual work to continue.
• Contracts - should be loosely set to allow evolutionary development
• Feedback - developers mainly complain that users don't provide feedback, which hinders BI application success
• Commitment from SMs - is a very important CSF
• Usage - the users are generally convinced about the benefits, however, there is still a lack of actual usage of the system due to lack of trust in the system.
• System and Data Ownership -  considered a very good practice to separate these two.
• Testing - generally developers avoid rigorous testing, which causes issues later on.

Week 8 Notes

Lecture Objectives:

• Be able to apply the basic concepts of organisational productivity measurement
• Understand the basic principle of Kaplan and Norton’s Balanced Scorecard

Productivity Measure

Productivity is a very common measure in BI terms, and it basically is a mathematical function of the outputs over the inputs. The production function is a graph which has all the variations of the inputs and the outputs for those inputs.

Law of diminishing returns

If one input increase the output is increase at a lower rate than the input, or in other words the others grow short relative to the input.

One thing to understand is that moving along the production curve, we are not being more productive, but in fact just getting a corresponding output for an input for the same level of productivity. The only way to become more productive is to shift to a new production function curve.

Production Indices

Using the production function we compare different production indexes over periods of time to see the changes in productivity.

However, keep in mind that without comparison the indices have no sense, in the same way there is no value in comparing across industries, the process has to be the same.

Balanced Scorecard

The aim is to focus on the organization as a whole, because each functions within the organisation has an effect on each other.

BS improves the communication and alignment between functions of the business. It looks at the organisation as whole.

There are four main and original perspective in the BS approach: 

• Financial - to succeed financially, how should we appear to our stakeholders?
• Objectives - write down the objectives to measure financial success
• Measure - the KPIs or target values
• Target - the goals to reach
• Initiatives - what are we going to do in order to reach our goals?
• Customer - To achieve our vision, how should we appear to our customers?
• Core Measures:
• Customer acquisition
• Market share
• Customer retention
• Customer satisfaction
• Internal - To satisfy our shareholders and customers, what business processes should we excel at?
• Learning & Growth - To achieve our vision, how will we sustain our ability to change and improve?

Cause and Effect Relationship

This is very important as here we link the four perspectives, and see how they influence each other. This ensures that we can view the effect on the organization as a whole, allowing SM to see the overall business performance of the oragnizations, and down the hierarchy.

Week 7 Notes

Learning Objectives:

• Be able to use the critical success factor method to gather requirements for BI systems.
• Be able to use the strategic business functions method to gather requirements for BI systems.
• Be able to apply concepts from management science, statistics and TQM to the design of BI systems

Critical Success Factors (CSFs)

Mainly focuses on the perception of the individual. It gives the the executives the control over the processes and define their information needs.The approach if useful where you have several analysts trying to gather information from various managers at the same time. It is a structures process which assures that everyone gathers information is the same way.

The analyst basically takes the interview approach asking the executive about their role, mission, and goals.

Based on the information CSFs are developed, prioritized, and then KPIs are developed for the corresponding CSFs

There can be three types of CSF:

• related to the organisation
• related to the manager's functional area
• relate to the manager as an individual.

However, all of these are focused on the manager's personal view.

Some problems with CSF method.

1. Inter-functional conflicts.

2. CSFs change over time. They could change as the business changes, as the individual changes or when the manager is replaced.

3. Managers might hide the actual information, if they don't trust you or have agendas.
4. Cannot analyze the "why" scenario

5. No cross-functional analysis

Strategic Business Functions (SBF) 

Overcomes the limitations of CSF.

Aims at aligning with the organization as a whole, instead of focusing on individuals.

Managers are gathered in a workshop environment (helps solve conflicts there and then) where you discuss their activities, what functions are critical for them for being competitive, then find linkages between the functions.

Steps:

1.BI system team performs business analysis in partnership with senior executives

2.Strategic business functions are defined

3.SBF’s are prioritised

4.Information linkages are determined

5.Information required to support each function is defined

6.A plan is developed for prioritised implementation of the BI system by SBF


KPIs

KPIs are the measures, so basically need to be measurable. One KPI cannot be a complete measure of a CSF, but in fact a CSF has multiple KPIs.


Objectives

Objectives are the direction the organisation needs to move in order to improve.

Objectives help create CSF and SBF; they are used to implement KPIs.

Objectives can have sub objectives also.

For many objectives, there are no measurable targets, for which we use the following 3 approaches:

• Subjective Index - ask an expert in the field to come up with a rating system on a scale to help monitor an objective
• Proxy target - this is an indirect measure, and can be subdivided into indirect, precursor, and  combined.
• Direct Preference Assessment - ask the decision maker to make a subjective judgement for each objective

Total Quality Management

This is a philosophy dealing with enhancement of business process by involving employees to collectively analyze their own performances.

The main graphical tools used are:

• Histograms
• Pareto charts
• Run charts
• Shewhart charts

Week 4 Notes

Lecture Objectives

•  Be able to explain the Monash EIS Method and how it can be used to guide the development of business intelligence applications.

The three main areas in Monash Method are:

1. System planning

2. System Delivery

3. System Use

1. System planning

Initiation

This sits outside the development, and occurs mainly due to some issue or problem, and occurs only once.

• High level objectives are set.
• Executive champion is chosen
• BI application objectives are established
• Initial application is chosen based on whats important and doable

Strategy and resourcing

• Development strategy is chosen, which should preferably be evolutionary.
• Project Manager is chosen, who needs to be jack of all trades, both business and technology.
• Development team is chosen, preferably with BI experience.
• Hardware and Software are chosen, and new tools are learnt.
• Can discuss the technology required surround Business Intelligence (Week 9)

High-level design

• Application requirements are prioritized
• Information required is identified, along with the sources
• General design standards are chose, like the look and feel of the UI
• BI App development plan is developed for the next few iterations
• Can include Dimensional modelling - conceptual (Week 5/6)
• CSFs/BSFs can be used at this stage (Week 7)
• Balanced Scorecard is applicable here also (Week 8)

2. System Delivery

Detailed Design

• EIS capabilites are designed
• Interface and Data Structure is designed
• Can include Dimensional modelling - logical and physical (Week 5/6)
• Designing of UI (Week 10/11) including tables and charts

Construction

• Programming functionality
• Constructing the EIS databases
• Linking the data to the front end
• Designing of UI (Week 10/11) including tables and charts

Testing

• Traditional type of rigorous testing is done
• Also include UI testing, surveys, etc.
• Ensuring that requirements have been met
• Comparison of new reports with old reports

3. System Use

Training

• Training can be done one-on-one, but needs to suits the type of user, e.g. different for executives
• Avoid bloat ware
• Evolutionary training is also important, where it is decided if new applications and reports are developed, whether we need to decide if new training is required.

Operation

• Monitoring the attitudes of users towards the EIS and the usage patterns, to see which reports are being generated and which screens are not being used and need to removed, to avoid bloat ware.

Maintenance and Support

• This involves the function evolution i.e. the user will use the system and will ask for changes to be implemented, and keep the system up-to-date with the business requirement.
• Also involves application evolution, where other departments see the benefits and also ask for something similar, which means new applications need to be designed.

Week 2 Notes

Lecture Objectives

• Understand the nature of the work of an executives and what effect that has on the design of business intelligence applications.
• Be familiar with the results of surveys on the use of business intelligence applications and understand what these results mean to the design of business intelligence applications. 

How do Executives Work?

According to Mintzberg, the nature of managers is described by three words which are:

• Fragmentation - on only 10% of the tasks SM can focus for more than an hour on significant level of  data. 90% of the time they do tasks with less than 1hr duration with significantly less data, because they get interrupted a lot.
• Brevity - SM have a constant time constraint, it is very difficult to pin a SM down, at any time, for any duration. They can be thought of as a moving target
• Variety - SM do many different things, which can be planned/unplanned during the course of their work day, and primarily are unplanned

Furthermore, Mintzberg divided the SM's role into 10 generic roles, which need to be integrated and cannot work in isolation:

• Interpersonal
• Figurehead - presidential type of role
• Leader - the head, who inspires people below them to inspire the people below them.
• Liaison - key connection point b/w organizations and external organizations. Sets the direction of negotiations, represents in lobbies, and has political power to get things done.
• Information processing
• Monitor - in snapshot it seems CEO doesn't know anything, but they ask questions from relevant people, instead of checking dashboards and reports. They file the small information snippets in their brain.
• Disseminator - disseminate information down, this happens through the hierarchy, top to bottom, between departments, even in different geographical locations
• Spokesman - communicates with the board and public about the performance of the company
• Decision making
• Entrepreneur - what the business is going to do? set the new direction, put their stamp on the business. new CEO, kill off all old projects, and get new projects on line, sets new direction, no prodigies from previous management.
• Disturbance Handler - settles the difference in opinion between functions
• Resource Allocator - budget is not infinite, set the priority for the resources
• Negotiator - external role, contract negotiations, like acquisitions are about to occur, set the limits e.g. buying a contract

Regarding BI systems Mintzberg said that its a waste of money on EIS systems, because these senior managers don't use them. He re-did his research in 90's to check if work of Managers had changed after economic crashes and changed organizations, and hardly anything had changed, everything mostly the same and hold today even.

Kotter on the other hand criticized Mintzberg, that what if he had just picked a bad sample, or maybe the sample was too small. Kotter said he'll study more managers and only check managers from successful companies with certain success experience, in their career and their role, and peers say that they're good.

In results, Kotter had mostly the same findings, including how SM get snippets of information through the networks that they form, and in addition he said that SM had "Agendas" - the agendas that they want to get done, and sometimes its not what the company publicly states what they want done - some of it is informal and not written down.

Regarding usage of BI systems Kotter didn't agree that BI system costs are a waste, because even if the SM doesn't use it the manager or analysts from whom the SM gets summarized information do use them - so management systems are actually useful to SM indirectly.

According to Isenberg, Managers design the organization processes, the reporting relationships, the structure, in other words, is the architect. SM do problem management, but they don't work on all problems at the same time. They tend to pick a pet project, whatever interests them, not all the problems in the organization, but in parallel they keep finger on the pulse of other problems also. Basically "what can i resolve easily and quickly", because sometimes big problems, tend to resolve themselves.

SM seem to know, what to do intuitively in "unconscious competence mode".

Survey Results of EIS Usage


Survey 1

• Ideas were initiated at the manager level, and mostly by the IT function - which is not a good sign because of their lack of business sense
• No specific development techniques were used
• CSFs were listed at the outset of the development
• Surprisingly there was no indication of information politics - might be because the EIS was already mature
• Little evidence of technophobia

Survey 2

• Survey was conducted mostly on large organisations
• Most of the platforms were custom built
• Training was in-house and light
• Mainly the system was used for accessing, assessing information and planning
• User were generally satisfied with the features and the system as a whole
• Ex-user showed the same good results, and gave the reply that their organizations had changed structure, but the EIS was not updated, rendering it irrelevant, and hence, was abandoned.
• Non-users are excited to start using EIS

Survey 3

• SM are and executives are very experienced with BI
• BI systems mainly used in large organizations, mostly for decision making, resource allocation and identifying problems
• business decision makers classify themselves as ‘analytical decision makers’, but the predictive analysis / what-if functions of BI systems is not often used;
• Systems have a very short life span, and need to be kept up-to-date
• Overall BI Systems are seen as important, and are being installed, and planned, meaning that the market is still growing

First Tut! It's all starting to make sense now!

Ah this semester is so busy!

I’m running very late on my blogging so have to keep this short, and as concise as possible, and catch up!

After an exciting intro to the world of Business Intelligence in Steve’s first lecture, Sindy's tutorial was amazingly very useful in tying everything together, and clarifying what exactly BI is, and how it’s connected to other technologies.

The image below illustrates what we learnt and discussed, and for people like me, who haven’t taken FIT5095 – Data Warehousing, there were a lot of new terms introduced. :)

The story is that organizations have multiple departments (e.g. Sales, Finance, Marketing, etc.) and hence, multiple data sources, i.e. their transactional systems (e.g. ERPs, CRMs, etc.). If these systems are in silos, not sharing information, there are multiple issues arising such as data redundancy, varying and inconsistent data formats, etc.

To solve this, in layman's terms, we go for integration!

Basically, a centrally managed and integrated database containing data from all transactional sources in an organization, in other words, the famous -> Data Warehouse - a dedicated database which contains detailed, stable, non-volatile and consistent data which can be analyzed in the time variant.

Now to source this data, we use the ETL process. Another new term…yay!

• Extracting data from transactional systems
• Transforming it to fit operational needs, including the transformation from E-R model to dimensional model (discussed in detail later on below)
• Loading it into the end target, i.e. the data warehouse

So now we have multiple goodies that come out, such as the OLAP Analysis and Data Mining. Regarding Data Mining, which I’ll study at the end of next year, for now, all I need to know is that it’s a study of drilling down patterns, trends, and behaviors.

In this course, we’ll be focusing on the OLAP / presentation layer…hmmm so what is OLAP???

OLAP, On-Line Analytical Processing, is a class of technology used to build shared, data-oriented decision support systems, in which we can slice and dice data. In Steve’s words, they are often talked about as “pivot tables on steroids”.
In essence, the format of data is such that decisions can be made at the speed of thought! (love this statement) :D

OLTP (On-Line Transactional Processing) Systems are built to optimize transactional processing speeds, hence, the E-R model is utilized. However, OLAPs are designed to get an overview analysis of whats happening.

This is done through dimensional modelling, i.e. the star-schema, where there is a Fact Table (numeric values that can be aggregated), and Dimension Tables (groups of hierarchies and descriptors that define the facts.

So what are the cubes then???

Well, OLAP cubes (not strictly cuboids - no wonder the term is controversial - it's not applicable if there are more than three dimensions), is the name given to the process of linking data from the different dimensions. The cubes can be developed along business units such as sales or marketing. Or a giant cube can be formed with all the dimensions.

This blog seems more like an reiteration of the tutorial than a reflection, but hey, it means I was listening carefully to each word. :D

Note-to-self:

- The process of analyzing your data for making informed decisions using data from the OLTPs, and the data that results, are collectively called business intelligence.

- An OLAP database is a specialized database designed to help you extract business intelligence information from your data.

Resources:

• http://thomaslarock.com/2011/02/sql-university-data-warehousing-week/
• http://www.dwreview.com/OLAP/Introduction_OLAP.html
• http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com.ibm.db2.doc.admin/dstarjn.htm
• http://msdn.microsoft.com/en-us/library/aa140038(v=office.10).aspx