[tta_listen_btn]
October 27 2016
For correct business decision-making it’s important to have detailed information and insights on the state of affairs in the company. But often such information is limited to the annual or quarterly reports.
To effectively analyze the organizational processes and progress, enterprises often implement business intelligence systems (BI-system) that enable them to make more informed decisions. Here I’m sharing a few tips that may help when you create a BI-system for your company.
Design
Store the raw data, rather than sections. Do not hope that the fixed schedules and reports will be sufficient. Instead, managers will ask you to build more and more detailed graphics. There will be lot of updates required.
For example, we received a request for how much money singles in California under the age of 25 years spent for each month of the previous year. To be prepared to answer this question, we need to have not only their complete profile but also their financial reports.
Analyze the “raw” data, not ready slices
Try to analyze the “raw” data. Do not pre-aggregate. Remember: once you do the aggregation of data, you lose information.
For example, you need to obtain statistics on the number of new contacts per day for New Yorkers. If you do an analysis of the data itself directly, you can confirm these results with concrete examples: who, when and with whom.
Look for the right solutions that complement your custom BI system.
Remember that you are not the first one to create a BI-system. And for many problems ready-made solutions already exist. Because of this factor, most of the development can be reduced simply to a data collection and analysis software configuration.
Collect data asynchronously
If you want to start collecting data about user behavior, do so asynchronously. You can write in the logs, you can write to the Scribe.
In the development of infrastructure for the collection of information about the behavior of our customers, we know about the large volume of data to be processed. All this data was to be collected in a single data repository.
Of course, any problems in the work of the store should be invisible to users of the site. Therefore, it was decided to record the raw data in log files, and only then move them to a separate storage back-end script.
Check the data collected
With the implementation of BI-systems incoming data should be checked, and you need to do this very carefully. For example, if you get the parameters of the system users, be sure to check the distribution of registration dates, birthdays, etc. Ideally you should check the distribution of values in each column or columns in pairs.
Often, when new data is added there is a situation when the column value in all rows is the same. Almost always the cause is the human error – the developers simply forget about this column.
Superfluous Data do not exist, there are repetition
When you look at what data is to be imported into the system, remember that extra data is not there. There are these repetitions. And now to the repetitions should be treated with suspicion. It is better to take the additional data and make sure that you have the same values than previously refused repetitions.
This helps to identify errors in the system. Our Team at PureLogics has fixed a lot of errors like this in BI-system implementation. These were mistakes in user profiles, and errors in the data on the cities, and even a few errors in the financial data.
Do not strive for 100% compliance
By comparing and contrasting data from different sources, do not chase for 100% satisfaction. If you reach the 95% match, it is good enough.
Very often the discrepancy of data is caused by objective reasons, for example, out of sync time. For example, the registration time of payment in their own “billing” and in the payment system. The difference in time of 1 second on December 31 may lead to the fact that one and the same payment will be dated in different years.
These tips are not universal, each of them can find an exception. They should not be taken as absolute truth. On the contrary, the more counter-examples you can give, the better you will understand the essence of these rules. And if you have specific questions, we will try to answer them.
