Hadoop & Co.: Trial and error welcome

The digital transformation is underway. The buzzword digitization conceals opportunities, but also challenges for IT. It must be able to meet the expectations placed on it by the business, and do so as ad hoc as possible.

360-degree analyses, forecasts, buyer behavior or market trends are needed several times a day in order to react quickly and make the right decisions in time.

If IT staff are not familiar with new technologies such as Hadoop, "R", Python or graph-oriented databases, they and the business often miss out on great opportunities.

A survey by the digital association Bitkom shows that a large number of companies are not yet prepared for big data. Among other things, technical know-how and technical equipment are lacking for Big Data readiness.

IT should ensure that it is up to date with the latest knowledge and deals with the topics of Big Data and digitization before the business departments approach it with specific requests.

Big Data for IT

This can be achieved if those responsible for IT are aware of the technical possibilities and also use them for themselves, keyword: Big Data for IT.

Business analytics has long been used as a standard tool in business departments, but IT itself has rarely used analytics tools to optimize system operations.

It is advisable to use scenarios to examine the methodologies, technologies and tools that corporate IT already has and masters.

An example of Big Data for IT is the creation of a simple data growth forecast.

The first step is to determine: Where am I today, what data is there, where is it stored and how fast is it growing?

With an "R" server and some freely available algorithms, this can be used to create a much more accurate forecast than Excel.

Another option is to analyze user behavior in the system: Which users use which transactions and which reports how often? Do they use them at certain times, for example for the monthly or quarterly closing?

Here, technologies such as Hadoop for mass data storage, such as "R" or Python can be used as programming and statistical languages.

The IT department should first generate initial added value internally using these technologies in order to define and test methodologies and form its own opinion.

Through these analyses, IT managers can gain valuable insights to optimize system operation. For example, which parts of the system or even which data are really used and how the data volume relates to usage.

Performance bottlenecks can also be analyzed. This knowledge can also be used as a basis for archiving, for optimizing processes, and for implementing Big Data scenarios or Industry 4.0 applications.

Randstad Germany uses this intelligence with the help of the Datavard HeatMap analysis tool. The tool performs a usage analysis of data and objects in the running system.

"Datavard HeatMap is unique. We get significantly more knowledge about the system without having to interview a single end user. With this intelligence, the most important "pain points" of system operation can be elucidated and resolved"

Look into the future

Advances in machine learning and predictive analytics are becoming increasingly important. Machine learning is the artificial generation of knowledge from experience. The system learns from examples from the past.

Algorithms specializing in pattern recognition derive predictions for the future from this data (predictive analytics). Log files from SAP systems, networks and Hadoop clusters are used for this purpose.

What influence the interconnected machines have on each other can be programmed afterwards, or a machine can recognize these influences and expand them on its own.

The machine learns to generate added value.

At the same time, cloud environments for Big Data are becoming increasingly popular. Cloudera, for example, is a completely secure platform: everything from data transmission to data storage is encrypted.

On this basis, a cloud-based user analysis can be created for SAP and Hadoop. This shows how users use the data in the SAP Hadoop environment, which reports and data they access and when, and how they combine data to gain new insights.

New technologies put to the test

IT specialists should learn as much as possible about new technologies and the opportunities they offer, and apply them.

Cloud providers such as AWS Amazon Web Services or Azure make it easier to get started. Hadoop clusters can be set up there and "R" servers set up to get to know the technologies first.

Datavard offers innovation workshops where use cases, digital processes and business models are defined together.

Participants then create an initial prototype using the latest technologies. This allows IT managers to learn how to use new technologies while testing out ideas and business models.

Only when the technical possibilities have been explored and also tried out is it advisable to get those responsible from different areas of the company around the table to find new business models.

IT as a business enabler

Practical experience from internal Big Data projects pays off when it comes to developing new business processes.

IT managers can use their knowledge advantage to act as business enablers by actively approaching the business departments and making suggestions for the implementation of new business processes.

The most promising approach is for a heterogeneous group from different departments to meet for a workshop to explore the possibilities of Big Data for the company.

Knowledge of design thinking approaches or the Osterwalder Business Canvas can be helpful in this process. It is also advisable to look at use cases from other companies and draw inspiration from them.

Here, an external consultant can be helpful who brings know-how from Big Data projects that have already been implemented.

Often underestimated in companies is the fact that, due to the intensive use of smartphones and tablets, it is now a matter of course for most employees to get answers to their questions ad hoc and in an appealing visualization.

Big Data has already arrived in their lives and they want to reap the benefits in their day-to-day work as well.

The 2016 Future Workforce Study, conducted jointly by Dell and Intel, found that for 21 percent of young Generation Y employees, technology is a deciding factor when choosing a job.

Big data technologies are becoming an important differentiator in the "war for talent".

Digital transformation and the use of Big Data analytics will play a greater role in companies in the future.

A survey conducted by Forbes Insights on behalf of Hitachi Data Systems found that the potential of data and analytics has been underutilized.

According to the study, investment in new digitization technologies is a top priority for 51 percent of companies.

The same applies to the expansion of capacities for data and analyses. The top drivers are new business models (41 percent) and new technologies (40 percent).

Only 44 percent of the companies surveyed consider themselves advanced or leaders in data and analytics. However, 91 percent have already achieved revenue increases through their use.

Big data readiness is increasingly becoming a success factor. However, it can only succeed if IT is familiar with the technical possibilities in practice and uses them for its own benefit.