Big Data 2014: Powering Up the Curve

December 5, 2013

Last year, I predicted that 2013 would be the year big data analytics started to go into mainstream deployment and the research we recently commissioned with Enterprise Management Consultants indicates that’s happened. What really surprised me though is the extent to which the demand for data blending has powered up the curve and I believe this trend will accelerate big data growth in 2014.

Prediction one: The big data ‘power curve’ in 2014 will be shaped by business users’ demand for data blending
Customers like Andrew Robbins of Paytronix and Andrea Dommers-Nilgen of TravelTainment, who recently spoke about their Pentaho projects at events in NY and London, both come from the business side and are achieving specific goals for their companies by blending big and relational data. Business users like these are getting inspired by the potential to tap into blended data to gain new insights from a 360 degree customer view, including the ability to analyze customer behavior patterns and predict the likelihood that customers will take advantage of targeted offers.

Prediction two: big data needs to play well with others!
Historically, big data projects have largely sat in the IT departments because of the technical skills needed and the growing and bewildering array of technologies that can be combined to build reference architectures. Customers must choose from the various commercial and open source technologies including Hadoop distributions, NoSQL databases, high-speed databases, analytics platforms and many other tools and plug-ins. But they also need to consider existing infrastructure including relational data and data warehouses and how they’ll fit into the picture.

The plus side of all this choice and diversity is that after decades of tyranny and ‘lock-in’ imposed by enterprise software vendors, in 2014, even greater buying power will shift to customers. But there are also challenges. It can be cumbersome to manage this heterogeneous data environment involved with big data analytics. It also means that IT will be looking for Big Data tools to help deploy and manage these complex emerging reference architectures, and to simplify them.  It will be incumbent on the Big Data technology vendors to play well with each other and work towards compatibility. After all, it’s the ability to access and manage information from multiple sources that will add value to big data analytics.

Prediction three: you will see even more rapid innovation from the big data open source community
New open source projects like Hadoop 2.0 and YARN, as the next generation Hadoop resource manager, will make the Hadoop infrastructure more interactive. New open source projects like STORM, a streaming communications protocol, will enable more real-time, on-demand blending of information in the big data ecosystem.

Since we announced the industry’s first native Hadoop connectors in 2010, we’ve been on a mission to make the transition to big data architectures easier and less risky in the context of this expanding ecosystem. In 2013 we made some massive breakthroughs towards this, starting with our most fundamental resource, the adaptive big data layer. This enables IT departments to feel smarter, safer and more confident about their reference architectures and open up big data solutions to people in the business, whether they be data scientists, data analysts, marketing operations analysts or line of business managers.

Prediction four: you can’t prepare for tomorrow with yesterday’s tools
We’re continuing to refine our platform to support the future of analytics. In 2014, we’ll release new functionality, upgrades and plug-ins to make it even easier and faster to move, blend and analyze relational and big data sources. We’re planning to improve the capabilities of the adaptive data layer and make it more secure and easy for customers to manage data flow. On the analytics side, we’re working to simplify data discovery on the fly for all business users and make it easier to find patterns and catch anomalies. In Pentaho Labs, we’ll continue to work with early adopters to cook up new technologies to bring things like predictive, machine data and real-time analytics into mainstream production.

As people in the business continue to see what’s possible with blended big data, I believe we’re going to witness some really exciting breakthroughs and results. I hope you’re as excited as I am about 2014!

Quentin Gallivan, CEO, Pentaho

Big-Data-2014-Predictions-Blog-Graphic


The Diary of a Construction Manger in Love with His BI Tool

June 7, 2012

Hi, my name is Bob and I am a construction manager. I oversee all aspects of managing the operations of a construction project, including budgets, staffing, and the compliance of the entire construction project.

In 10+ years of my experience, I have never had a Business Intelligence (BI) tool. I had to create spreadsheets to track daily activities, calculate risks and build formulas to measure impact. Given the size of the projects I worked on, this was extremely complex.  As a result, I would spend a lot of my time putting out fires to problems that I knew could have been prevented if I had the right information.

Recently my company introduced BI to our team. Since I’m using BI for the first time, I decided to create an activity log similar to a diary of my project.

Let me share some highlights with you:

October 28, 2011

We are 4 weeks into the project. We have the crew working on the ground. The foundation is done. The structural engineer has finished his design. We are ready to roll.

January 11, 2012

This morning I received an alert about my Preventative vs. Corrective Maintenance. My monthly work mix by type looks like this: preventative 36%; repair 24%; rebuild 5%; and modify 35%. My preventative costs have gone down from an optimal 40% to 36% and my repair costs have increased correspondingly.

When I drilled down into the repairs, I see that we are responding to higher than normal number of heating and insulation work items. I am going to talk to Edward – my HVAC contractor – about it.

February 29, 2012

I have been monitoring our electrical work. Our average Cost per SQ Foot is 13% less than industry average. This is a breakthrough thanks to the changes I have made monitoring the project with BI and making data-driven decisions. It lets me monitor these costs on an ongoing basis, so I can take preventative actions to stay below industry average to protect our funding and even justify additional headcount.

March 16, 2012

Productivity Rate is one of my favorite indicators – because it truly provides me with real-time info about the performance of my team. On average, our productivity rate stays on optimal levels. However, the plumbing trade group’s actual cost is exceeding the estimated costs. This will affect my cost-to-complete and margins, as I have to pay overtime for this contractor.

But I don’t have to worry… my BI tool lets me drill into this indicator to see whether the reason is ‘labor’ or ‘supply’ related. Drill-thru was something a spreadsheet could never let me do.

March 30, 2012

Two weeks have passed since I shifted resources for plumbing. Our productivity rates have improved since then and the project is looking on time and on budget.

With 40 more days to go, I want to make sure we deliver on time and meet our SLA with the building owners. I see no bottlenecks. Cycle Times – the average time to complete an activity – shows me that we are actually 4 days ahead of the schedule.

May 21, 2012

I’m very happy to report that we are done with the construction. The ROI on this project was greater than we expected and my client is very happy. Next weekend is the Memorial Day weekend. I have the time and money I need to take a nice vacation with my wife and son.

-As told by Bob, a fictional construction manager.

Even though the story is fictional, it’s based on reality. Business users and project managers – such as facility managers, supply chain logistic specialists, even dairy farmers – use Pentaho business intelligence to make their jobs easier and to make smarter, data driven decisions – just like our fictional friend, Bob.

Who knew BI could be so handy for construction managers?

What is your secret BI story? Drop me a line.

- Farnaz Erfan, Product Marketing, Pentaho


IT needs vs. Business needs

March 22, 2011

Can Business and IT finally live in harmony when it comes to BI?

This is not a new concept or question. In fact, for the last several years pretty much all BI vendors claimed that they have solved the “Business and IT Collaboration” needs. Or, at least their marketing departments did!

To truly solve a problem, we must first fully understand it. In this case it is important to ask questions such as: Why is there a lack of collaboration between these two groups? What is so drastically different about these two groups that have forced such a gap between them?

The truth is that IT needs a central ownership to information to streamline processes and ensure sustainability, while business users want their own self-service and ownership to gain results faster. After all business users have become a lot more analysis and data savvy these days as compared to the past; so, an old-school approach of letting IT do the work and just being the consumers of canned reports doesn’t cut it anymore.

Perhaps this picture illustrates the differences more clearly.

As you can see, these two groups are clearly in conflict when it comes to how they like to manage their information. So, we ask: What will help these two groups to start working in harmony?

The truth is that it won’t happen… unless there is a ‘balance’ between their needs.

As much as business users want quick time to value out of their BI projects, one-off applications are not sustainable overtime. They become monsters that are too hard to keep up-to-date, considering all the changes that happen to business requirements over time. Sooner or later, business users will need to reach to their IT friends for help.

The ‘balance’ lies in letting the business users get fast time to value, but still building applications that are sustainable to change. We define this ‘balance’ with an Agile BI approach:

  • Quick prototyping and visualization of the results
  • Frequent iterations and reviews between business and IT users to ‘get it right’
  • Once the data is ‘fit-for-purpose’, providing self-service tools for business users to be self-sufficient in building their own reports, analysis, and dashboards
  • Having a strong ‘shared’ metadata foundation across the board to adjust to changes quickly and to scale up with cumulative iterations

So back to our point about collaboration between business and IT: It is possible? Yes. Does it happen because a set of ‘tools’ facilitate this collaboration? Not necessarily, but they can help. What is the secret ingredient then to ensure such collaboration occurs? Simple: This collaboration happens as long as these two groups need each other, and are working towards a set of common / balanced goals for their BI projects. Something that is only possible with an Agile BI approach.

For more information about this topic and to explore how Pentaho has made Agile BI possible, attend our upcoming webinar on How to Fast Track Your BI Projects with Agile BI and see for yourself how Pentaho customers have come to reap the value of their BI projects with Pentaho’s Agile BI initiative.

Farnaz Erfan
Product Marketing Manager
Pentaho Corporation


Follow

Get every new post delivered to your Inbox.

Join 88 other followers