IT needs vs. Business needs

March 22, 2011

Can Business and IT finally live in harmony when it comes to BI?

This is not a new concept or question. In fact, for the last several years pretty much all BI vendors claimed that they have solved the “Business and IT Collaboration” needs. Or, at least their marketing departments did!

To truly solve a problem, we must first fully understand it. In this case it is important to ask questions such as: Why is there a lack of collaboration between these two groups? What is so drastically different about these two groups that have forced such a gap between them?

The truth is that IT needs a central ownership to information to streamline processes and ensure sustainability, while business users want their own self-service and ownership to gain results faster. After all business users have become a lot more analysis and data savvy these days as compared to the past; so, an old-school approach of letting IT do the work and just being the consumers of canned reports doesn’t cut it anymore.

Perhaps this picture illustrates the differences more clearly.

As you can see, these two groups are clearly in conflict when it comes to how they like to manage their information. So, we ask: What will help these two groups to start working in harmony?

The truth is that it won’t happen… unless there is a ‘balance’ between their needs.

As much as business users want quick time to value out of their BI projects, one-off applications are not sustainable overtime. They become monsters that are too hard to keep up-to-date, considering all the changes that happen to business requirements over time. Sooner or later, business users will need to reach to their IT friends for help.

The ‘balance’ lies in letting the business users get fast time to value, but still building applications that are sustainable to change. We define this ‘balance’ with an Agile BI approach:

  • Quick prototyping and visualization of the results
  • Frequent iterations and reviews between business and IT users to ‘get it right’
  • Once the data is ‘fit-for-purpose’, providing self-service tools for business users to be self-sufficient in building their own reports, analysis, and dashboards
  • Having a strong ‘shared’ metadata foundation across the board to adjust to changes quickly and to scale up with cumulative iterations

So back to our point about collaboration between business and IT: It is possible? Yes. Does it happen because a set of ‘tools’ facilitate this collaboration? Not necessarily, but they can help. What is the secret ingredient then to ensure such collaboration occurs? Simple: This collaboration happens as long as these two groups need each other, and are working towards a set of common / balanced goals for their BI projects. Something that is only possible with an Agile BI approach.

For more information about this topic and to explore how Pentaho has made Agile BI possible, attend our upcoming webinar on How to Fast Track Your BI Projects with Agile BI and see for yourself how Pentaho customers have come to reap the value of their BI projects with Pentaho’s Agile BI initiative.

Farnaz Erfan
Product Marketing Manager
Pentaho Corporation


Q&A with Pentaho Trainer Lynn Yarbrough

March 14, 2011

Q&A is a series on the Business Intelligence from the Swamp Blog that interviews key members of the Pentaho team to learn more about their focus at Pentaho and outlook on the Business Intelligence industry.

If you have attended a Pentaho training class, most likely you have had a chance to meet the knowledgeable and entertaining Lynn Yarbrough. To get to know Lynn and our training classes better we asked Lynn 5 questions.

1.  What brought you to Pentaho and what do you do?

I am a trainer at Pentaho. After almost 20 years in the BI and Data Integration industry working at companies like Information Builders and Hyperion, I wanted to work at a smaller Business Intelligence company and I found a job posting for a training position at Pentaho on Monster.

2.  How many classes have you taught and do you have a favorite that stands out?

I have been with Pentaho almost 4 years and probably taught 40 classes last year alone.  My favorite class is the Pentaho BI Suite BootCamp.  I especially love the morning of Day 3, because that when we complete the creation of an OLAP cube and can view the data of Pentaho Analyzer in the Pentaho User console.  It is very rewarding to see the result of all of the work and often this is when students have an ‘aha’ moment, and say ‘this is why we need Pentaho, to give our users this power.’

3.  What elements do you think are necessary to make a successful training class?

Knowledge of the product, knowledge of the industry (in my case Business Intelligence) and a sense of humor.  We in training have also made many of the classes very ‘hands on’ which help students learn and retain the product. One of the problems we face in the classroom is that we are teaching students who have a variety of skill levels to make the class beneficial to all we have added  “optional” labs for the more advanced students.

4. Can you tell us more about the Agile BI for Business Analyst class and how it contrasts to the ‘early days’ of training

We began talking about and shaping the Agile BI class over 2 years ago, but it didn’t really take-off.  The product didn’t have the “ease of use” it has today. Now, Pentaho Agile BI has matured to the level where the class is engaging and quite fun to teach since is so easy to create a data model and analyze the data.

5.  As we kick off 2011, what are you most excited about regarding training for the upcoming year?

2011 will add many new key features to the product which will not only impact customers, but will make training easier, as the product improves in “ease of use” it also get easier and more fun to teach.  We also plan to add new training classes suggested by our customers such as ‘Pentaho for the End User’ giving hands-on training for our front end tools and ‘Pentaho On-Demand’ for customers interested to gain more experience with our cloud solution. Things are never boring at Pentaho.

Do you have additional questions for Lynn? Is there someone or a certain role at Pentaho you would like us to interview? Leave your questions in the comments section below. We’d love to hear from you.


How fast is lightening fast?

February 15, 2011

A huge congratulations to our partners at Ingres, who today announced that their lightening fast database VectorWise has set a new record for the Transaction Processing Performance Council’s TPC-H benchmark at scale factor 100. Not only did Vectorwise set a new standard, but it blew the previous record holder out of the water, delivering 340% of the previous record.

Equally outstanding about this news is the fact that VectorWise has not only changed the game in terms of performance, but the database also comes in at a fraction of the price of its competitors. Forward-thinking innovation, high performance, and low cost… sound familiar? It should.

What does this mean to Pentaho users?

Pentaho and Ingres established a partnership last October, with the goal to combine enterprise-class business intelligence with the speed and performance of the fastest analytical database on the market. With over 250,000 QphH (Queries per hour) for 100 GB of data, VectorWise is the epitome of agility at the database level. This means lightening fast query response times, more iterative cycles, and at essence, even more agile business intelligence.

For more


3 questions that every CFO wants to know about Pentaho (and how to answer them)

January 25, 2011

1. Will it do what I need?

2. Will it get done on time?

3. Will it fit my budget?

These are pretty much the 3 questions that any IT Manager will ask him/herself when it comes to evaluating and investing in any type of software. Open Source BI is no different. Call it an over-simplified RFI (Request for Information) or “make my CFO approve my gut feeling” checklist, these questions are essentially what you need to know to go forward with a purchasing decision.

The snippets below show how 4 Pentaho customers have answered these questions.

ZipRealty, a well known real estate firm, wanted to increase the quality and quantity of their reports to their users. Their previous BI solution was cost-prohibitive when it came to rolling out these reports to “all” their agents and district managers. ZipRealty’s Director of Business Intelligence, Salvatore Scalisi quotes: “It wasn’t economical to add the number of users we wanted. The licensing fees became substantial.”

With Pentaho, ZipRealty was able to help its agents and managers track inventory and conversion rates. They also enhanced their client services and satisfaction by rolling out analysis to their client base, allowing them see real estate listings along with embedded analysis and map-based mash ups directly from the ZipRealty’s website.

Estalea L.P., a technology incubator with strong Java programming expertise wanted to find a similar technology-base solution to prepare “data services” for its clients’ independent BI applications. In looking for a solution that was Java base, and did not require very deep technical skills, stating, “Pentaho was the only viable open source product, particularly in the ETL area.” Given that other Java ETL solutions are on a per user licensing model, Pentaho’s unlimited user license was a better option to them.

The simplicity and ease of use of Pentaho Data Integration, further enabled Estalea to save on costs by not hiring more expensive programmer staff. Jon Cotter, Director of Product Design and Technology at Estalea explains, “This way I can take on projects with people with less experience and move my senior people to other projects.”

For Loma Linda University Health Care, the value was mainly around user adoption. Loma Linda’s physicians, a non-technical group in the field of Business Intelligence and ETL, are now very interested to use Pentaho for their analysis. Historically, physicians are primarily Excel users. Their positive experience on the ease of use of Pentaho BI Suite Enterprise Edition is absolutely clear in in the quote by Darrin Blocker, Decision Support Analyst at Loma Linda, “Just the idea of being able to select attributes and pivot that data and get a response in seconds, versus having to send in a report request and waiting for a week was a big selling point”.

Power Costs, Inc.’s goal was to offer an analytical application to its utility customers that can help them with energy bidding process. Monthly gigabytes of data and a seven-year history of data from its customers, combined with Power Costs’ own market data was processed to build these sophisticated analysis. Pentaho Analysis fit the bill.

For the volumes of data that Power Costs needed to analyze, Pentaho Data Integration was well-integrated into BI stack. According to David Nilsson, Power Costs Vice President of Product Strategy, his analysis of Pentaho’s ETL tool was “ One of the better ones that we had seen….Most vendors’ products don’t integrate as well.” The pricing model and easy integration helped Power Costs achieve ROI in less than four months.

4 customers, 4 different business use cases. They all needed Pentaho to not only deliver the functionality they needed, but also to provide them with tools that can easily be implemented and deliver results fast. Needless to say these customers had budget constraint that could not bless them to a ½ million dollar solution! To summarize their results in a few bullets, what they got out of Pentaho was:

  • Ease of use in implementation for the ETL and BI developer
  • Full integration of ETL and BI to speed project cycles
  • Letting business users make changes and see results fast, without having to wait for IT
  • Scalability of the solution for large volumes of data
  • Low cost in rolling out BI to the masses

For more information on these customers, download the white paper “Realizing the Pentaho Agile BI Opportunity: BI for the Masses and Customer Success” by Joshua Greenbaum, Principal at Enterprise Applications Consulting.

Farnaz Erfan
Product Marketing Manager
Pentaho Corporation

To read more examples of customers like Loma Linda University and Estalea experiencing success with Pentaho visit the Pentaho Customer Success webpage.


Reflections on the Wisdom of the Crowds BI Market Study

January 10, 2011

I recently came across a very interesting study that has been published by Dresner Advisory Services. The Business Intelligent market study titled, “The Wisdom of Crowds,” is written by Howard Dresner, one of the foremost thought leaders in the BI market. This in-depth market research has used a survey tactic to gather the wisdom of 457 BI users (quite a crowd!) on 32 essential metrics for choosing a BI technology. The results are astonishing.

Among the many trends two were the most interesting to me:

  1. Today, smaller BI deployments dominate across all geographies and all industries.
  2. Smaller BI deployments have started to take off in the last 2 years or so and are growing in the expense of large deployments that were once popular 5 to 15 years ago.

This is a huge market shift. 1,000+ user base deployments are no longer ideal or even desired. They have been replaced by smaller deployments for individual organizations, departments, and line of businesses.

While this is no surprise due to the economic conditions that has forced companies to cut back in their capital expenditure, including huge licensing costs just to entitle “every” user to the software and tools in hand, plus money spent on training and hiring skilled users, what has truly made this “shift” possible is the opportunity that open source BI has presented to these clients.

In my opinion, Open Source BI provides this opportunity in two ways:

1. Lower cost and easier to deploy BI software is now available. Notice that I am not just talking about the low cost here. Yes, Open Source BI has been a disruptive technology in the past couple of decades. Newer to the BI world, Open Source bears no acquisition cost for the software. Instead it offers a subscription model for support, which has made it the most attractive alternative for most clients. All the required functionality with only 10% of the cost! What else is new?!

Here, however, my emphasis is on the “easier to deploy” factor. What in the past took a symphony of data modelers, BI developers, ETL developers, data analysts, data architects, data warehouse managers, and DBAs, and a 12 month implementation cycle, is now done “only by a few and only in a few weeks”.

What does this mean for smaller organizations or even departments within larger organizations? It means that now they are able to invest in BI. Something that was not viable a few years ago. They not only don’t have to pay high dollars for BI tools, but also don’t need an exhaustive list of skills, consulting, and expertise to get going.

Dresner’s report shows exactly this. More and more smaller deployments are becoming popular in the last couple of years and are replacing the large ones that were on most corporate priority lists 5-10 years ago.

2. The second reason for this market shift is due to the connection that emerging technologies (such as open source) have made to the line of business owners. Business users have found a way to “free” themselves from IT latency. There is no question that you need your technical staff to initially set up the BI infrastructure and build the first round of reports and analysis, but that shouldn’t mean that every time you want add a dimension to your calculations, or measure something slightly different, you would have to go in a waiting list queue behind several other requests and get an answer 3-4 weeks later. Business users now realize that they can take control, and find the answers for themselves — at least in most cases.

Open Source BI has emerged to enable these folks with tools that let them manage changes in their business processes a lot faster. Dresner’s report is an evident to this fact. The study shows that business users are the most likely to chose emerging technologies over BI tools from the Titans (IBM, SAP, Oracle, and Microsoft) and the BI Pure Plays (Actuate, MicroStrategy).

I would encourage you to download and study this report further. You will be intrigued!

To conclude my points, I’d like to point out a chart from the 2010 Wisdom of Crowds Market Study (The 2011 study is underway). It shows the life cycle of all BI vendors. Several vendors matured 2 to 5 years ago. They saw the most deployments and new customer acquisitions in that time frame. But since then, their deployments have declined. Among these vendors are SAP Business Objects, Oracle, MicroSoft, IBM Cognos, and Actuate. With cobbled together tools from different acquisitions that are not integrated and require deep technical skills and long deployment cycles, not to mention the huge software acquisition costs for clients, there is no wonder to this trend. These tools are anything but suitable for small, agile, and high-value projects.

Farnaz Erfan
Product Marketing Manager
Pentaho Corporation

This blog was originally posted on  Open Source Business Intelligence


13 highlights Pentaho BI Suite Bootcamp

December 15, 2010

As the new product marketing manager at Pentaho, I wanted to write my first blog on my experience attending the four-day Pentaho BI Suite Bootcamp. This class is key to better understand the architecture of the Pentaho BI Suite and get hands on experience with the Pentaho BI Suite, ETL/Data Integration, Analysis and Reporting.

I have to say that coming from an enterprise data integration vendor such as IBM, I was a bit skeptical of any open source solution. What I expected was the bare bones: limited data type coverage, no metadata, basic data modeling and simple BI reporting. What I discovered was to the other extreme. I was taken by surprise on both the breath and the depth of the solution.

Here is my list of lucky 13 highlights from the class:

1.     Coverage of all types of data from what you expect (DB2, Oracle, & SQL Server) to Hadoop, Netezza, Teradata, SAP ERP systems, and of course MySQL and PostgreSQL.

2.     Parallel processing of data transformations – with an ‘intuitive’ UI that eliminates the need for coding. Unlike other tools in the market from both Open Source as well as major proprietary ETL vendors that are code generators, Pentaho Data Integration eliminates the need for coding. Talk about a low TCO!

3.     A shared repository gives a great opportunity for team work and collaboration in the process.

4.     Can handle very sophisticated dimensional modeling concepts – combination, degenerate, and conformed, and slowing changing dimensions, star and snow flake schemas.

5.     Power of OLAP for dynamic aggregation, so developers don’t have to figure out the reporting requirements ahead of time with SQL. This is an extraordinary piece of the puzzle.

6.     Caching and aggregate-aware constructs, so look ups don’t need to hit the database for every row. The wait times will be a lot less and latency is reduced.

7.     Removing the SQL layer for BI developers and data analysts. This has two benefits:  1. Reduces the complexity. 2. Protects databases from novice SQL users that can harm the system. Instead, a set of “business objects” (customer, product, etc) are presented for building reports and dashboards.

8.     Covers all types of reporting needs: Production reporting (for managed and high volume pixel perfect distribution), ad hoc analysis (for interactive analysis to measure business performance against time, location, product lines), operational reporting (for detailed reporting on the current state of data supporting ERP and CRM applications), and data mining (for predictive analysis to detect fraud / next best offers).

9.     Full drill in and out capabilities on reports. Can drill in from summary information to details. Can also drill from report to report with hyperlinks.

10.  Parametrized reports for the interactivity factor needed for analysis. Examples: What if I change the time dimension, or region? What changes I will see and how I need to adjust my operations accordingly?

11.  Dashboards, scorecards, and metrics that allow business users assemble previously built reports, queries, and analysis, needing “zero-training”.

12.  A quick installation got me up and ready in a few minutes. Quote from a customer in the class: “was able to kick this off in 10 minutes”. This customer has evaluated Pentaho’s other Open Source rivals and had found Pentaho to be superior.

13.  And now, for the cream of the crop…Something that the Big guys of ETL are light years behind: Pentaho’s integrated ETL, Visualization, and Modeling environment. It was really cool to be able to create a live connection to an transactional systems (OLTP), select some tables, and from there click right into the modeling and data visualization environments, analyze the content of those tables, do some profiling and decide the best ETL paths as well as best data structures for our new target schema. This was really exciting for me to see, as I have seen struggles around it before. Pentaho not only has all the pieces of the puzzle (ETL, Analysis, Modeling, OLAP, Predictive), which a lot of other vendors don’t even have, but it also has ‘integrated’ them into one development suite that helps the user walk through the process of building a BI solution one step at the time and on a consistent path to completion.

Seeing this set of “sophisticated” ETL, Data Modeling, OLAP, Visualization, and BI tools just left me with one puzzling question at the end. This is what I wrote in my notes at the end of the class:I can’t believe this is all so cheap! Where I was we used to charge hundreds if not millions of dollars for this!”

Farnaz Erfan
Product Marketing Manager
Pentaho Corporation


Agile BI from the road – reflections of a road warrior

December 14, 2010

Last week was not just the celebration of the anniversary of the Pentaho Agile BI initiative, it was also the conclusion of my detective tour throughout the US to both teach and learn about Agile BI.  I call this a detective tour because we really use these types of events to investigate and better understand just what’s going on in the marketplace and what BI challenges companies are trying to overcome. Over the past 4 months Pentaho has visited over 26 cities worldwide with the Agile BI Tour – Data to Dashboards in Minutes. I was at 15 of the cities across the US presenting on our Agile BI Initiative along with our partners.

These live, free workshops were ideal for attendees to learn because of the depth of demonstrations,  discussion of our Agile BI integration development environment and actual customer implementation examples. I personally found it very rewarding being on the ground and speaking with hundreds of BI users (or soon-to-be users). I had the usual slide presentations and demonstrations, yet found that in the majority of the cities, our time would consist of 70% Q&A and 30% slides. Along with the in-depth conversations I had with companies of all sizes across the US, I was also able to analyze the results of the survey’s we received back from almost every attendee of the Agile BI tour. Drawing from my experience on the road and survey analysis, I have four major observations of the current state of Agile BI:

  1. There remains a crying need for a better BI solution in the marketplace.
  2. In almost all of the cities, the majority of attendees were already users of existing BI solutions, mostly those from the traditional vendors such as SAP/Business Objects, IBM/Cognos, Oracle, Microstrategy and others.  While this was initially was a shock, their answers to the questionnaire revealed the dissatisfaction and challenges with existing solutions, and to a large degree, why they are looking for alternatives.
  3. An overwhelming number of attendees responded that their number one challenge with their existing BI solutions was cost followed closely by the lack of IT resources to get their projects done.  Additional related factors included the complexity of existing BI solutions and the time it takes to get BI reports.
  4. Simply put, most users of BI need better, less expensive solutions that can deliver critical analytic data and reports faster. In spite of all of the BI solutions that are available today from the traditional BI vendors, the market and users still find themselves struggling with the core basics of putting more and better BI in the hands of users.

I can’t wait until January and we are back on the road to more locations and more users with the Agile BI Tour.  And of course, we are not slowing down on the product front either with very exciting product plans in the works for next year.  For Pentaho and our users, 2011 promises to be an exciting year.

Joe Nicholson
VP Product Marketing and Agile BI Road Warrior
Pentaho


Follow

Get every new post delivered to your Inbox.

Join 101 other followers