Dinosaurs Have Had Their Day

June 16, 2014

dinosaur

Once upon a time, (not so) long ago in 2004, two young technologies were born from the same open source origins – Hadoop and Pentaho. Both evolved quickly from the market’s demand for better, larger-scale analytics, that could be adopted faster to benefit more players

Most who adopt Hadoop want to be disruptive leaders in their market without breaking the bank. Earlier this month at Hadoop Summit 2014, I talked to many people who told me, “I’d like to get off of <insert old proprietary software here> for my new big data applications and that’s why we’re looking at Pentaho.” It’s simple – no company is going to adopt Hadoop and then turn around and pay the likes of Informatica, Oracle or SAS outrageous amounts for data engineering or analytics.

Big data is the asteroid that has hit the tech market and changed its landscape forever, giving life to new business models and architectures based on open source technologies. First the ancient dinosaurs ignored open source, then they fought it and now they are trying to embrace it. But the mighty force of evolution had other plans. Dinosaurs are giving way to a more nimble generation that doesn’t depend on a mammoth diet of maintenance revenue, exorbitant license fees and long-term deals just to survive.

In this new world companies must continually evolve to survive and dinosaurs have had their day. It’s incredibly rewarding to be  part of a new analytics ecosystem that thrives on open standards, high performance and better value for customers. So many positive evolutionary changes have taken place in the last ten years, I can’t wait to see what the next ten will bring.

Richard Daley
Founder and Chief Strategy Officer
Pentaho

Image: #147732373 / gettyimages.com


Pentaho and Cloudera Impala in 5 words

April 29, 2013

Today our big data partner Cloudera, joined us in continuing to deliver innovative, open technologies that bring real business value to customers. Pentaho and Cloudera share a common history and approach to simplifying complex, but powerful technologies to integrate and analyze big data. Our common open source heritage means that we can innovate at the speed of our customers businesses.

What is Cloudera’s latest Innovation? Cloudera Impala powers Cloudera Enterprise RTQ (Real-time Query), the first data management solution that takes Hadoop beyond batch to enable real-time data processing and analysis on any type of data (unstructured and structured) within a centralized, massively scalable system. Impala dramatically improves the economics and performance of large-scale enterprise data management.

Pentaho and Cloudera Impala in 5 words = Affordable scalability meets fast analytics. Cloudera Imapala enables any product that is JDBC-enabled to get fast results from Hadoop, making Hadoop an ideal component for a data warehouse strategy. Customers no longer have to pay for expensive proprietary DBMS or analytical DBs to house their entire data warehouse.

Cloudera’s innovation makes it even easier for customers to use common analytic tools that can access and analyze data in all of these formats. What does this really mean? It means you don’t have buy expensive, proprietary products that can’t work across all of your data platforms.

With Pentaho and Cloudera you can quickly analyze large volumes of disparate data significantly faster with Impala than with Hive. Take a look at how Cloudera Impala is driving a major evolutionary step in the growth of the company’s Platform for Big Data, Cloudera Enterprise, and the Apache Hadoop ecosystem as a whole.

Richard Daley


The New, Game-Changing Rules of Enterprise Software

July 11, 2011

The rules of the game have changed for enterprise software. In Aaron Levie’s recent article on TechCruch, “Building An Enterprise Software Company That Doesn’t Suck”, he breaks down the changes in enterprise software business into three categories. Levie details how differently enterprise software is developed, sold, and supported today versus just a few years ago.

I found Levie’s perspective interesting – especially because it matches the model that we have followed at Pentaho to achieve great success.

1. How enterprise software is developed today.

It’s no longer about products that are feature-bloated merely to get into RFP wars and win multi-year, large contracts. Those days are over – all they produced were complex technologies that had no real usage.

Now what drives demand is the real business application of software. Success is in user adoption, not in feature checklists.

This is exactly why an open source business model has been successful for enterprise software. The products are developed because there was a real business need for them. Many features are implemented and submitted by the community members, because real users need these capabilities in their business applications. This is a true outside-in, end user focus.

2. How enterprise software is sold today.

Long gone are the days of interruption marketing and trying to sell to every poor soul who happens to pass by. The buying process is much more bottoms up today. As Levie puts it, “With web-delivered, freemium or open source solutions, we’re seeing viral, bottom-up adoption of technology across organizations of all sizes.”

The open source model allows users to buy into the software (aka use it) before actually paying for it. Rather than knocking on every door to find an interested buyer, which is the model many enterprise software companies still follow, our sales organization is focused on actually helping customers navigate their options, providing consultative support and knowledgeable market advice.

3. How enterprise software is supported today.

In a traditional software licensing model, the customer pays a hefty upfront fee, just to get entitled to use the software. In addition, the customer has to pay for an annual subscription and support.

Luckily, buyers have realized that there are better options out there. As Levie rightfully notes, “The unstoppable trend toward ‘renting’ vs. ‘buying’ software, means the vendor gets paid only as the software continues to solve problems for its customer.”

At Pentaho we are committed to our customers’ success and our high customer retention numbers speak to this.

Interested to find out more about our software? Download it now.

Farnaz Erfan
Product Marketing
Pentaho Corporation


Open Source Fuels “Power To The User”

June 22, 2011

Traditionally open source Business Intelligence (OSBI) has been adopted by individuals with an “I” in their acronym: IT, SI, and ISVs.  This made sense because Open Source software started with deep technical roots in operating systems, appservers, RDBMSs, ESBs, etc., and when Pentaho pioneered OSBI in 2004 it was no different. The OSBI landscape back then was filled with some very nice technical projects, some desire from the “then current” open source proponents to move higher in the stack, and the overwhelming BI market screaming for disruption in the form of better products, better architectures and better pricing.

Over the next few years Pentaho built out a great BI platform with end-to-end BI capabilities and quickly became known as “The” OS BI Suite vendor. In conjunction with building product, we built out vibrant open source communities that helped spur our growth and adoption. We were the only clear alternative to Business Objects, Cognos, Oracle and Microstrategy, and the market was hungry for a better and more appropriately-priced offering. We’ve experienced incredible growth over the last couple of years as we continue to deliver great, solid, scalable, secure and functionally-rich products to the market – both to our open source communities and to our paying customers and partners.

In 2009 we started delivering more products geared toward the “business user,” a.k.a. “end-user,” a.k.a. “casual user,” a.k.a. “non-techie.” We immediately started seeing more people convert into customers and this trend grew significantly all the way through 2010. This was a great move in that direction, but we knew that another gap needed to be plugged in order to really satisfy growing market needs – better web-based reporting and data visualization as well as an easier way to get to data. People want results fast and easy, especially the “non-technical user.”  Voila, Pentaho BI 4.0 directly addresses these needs and pain points.  Check it all out here in our Pentaho BI 4 resource center.

Pentaho BI 4 is our biggest release ever that’s geared toward the business user. This product makes it not only possible, but makes it easy for business users to access their own data and do their own reporting, analytics, data discovery and dashboards. We’ve shielded users from the underlying complexities while also giving them the power that they desire.  Our open source roots make it clear that we are all about the people, and this release specifically extends that philosophy to include the “non-technical people.”

Pentaho BI 4 will greatly expand our addressable market size and customer profile. Our early feedback from customers, partners and communities indicates that this is a huge inflection point for our product line and company.  Our launch webcast on Thursday June 23 – Pentaho BI 4 Live – has already broken our previous webcast registration by 250%!

There must be some real unsolved thirst for pushing power to the user.

Richard


Spotlight this week: LexisNexis, Infobright, OpenBI

June 17, 2011

We’ve been heads down the past month preparing for our next MAJOR product launch coming soon (sign-up to be the first to see the impressive new features). In the meantime, we’ve had three partners in the spotlight this past week with exciting new launches and customer success stories that are worth highlighting.


LexisNexis launched their High Performance Computer Cluster (HPCC) technology, making it possible to access and analyze complex, massive data sets in a matter of seconds. They plan to release the internally developed supercomputing platform, HPCC Systems, under a dual license, open source model.

Pentaho is proud to be a founding BI partner in this new exciting open source product making it possible for companies to process and analyze complex, massive data sets in a matter of seconds.

Learn more about HPCC Systems.

Infobright announced Infobright 4.0, the industry’s first database with built-in intelligence for near-real time analysis of machine-generated data. Quoted in the press release is our own CEO, Richard Daley:

“Infobright’s DomainExpert technology delivers new capabilities that allow Pentaho’s data discovery users to analyze their data much more quickly,” said Richard Daley, founder and CEO of Pentaho. “Since our technology is already tightly integrated with Infobright, these new capabilities will be available immediately for customers to generate data analytics faster, easier and more affordably.”

To learn more, read their press release and register for their webinar featuring Infobright 4.0 on June 22nd.

OpenBI published a great case study: Case Commons™, incubated and funded by the Annie E. Casey Foundation. This is a great story of how Pentaho Partner, OpenBI, and Case Commons™ join forces to provide cutting-edge analytics for human services agencies.

Here’s an insert from the results section of the case study. To read the full case study, click here.

Results
Using the Pentaho BI Suite with Pentaho Data Integration as the data transformation engine, OpenBI built the Casebook Analytics proof of concept featuring an analytical data mart and a front-end dashboard application, including advanced charting, filtering and paneling. The application integrates a number of external open source graphics libraries for advanced visualizations, including the ability to analyze ‘trajectories’ of life events and services. The system is designed to empower frontline teams to make timely decisions that increase the safety, permanency and well being of the children they serve.


Reflections on the Wisdom of the Crowds BI Market Study

January 10, 2011

I recently came across a very interesting study that has been published by Dresner Advisory Services. The Business Intelligent market study titled, “The Wisdom of Crowds,” is written by Howard Dresner, one of the foremost thought leaders in the BI market. This in-depth market research has used a survey tactic to gather the wisdom of 457 BI users (quite a crowd!) on 32 essential metrics for choosing a BI technology. The results are astonishing.

Among the many trends two were the most interesting to me:

  1. Today, smaller BI deployments dominate across all geographies and all industries.
  2. Smaller BI deployments have started to take off in the last 2 years or so and are growing in the expense of large deployments that were once popular 5 to 15 years ago.

This is a huge market shift. 1,000+ user base deployments are no longer ideal or even desired. They have been replaced by smaller deployments for individual organizations, departments, and line of businesses.

While this is no surprise due to the economic conditions that has forced companies to cut back in their capital expenditure, including huge licensing costs just to entitle “every” user to the software and tools in hand, plus money spent on training and hiring skilled users, what has truly made this “shift” possible is the opportunity that open source BI has presented to these clients.

In my opinion, Open Source BI provides this opportunity in two ways:

1. Lower cost and easier to deploy BI software is now available. Notice that I am not just talking about the low cost here. Yes, Open Source BI has been a disruptive technology in the past couple of decades. Newer to the BI world, Open Source bears no acquisition cost for the software. Instead it offers a subscription model for support, which has made it the most attractive alternative for most clients. All the required functionality with only 10% of the cost! What else is new?!

Here, however, my emphasis is on the “easier to deploy” factor. What in the past took a symphony of data modelers, BI developers, ETL developers, data analysts, data architects, data warehouse managers, and DBAs, and a 12 month implementation cycle, is now done “only by a few and only in a few weeks”.

What does this mean for smaller organizations or even departments within larger organizations? It means that now they are able to invest in BI. Something that was not viable a few years ago. They not only don’t have to pay high dollars for BI tools, but also don’t need an exhaustive list of skills, consulting, and expertise to get going.

Dresner’s report shows exactly this. More and more smaller deployments are becoming popular in the last couple of years and are replacing the large ones that were on most corporate priority lists 5-10 years ago.

2. The second reason for this market shift is due to the connection that emerging technologies (such as open source) have made to the line of business owners. Business users have found a way to “free” themselves from IT latency. There is no question that you need your technical staff to initially set up the BI infrastructure and build the first round of reports and analysis, but that shouldn’t mean that every time you want add a dimension to your calculations, or measure something slightly different, you would have to go in a waiting list queue behind several other requests and get an answer 3-4 weeks later. Business users now realize that they can take control, and find the answers for themselves — at least in most cases.

Open Source BI has emerged to enable these folks with tools that let them manage changes in their business processes a lot faster. Dresner’s report is an evident to this fact. The study shows that business users are the most likely to chose emerging technologies over BI tools from the Titans (IBM, SAP, Oracle, and Microsoft) and the BI Pure Plays (Actuate, MicroStrategy).

I would encourage you to download and study this report further. You will be intrigued!

To conclude my points, I’d like to point out a chart from the 2010 Wisdom of Crowds Market Study (The 2011 study is underway). It shows the life cycle of all BI vendors. Several vendors matured 2 to 5 years ago. They saw the most deployments and new customer acquisitions in that time frame. But since then, their deployments have declined. Among these vendors are SAP Business Objects, Oracle, MicroSoft, IBM Cognos, and Actuate. With cobbled together tools from different acquisitions that are not integrated and require deep technical skills and long deployment cycles, not to mention the huge software acquisition costs for clients, there is no wonder to this trend. These tools are anything but suitable for small, agile, and high-value projects.

Farnaz Erfan
Product Marketing Manager
Pentaho Corporation

This blog was originally posted on  Open Source Business Intelligence


Congratulations to QlikTech and the new wave of BI

July 26, 2010

It’s been awhile since a BI company went public, so I’m excited by the news of QlikTech’s IPO and what it bodes for our industry. Following the Era of the BI consolidation, when the Big 3 BI vendors grew stodgy and were consumed by mega-vendors like IBM, Oracle and SAP who were looking to add BI functionality (a.k.a. “buy their maintenance stream”) to their existing products, it’s great to see Wall Street showing enthusiasm for a pure BI player. It clearly demonstrates that the market sees significant value in BI innovation.

QlikTech did a great job of taking its technology and packaging it to appeal directly to the business user. It’s a proven enterprise sales model, similar to what Arbor Software did in 1995, and QlikTech should be congratulated for being smart and adapting to an important trend. It’s encouraging to see their market valuation and I’m sure it’s because:

  • Their strong growth rate
  • The BI market is very large and getting much bigger
  • Newer technology that enables direct sales to business users (not to mention a very sexy UI)
  • Other publicly held BI companies are relatively flat

Appealing directly to business users, making it easy for them to upload their data and begin analysis without waiting for support from developers is critical. Every BI vendor must do this and more. The market is also screaming for data integration (simple as well as sophisticated), deployment options (on-premise, private and public clouds/virtualization), and big data analytics just to name a few.

So my congratulations to QlikTech for what they achieved and for raising the excitement level in the BI industry even higher. With this outstanding achievement, I have to wonder how Wall Street might receive a company that is not only financially successful and has great growth momentum, but is also supported by an open source community, has its technology and sales model appealing to more constituents—developers, system integrators, and business users—and has its technology innovated around all the key BI trends. Hmm…

Richard


Improving product quality the open source way

July 7, 2010

Originally posted on opensource.com 7/7/10

If we look at the differences between closed and open source software development processes, we can identify aspects that can be generalized and applied to other industries and domains.

Open source development—that combination of transparency, iterative development with early-and-often releases, and open participation—leads to higher quality products. When we’re talking about software, people tend to think of quality in terms of bugs. But this is only part of the story of open development.

Defects can be anywhere within the requirements, design, implementation, or delivery processes, and we need to consider all of these areas to assess the full quality picture.

  • Requirements defects result in a product that does not meet the needs of the market or misses significant opportunities.
  • Design defects result in a product that tries, but fails, to meet the users’ needs.
  • Implementation defects result in a product that has lower customer satisfaction.
  • Delivery defects result in a product that no one hears about or can get hold of.

The earlier these defects arise in the process, and the longer they are unresolved, the more they cost to fix. When you compound defects in requirements, design, and implementation, the result is an expensive mess. (Windows Vista, anyone?)

A closer look at how this works inside the world of software development will yield larger principles to be applied to any project that aspires to use an open development model.

Under the closed model

Sales reps and account reps relay requirements to product managers, who then describe the required features to software engineers, who then design and implement the features and pass them to quality engineers, who try to test the features and report bugs that need fixing. After all this has happened, customers eventually receive the software. The lack of transparency means defects in the requirements and design don’t get spotted until after the product has been implemented and delivered. Another major problem is that, typically, the quality engineers don’t have any personal use for the software, so it is hard for them to imagine the different use cases that real users will have.

The final product suffers from the lack of connection between the software engineers and the software users.

Under the open model

A transparent requirements process includes consumers adding their own requirements and perhaps open voting to determine the most popular features. An open design process means consumers can ask questions about the design to validate it against their use case. Early-and-often releases during implementation mean that consumers can help find defects, which can be fixed early on. Fixing these defects during early development means features built later are not layered upon resolved defects from the earlier development.

Moving beyond software

So how do we apply these open principles outside of the software industry? Following are some good examples (and one bad one).

Open requirements

Some companies manage to meet unanticipated needs by enabling consumers to create new products for them to sell.

Amazon: As an independent author, Amazon allows you to sell your own books through their service. My mother wrote a book about British birth and death certificates. She uses a print shop in her village, and through Amazon UK she sells to a global market. Amazon sends her the customers’ addresses to mail her books to, and a check to cash, with Amazon’s commission already deducted.

Cafe Press: Create a cool slogan or logo, then upload it to Cafe Press and sell it on a wide array items. The designer needs almost no investment other than time and talent. Cafe Press gets an huge product set, over 250 million unique products—and a portion of each sale.

There are services with similar models for bands, musicians, photographers…

Open design

Lego: Using a free CAD design tool, Digital Designer, Lego customers can design new models, then order that kit from Lego. The creator can also upload the design to Lego’s Design by Me, so that other people can build it. The creator gets satisfaction and kudos, while Lego gets all the money. This builds community and revenue.

Nike: Nike ID lets you customize your own sports shoes. By allowing customization of the product appearance, consumers can create a unique-looking shoe that very few, if any, other people have. The Air Jordan basketball shoe has so many colors and customizable parts that even if every person on Earth bought five pairs, every pair could still be unique. Nike could take this further by letting people name their designs and allow voting for the best.

Local Motors: An open car company, Local Motors holds competitions for the concept and the design of their cars with open voting. Then they hold more competitions for the interior design, parts designs, exterior skins, and accessories. Then they put the vehicle into production. Their first is the Rally Fighter. They also encourage owners to participate in the manufacturing of their own cars. Their vision is to have small manufacturing facilities in most cities, hence their name. The effort put in by the contributors is stunning. The designs are awesome and it’s a highly supportive community.

Open delivery

Transparency and participation can also be used to help spread a message or engage consumers.

T-Mobile, UK: T-Mobile UK started with a successful advert where they staged a flash mob dance in London’s Liverpool St Station, an idea they must have borrowed from a Belgian radio station. Then they broadcast an open invitation to be part of their next event. Over 13,000 people showed up to find out the event was mass-karaoke. The result is really quite touching if you watch it all. It’s not often you can say that about a commercial.

Mountain Dew: Mountain Dew’s Dewmocracy was an open voting system for its next flavor. On their web sites you can see how the voting went down to the county level.

Kraft, Australia: An example of how to do it badly. When coming out with a new variant of their popular Vegemite spread, they had a naming competition. Fifty thousand people submitted entries. Unfortunately the winner was picked by a closed panel of “experts.” They selected “iSnack 2.0″ as the name, thinking it was edgy and cool. Public reaction was swift and very uncool. Within days Kraft announced they were revoking the name and opened a new poll to allow the public to choose the new name. The selected name was “Vegemite Cheesybite.”

Both the T-Mobile and Kraft campaigns involved large numbers of people participating of their own free will. The difference is that everyone participating in the T-Mobile event was part of the final product; if only 10 people showed up the result would have been very lame. In the Kraft case the closed selection panel proved to be the flawed element.

In all of these examples, there are similarities and differences. Some cases require a very flexible manufacturing process, while in others the inventory is electronic. Sometimes the individual contributors do their own manufacturing. In some cases the participants are highly skilled; while for others, little or no skills are required. But in all these cases (well, except the unfortunate Aussie Kraft example) the companies provide more choices, better products, or a better message by enabling open participation of individuals or communities.


Big data should not mean big cost

May 19, 2010

Data is exploding at rates our industry has never seen before and the huge opportunity to leverage this data is stymied by the archaic licensing practices still in use by the old school software companies.  Currently, the big guys like Oracle, IBM, SAP, Teradata and other proprietary database and data warehouse vendors have a very simple solution to “big data” environments – just keep charging more money, a lot more money. The only “winners” in this scenario are the software sales reps. Our industry (Tech) is artificially slowed in order to support these old school business models – they can’t afford to innovate in licensing and they surely don’t want to kill the golden goose – The Perpetual License fee.

A major gaming company, for example, had been using Oracle for its database and BI tech. With traffic reaching 100 million to 1 billion impressions per day, the database giant’s only answer was to sell more expensive licenses. Even then, the best it could do was analyze four days worth of information at a time.

Organizations like Mozilla, Facebook, Amazon, Yahoo, RealNetworks and many others are now collecting immense amount of structured and unstructured data. The size of weblogs alone can be enormous.  Management wants to be able to triangulate what people are doing at their sites in order to do a better job of

a)     Turning prospects into customers
b)     Offering customers what they want in a more timely manner
c)     Spotting trends and reacting to them in real time.

Any company, small or large, that is trying to sift through terabytes of structured and complex data on an hourly, daily or weekly basis for any kind of analytics had better take a long hard look at what it is really paying for. Just like the worldwide recession of 08-09 brought tremendous attention to lower cost, better value prop alternatives like Pentaho, the “big data” movement is doing the same thing in the DB/DW space. And where do you find some of the best innovations in the tech space? The answer is open source.

Specifically, an open source tech called Apache Hadoop is addressing the “better value proposition for Big Data.” It also is the only tech capable of handling some of these big data applications. Sounds great, right? Well not exactly. The issue with Hadoop is it is a very technical product with a command line interface. Once that data gets into Hadoop, how do you get it out? How do you analyze that data? If only there was an ETL and BI product tightly integrated with Hadoop, and available with the right licensing terms…

Today I’m proud to announce that Pentaho has done just that. Early May 19th we announced our plans to deliver the industry’s first complete end-to-end data integration and business intelligence platform to support Apache Hadoop.  Over the next few months we’ll be rolling out versions of our Pentaho Data Integration product and our BI Suite products that will provide Hadoop installations with a rich, visual analytical solution. Early feedback from joint Hadoop-Pentaho sites have been extremely positive and the excitement level is high.

Hadoop came out of the Apache open source camp. It is the best technology around for storing monster data sets. Until recently, only a small number of organizations used it, primarily those with deep technical resources. However, as the tech matures the audience is widening and now with a rich ETL and analytical solution it is about to get even bigger.

Stay tuned to our website and to this blog as I’ll be sharing many success stories over the next 90 days. And most importantly, watch out for the ‘Golden Goose’ licensing schemes from the old school vendors.

Richard

Visit www.pentaho.com/hadoop to watch a demo of Pentaho Enterprise integration with Hadoop and reserve your place in the beta program.


Follow

Get every new post delivered to your Inbox.

Join 96 other followers