Analyze 10 years of Chicago Crime with Pentaho, Cloudera Search and Impala

December 23, 2013

Hadoop is a complex technology stack and many people getting started with Hadoop spend an inordinate amount of time focusing on operational aspects – getting the cluster up and running, obtaining foundational training, and ingesting data. Consequently it can be difficult to get a good picture of the true value that Hadoop provides, namely unlocking insight across multiple data streams that add valuable context to the transactional history comprising most of the core data in the enterprise.

At Strata Hadoop World in October, Pentaho’s Lord of 1’s and 0’s or CTO, James Dixon, unveiled a powerful demonstration of the true value that Hadoop – combined with enabling technology from Pentaho and our partner Cloudera – can provide. He took a publicly available data set provided by the City of Chicago and built a demo around it that enables nontechnical end-users to understand how crime patterns have changed over time in Chicago, unlocking insight into the type of crimes being committed in different areas of the city – not only historically but also broken down by time of day and day of week. As a result, citizenry as well as law enforcement have a much better sense of what to expect on the streets of Chicago from the insight the demonstration provides.

In the demo, end-users start with a dashboard that provides a high-level understanding of the mix of crimes historically committed on the streets of Chicago over the last ten years. Watch the demo here:

This kind of top-to-bottom understanding of (in this case) crime patterns is uniquely enabled by the capability Pentaho delivers to the market, combining dashboarding, analytics and data integration into one easily-embedded platform that leverages blending across multiple data sets.

The deep understanding that Pentaho’s solution delivers to end-users is enabled by two key technologies from Cloudera: Cloudera Search and Impala. The original data set provided by the City of Chicago was loaded into a Cloudera Hadoop cluster using Pentaho’s data integration tool, Pentaho Data Integration (“PDI”). End-user drilldown is powered by Cloudera Search, which executes a faceted search on behalf of Pentaho’s dashboard. Once an area of interest has been located, Cloudera’s Impala executes low-latency performance of SQL on the raw data stored in the Hadoop cluster to bring up individual crime records.

Although Hadoop is often perceived as a geek’s playground, the power of Pentaho’s business-friendly interface is readily apparent when engaging this demo. Unlocking the power of Hadoop can be as simple as engaging Pentaho’s integrated approach to analytics together with Cloudera’s foundational platform to deliver an integrated solution whose value is apparent to nontechnical executives wondering whether Hadoop is the right choice for a key initiative.

Rob Rosen
Field Big Data Lead
Pentaho


Big Data, Big Revenue for Marketers

December 12, 2013

Why might Big Data mean millions for marketing?  Because it has the potential to create a more complete picture of the buyer, thereby empowering marketers to more effectively deliver the right message to the right individual at the right time – and ultimately increase sales.  In the following brief video from DMA 2013, Marketo VP/Co-founder Jon Miller and Pentaho CMO Rosanne Saccone provide a crash course on what Big Data means for marketers.  It covers:

  • The defining characteristics of Big Data – Velocity, Variety, & Volume
  • How marketers can leverage Big Data to blend operational information (CRM, ERP) and online data (web activity, social networking interactions) for new insights
  • Sample Big Data use cases that organizations are green-lighting today to optimize customer interactions and drive marketing’s contribution to revenue

Note that this is an excerpt from a larger presentation – for the full video please click here.

We’d also recommend this blog post by Jon Miller for more context on Big Data in marketing.

For additional compelling use cases that leverage Big Data for marketing and other functions, see here.

Ben Hopkins
Product Marketing
Pentaho


Weka goes BIG

December 4, 2013

funny_science_nerd_cartoon_character_custom_flyer-rb4a8aff0894a4e25932056b8852f8b18_vgvyf_8byvr_512.jpgThe beakers are bubbling more violently than usual at Pentaho Labs and this time predictive analytics is the focus.  The lab coat, pocket-protector and taped glasses clad scientists have turned their attention to the Weka machine learning software.

Weka, a collection of machine learning algorithms for predictive analytics and data mining, has a number of useful applications. Examples include, scoring credit risk, predicting downtime of machines and analyzing sentiment in social feeds.  The technology can be used to facilitate automatic knowledge discovery by uncovering hidden patterns in complex datasets, or to develop accurate predictive models for forecasting.

Organizations have been building predictive models to aid decision making for a number of years, but the recent explosion in the volume of data being recorded (aka “Big Data”) provides unique challenges for data mining practitioners. Weka is efficient and fast when running against datasets that fit in main memory, but larger datasets often require sampling before processing. Sampling can be an effective mechanism when samples are representative of the underlying problem, but in some cases the loss of information can negatively impact predictive performance.

To combat information loss, and scale Weka’s wide selection of predictive algorithms to large data sets, the folks at Pentaho Labs developed a framework to run Weka in Hadoop. Now the sort of tasks commonly performed during the development of a predictive solution – such as model construction, tuning, evaluation and scoring – can be carried out on large datasets without resorting to down-sampling the data. Hadoop was targeted as the initial distributed platform for the system, but the Weka framework contains generic map-reduce building blocks that can be used to develop similar functionality in other distributed environments.

If you’re a predictive solution developer or a data scientist, the new Weka framework is a much faster path to solution development and deployment.  Just think of the questions you can ask at scale!

To learn more technical details about the Weka Hadoop framework I suggest to read the blog, Weka and Hadoop Part 1, by Mark Hall, Weka core developer at Pentaho.

Also, check out Pentaho Labs to learn more about Predictive Analytics from Pentaho, and to see some of the other cool things the team has brewing.

Chuck Yarbrough
Technical Solutions Marketing


Customer Spotlight: WiMP Music, the 2013 BI Award winner for Innovation

December 2, 2013

wimpAwardIn today`s blog post, I want to put the spotlight on our Norwegian partner Conduct and congratulate them for their great work implementing Pentaho for WiMP Music, a music streaming service similar to iTunes that is very popular in Northern and Central Europe. The deployment received the 2013 BI Award for Innovation in Norway given by the prestigious Norwegian Computer Society on October 31st in Oslo.

Through local editorial teams in each country, WiMP provides daily recommendations, tips and playlists for any occasion for its audiences. The ad-free service is available on computers and mobiles, tablets and network players in Denmark, Germany, Norway, Poland and Sweden. The business is entirely digital and data-driven, with music files licensed from a large number of different sources delivered by a huge partner network and priced dynamically, according to where and how it is sold. Wimp’s BI solution was built by Conduct on Pentaho Business Analytics and has been in production since 2010.

Pentaho has become essential for delivering the music streaming service WiMP provides, because it controls income distribution and settlement. It also provides information that WiMP is contractually obliged to provide to content providers and partners. In addition, the solution provides a huge range of easily accessible management information for decision-making based on facts, not gut feelings.

The Jury stated that this year’s winner of the BI Award for Innovation has:

  • Adopted BI as a core component of its business
  • Been thinking outside the box and dared to challenge traditional IT architecture when establishing its business and its IT portfolio. The solution is flexible and meets the changing challenges of a dynamic market.
  • Used Open Source in its solution design and decided to use the data warehouse as the core of its business systems and business model
  • Created a robust solution that satisfies both audits and controls and creates numbers its partners trust

Congratulations to WiMP for its foresight of truly extracting the real value of data and building its business model based on it. And of course a big congrats to our channel partner Conduct who built this great deployment on our Pentaho Business Analytics platform.

Pentaho is supporting its partners also with its marketing activities. If you have impressive customer stories like WiMP, our team is on hand to help you promote and celebrate them by writing up case studies, press releases, co-hosting a webinar or completing award applications. If you are a Pentaho partner and want to get the most out of your partnership, please contact me.

Erik Nolten
Director Channel EMEA & APAC


Happy Thanksgiving from Pentaho

November 27, 2013

To all of our readers in the US, we hope you have a Happy Thanksgiving and are enjoying some time off with friends and family.

In the spirit of Thanksgiving we want to say that we are thankful for our amazing community, customers, partners and colleagues. Happy Thanksgiving!

Visit our Facebook page to see photos from our Worksgiving potluck in the San Francisco office. Everyone brought in their famous Thanksgiving recipes such as Thai sesame chicken wings, spring rolls, jalapeno corn bread, lemon cream bars and of course a big turkey with stuffing.

1453340_10151775699832724_1030180799_n

Click here to see more photos 


Pentaho wins Red Herring Global 100 Award

November 22, 2013

RH_global100Following Pentaho’s success this summer in winning the Red Herring North America competition, which recognizes the most promising private technology companies, Pentaho was invited to participate in this week’s Red Herring 2013 Top 100 Global event and award competition. And we’re pleased to announce that Pentaho won!

We’re honored to be recognized as a Red Herring 2013 Global Top 100 company!

Red Herring Global culminates a full year of work scouring the globe by the Red Herring editorial team and venture capitalists reviewing thousands of privately held companies in regional competitions around the world. The world’s top technology companies are selected based on financial performance, technology innovation and a variety of other metrics that have made the Red Herring Global 100 a mark of distinction for identifying the most promising private companies and entrepreneurs.

Pentaho was joined by a list of impressive company finalists for the Global 100 competition including Pentaho customers Spreadshirt GmbH and Active Broadband Networks (both ended up on the top 100 list as well – our heartfelt congratulations go out to these innovators!). Top private companies from the Red Herring’s Regional competitions in Europe, North America and Asia flew to the Red Herring Global forum in Los Angeles for the final competition on November 19 and 20.

Pentaho CEO, Quentin Gallivan, was asked to present at the forum, where he shared our point of view about how the big data and analytics markets are transforming, particularly with the need to easily blend big data with other data sources for better business insights. Quentin provided insights from our customer front lines about how Pentaho’s Big Data pilot projects are transitioning into widespread deployments with real business impact—with the most powerful insights coming from blending relational and big data sources.

The Red Herring Global forum concluded with an awards gala, where the 2013 Red Herring Global 100 companies were announced.

photo.JPGThis is great validation for Pentaho, as well as our customers, partners and community. Together we’re driving the future of analytics, and the Global 100 award provides a very solid foundation to build upon as we push the boundaries of Analytics with Storm, YARN, and predictive analytics in 2014.

Rosanne

Rosanne Saccone
Chief Marketing Officer
Pentaho


Amplify SaaS Apps by Tapping Diverse Data

November 14, 2013

How can analytics generate value in SaaS, CRM, HCM, and the proverbial “alphabet soup” of cloud applications?  Read below for the “cliff notes” from Pentaho’s recent webinar series on how vendors are leveraging analytics inside a variety of different SaaS (Software-as-a-service) apps. 

Pentaho just wrapped up several webinars with Montclare, a leading provider of SaaS advisory services, on how embedded analytics can benefit SaaS vendors in the Customer Success, Human Capital Management (HCM), and Customer Relationship Management (CRM) categories.  One clear trend was that thinking outside the box to incorporate new & novel data sources can create tremendous value for end users.  It’s not just about pretty visualizations and dashboards – it’s about insights that bubble up when diverse data is blended together.  We’ve broken out some webinar highlights below, and have also created a short summary video for each webinar – see links in each section.

Customer Success with Nick Mehta of Gainsight

Gainsight is a provider of customer success solutions.  CEO Nick Mehta stressed that before you can really build process around customer success, you need a data management strategy.  This means figuring out where crucial success information lives – whether in CRM, Product usage databases, customer surveys, billing systems, or other places.  It also means finding the right way to integrate this information and understanding the data granularity necessary for useful analytics (time frequency, aggregation, etc).

Gainsight’s “at a glance” view of customer base according to risk level (dot color), revenue (y axis), and time since they were signed (x axis)

Gainsight’s “at a glance” view of customer base according to risk level (dot color), revenue (y axis), and time since they were signed (x axis)

Big Data is one path for streamlining a data management approach.  Gainsight leverages a hierarchical infrastructure, where Hadoop serves as a high-volume raw data store, and NoSQL is used as an operational store for more summarized data, which then connects to a final end-user reporting layer.  Data and architecture innovation has allowed Gainsight to create an “at a glance” view of which customers may be at risk (see picture).  This resonates with an evolving trend where SaaS vendors are innovating via Big Data architectures to create more powerful, intelligent, & valuable apps for their customers.  The Customer Success video can be found here.

HCM with Janice D’Aloia of Mo’mix

Mo’mix is an IT software and services provider that embeds Pentaho Analytics in its Performance Center offering – which it launched to give customers a single view across a variety of enterprise data systems.  While self-service analytics and quick deployment helped drive the decision to partner with Pentaho, Mo’mix has really been able to generate value for customers by being able to connect to virtually any data source (ERP, HCM, niche sources, and more), bringing together an end-to-end view that’s accessible to all types of end users, from HR administrators up to the executive level.

Janice went over a few use cases where its clients improved HCM decision-making.  In one example, a client discovered that it was missing employee termination reasons in its source systems, and then was able to fix the discrepancy and track the relevant trends at a more granular to reduce voluntary terminations.  She also touched on retention use cases that focused on identifying high-performing and at-risk employees.  The HCM video can be found here.

CRM & Marketing with Montclare & Pentaho

In this presentation, Kevin Dobbs of Montclare shared some of the analytics-related trends driving CRM & Marketing application adoption – including demand for data insights on a variety of social media channels.  This was exemplified in the case of Paytronix, a Pentaho OEM partner that provides online loyalty solutions to restaurants.  As part of their deployment, Paytronix integrated social and mobile data sources, as well as Big Data technologies, to help clients optimize campaigns against end customer behavioral data more effectively.  At the same time, as they streamlined multi-source data for embedded analytics, Paytronix was also able to reduce ETL processing time by 80%, generating major operational benefits for customers.  The CRM & Marketing video can be found here.

These webinars highlight just a few examples of how application providers across the SaaS/software landscape are striving to deliver new data sources to their customers for additional analytics context and value.   You can access the full-length webinar videos here.  

Ben Hopkins
Product Marketing
Pentaho


Integration Solutions are Key in Unlocking Big Data Benefits

November 6, 2013

VR_HorizontalLogoWe’re tremendously excited that Ventana Research is launching a new benchmark research on Big Data Integration.  As it becomes clearer to the market that Big Data has huge potential to deliver business value, identifying the right tools & technologies to manage it in the context of specific organizational goals is crucial.  The new research aims to take a step forward in this realm, as it promises to take a hard look at current and planned Big Data deployments, honing in on the solutions for accessing and transforming Big Data in a variety of different technical architectures and business use cases. It should provide great insight into ways that organizations can “unlock” Big Data’s potential.  See the press release here – Ventana Research Launches Big Data Integration Benchmark Research

For further context, we recommend reading a recent blog post from Ventana’s VP & Research Director, Tony Consentino titled, Big Data and Analytics Helps Business Transform and Gain Competitive Advantage. Consentino highlights the broad applicability of Big Data solutions across industries to empower organizations in the face of accelerating changes in markets, operations, & customer preferences.  The blog also cites staffing and training as the two biggest challenges to Big Data analytics, which underlines the importance of the new benchmark research in helping businesses identify which integration approaches will help accelerate time to value through usability by the broadest base of team members.

If you are interested in learning more about or participating in this benchmark research visit ventanaresearch.com/BigDataIntegration. There are several incentives for qualified research participants such as a $5 Amazon.com gift certificate and a complementary report valued at $995 as well as access to a free educational webinar on best practices from the benchmark research.

Finally, the Ventana Research Innovation Summit is next week, November 11-12 in Chicago. Let us know if you will be attending as we would love to meet-up!

Ben Hopkins
Product Marketing


#PPSUMM13

October 15, 2013

More than forty partners from around the globe attended Pentaho Partner Summit 2013 (#PPSUMM13), October 3-4 in the beautiful village of Sintra, Portugal, famous for its 19th-century Romantic architecture and landscapes. The following video summarizes the two days in under two minutes:

Informative sessions

To further help partners to maximize and monetize big data opportunities, the Summit included sales and technical training sessions focusing on different aspects of the big data ecosystem. During these sessions Steve Jenkins, Vice President EMEA of Hadoop distributor MapR, and Luca Olivari, Director of Business Development EMEA of MongoDB, the NoSQL database, explained how Pentaho supports these important and popular big data technologies.

Partner of the Year Award

The Pentaho partners that were the most successful over the last year, delivering outstanding customer success, technical innovation and commercial results, were honored at a gala dinner with the Pentaho Partner of the Year Award. This year’s winners were Italy’s BNova (EMEA), Brazil’s OnCase Brazil (LATAM) and Japan’s KSK (APAC).

 New partner program

The Pentaho Partner Summit 2013 was the perfect occasion to share more details on our new enhanced Partner Program in EMEA, Asia Pacific and Latin America. The new program consists of three levels: Registered Member, Certified Partner and Premium Partner. Registered Member is a new entry level that makes it easier for new partners to sign up to the partner program. The main partnership level, now called ‘Certified Partner’ has been simplified. To reward commitment, Certified Partners that offer advanced services and sell successfully are eligible to earn the top ‘Premium’ status. These new levels replace the previous Bronze, Gold and Platinum and make it easier for customers to select the right partner for their needs.

Learn more about the Pentaho Partner Program visit pentaho.com/partners


Two meetings of hearts and minds in Sintra

October 9, 2013

I consider myself very lucky to have spent last week in the beautiful village of Sintra, Portugal for Pentaho’s Regional Partner Summit followed by the Pentaho Community Meeting. Over four days our bright, enthusiastic partners and community developers animated the romantic Portuguese scenery. If anyone thought that analytics and romance might be incompatible, they were definitely proven wrong!

Partner Summit

Over the first two days more than forty Pentaho partners from around the globe including Australia, Brazil and the Norwegian fjords came together in the Tivoli Hotel to network, share their experiences and gather insights from Pentaho folks onsite.

Big data has injected new vitality to the business analytics space, providing major new opportunities for our partners.  An informal survey we carried out at the summit indicates that about 20 percent of our partners are already working on big data deployments.

To help our partners maximize and monetize big data opportunities, the Summit included sales and technical training sessions focusing on different aspects of the big data ecosystem. For example, Steve Jenkins of MapR and Luca Olivari of MongoDB explained how Pentaho supports these important and popular big data technologies. Thanks to Steve and Luca for taking the time to speak at the Summit!

Pentaho Partner Meeting 2013

Pentaho Partner Summit 2013

Pentaho Partner of the Year Award

One of the highlights of the Pentaho Partner Summit was the Gala Diner in the Seteais Palace – a fitting place to honor the outstanding results from our partners and treat them like the royalty they are! Congratulations once again to Italy’s BNova (EMEA), Brazil’s OnCase Brazil (LATAM) and Japan’s KSK (APAC) for winning the Partner of the Year Award for their respective regions.

Here are some additional photos from the Pentaho Partner Summit and Partner of the Year Awards Dinner

#PCM13

After two intensive days, the Partner Summit drew to a close but the Pentaho party raged on! Most partners stayed on in Sintra to take part in the Pentaho Community Meeting, organized by our new SVP of Community, Pedro Alves and his Webdetails team. With more than a hundred participants gathered in the Pena Palace’s Gardens, this was the biggest community meeting ever to take place in EMEA. It was so inspiring to learn about the many exciting projects that this vibrant community has generated.

Pentaho Community Meeting 2013

Pentaho Community Meeting 2013

Thank you!

After a fun and interesting week teaming with Pentaho spirit it’s only left for me to say a warm ‘thank you’ to everyone who attended and spoke and organized these two fantastic events. Personally, I am leaving impressed and inspired after seeing first hand the impressive projects that our partners and community are creating with Pentaho. I hope you took away new ideas, opportunities, knowledge, and perhaps even a little bit of Sintra’s romance!

Doug Johnson
EVP and COO, Pentaho

Here is a growing list of recap blogs and videos from the events: 


Follow

Get every new post delivered to your Inbox.

Join 88 other followers