Mondrian in Action – win a free copy

September 4, 2013

Image

Mondrian fans your wait is over – the book, Mondrian in Action is complete and available to purchase. We sat down with the three authors to get the inside scoop about the book. Below is our interview with:

  • William (Bill) D. Back @billbackbi – Enterprise Architect and Director of Pentaho Services
  • Nicholas (Nick) Goodman @nagoodman – Business Intelligence pro who has authored training courses on OLAP and Mondrian
  • Julian Hyde @julianhyde – Mondrian Founder and the project’s lead developer

Read below to learn how to win a FREE copy of Mondrian in Action and for a special discount offer from Manning Publications.

1. What inspired you to write Mondrian in Action? 

Bill:  I was assigned to be the Mondrian “expert” on my team and started looking at the documentation.  I discovered that it was pretty spread out and some functionality was in people’s heads.  I started pulling together a set of personal documentation and thought it would be a good resource for others.  I don’t think the book will answer every edge question someone might have about Mondrian, but I do think it does a very good job of covering Mondrian and how it is used for Analysis.

Nick: It was a long overdue book and somehow had to write it!   A full decade after Julian began work on it, there still wasn’t a comprehensive guide addressing the practical soup to nuts of a Mondrian project.  When Bill floated the idea, I was happy to join the project.

Julian: As Nick says, this book is long overdue. It’s my way of saying thank you to the community who have helped Mondrian and Pentaho become successful. In the early days, people were taking a risk when they used a product with little documentation beyond what they could find on the developer mailing list. Yet many people did use it, and their projects were successful, and they wrote blog posts and articles that inspired others to try it too.

2. What is the main goal if the book? What do you aim to bring across?

Bill:  I wanted to create a single resource that people can turn to answer their questions about Mondrian.  This includes understanding how it is used by organizations to analyze their business data, how it is used by various tools, such as Pentaho Analyzer and Reporting, and how to integrate Mondrian into a user’s application.

Nick: The goal of the book was to bring together practical knowledge, from 3 of the top practitioners who have worked with Mondrian at over a hundred customers.  The mix of skills between the 3 authors meant we got diverse topics to make a comprehensive and practical book.  Julian is the code expert, Bill knows the integration/Pentaho aspects, and I have extensive customer modeling/configuration experience.  Readers should benefit from our varied experience with Mondrian!

3. What do you like so much about Mondrian to make you write a book about it? 

Bill:  I love how easy it is for someone with limited technical knowledge to analyze large amounts of data.  Reports are great, but they often provide too much data and are difficult to change to do analysis.  Mondrian lets users quickly analyze their data and make information-based decisions.  It’s a powerful concept for any business in today’s highly competitive environment.

Julian: I was inspired to write Mondrian because I wanted to bring powerful BI to a wide audience. I had used a few multidimensional databases and loved how they enabled business users to ask sophisticated questions of large data sets. But they were expensive, not open source, difficult to install and integrate with, and especially difficult to use from within a Java environment. With Mondrian, I set out to address all of these issues. The book is the logical next step. We can bring BI to a wider audience if we gather everything they need to know about Mondrian into one place.

4. Tell us more about the cover of the book – who is the man on the front and what is he doing? 

Bill:  He’s a man from Konavle, a town Southeast of Dubrovnik, Croatia.  Manning uses a variety of historical figures to demonstrate the diversity in technology.

Julian: I think the “man from Konavle” story is just a cover. I think it’s actually Bill in a false mustache and one of his more elaborate Halloween costumes!

5. What’s next?

Bill:  I’m still trying to figure this out.  I’ve got a huge backlog of things I want to research that I’ve been putting off for the last year and a half.  I have some analysis that I want to do with Mondrian based on topics I’m interested in.  I’m also interested in looking at the emerging predictive and Big Data technologies that take analysis to the next level with machine assisted analysis.

Nick: I’m taking some time off from the Big Data hype machines and staying home with my 6-month-old daughter for the next year.  After that, I’ll look at new data/analytics technologies and jump back in – Mondrian will certainly be relevant when I return. I’m  looking forward to all the great stuff Julian is doing with Optiq and the possible synergy between the two projects.  Also, I’m tinkering with a detailed time based analysis of my infants eating and sleeping schedule and am considering building a “baby-mart.”  Just kidding!

Julian: First of all, I am working on the next Mondrian release. There are lots of new features and architectural improvements, most of which are described in the book. The whole Mondrian dev team is really excited to get it to market.

Outside of Mondrian, in my Optiq project, I am also developing some ideas that will allow fast, easy access to data wherever it resides. It is easy to build an adapter to a new data source, so several projects are already using it, including Apache Drill and Cascading. Optiq could be used to extend Mondrian’s capabilities: to run natively on NoSQL databases and Hadoop just as it currently does on relational data, and to improve Mondrian’s performance by pushing more work into its distributed cache. But Optiq can solve a lot of data management problems besides OLAP, so an option I am considering is to build Optiq into a full product.

*************************************

Win a free copy of Mondrian in ActionLike Pentaho on Facebook and leave a comment here about which chapter(s) you think will be most useful for you and why (you can see the full index in the book here). You also have the chance to win on Twitter by following Pentaho and tweeting your comment with the hashtag #MondrianIA. Nick, Bill & Julian will pick their favorite comment to win. Deadline to leave a comment is Monday, Sept 9 at 12pm/EST.

Manning Publications is offering an exclusive 37% discount off Mondrian in Action when you purchase through Manning.com. At the shopping cart, simply enter the discount code mlbackp.


Our New San Francisco Home

July 23, 2013

The Pentaho team is expanding rapidly worldwide. More awesome team members equals = the need for more space. We are excited to announce that we have moved offices in San Francisco giving our sales and marketing teams a lot more room to spread out and grow. Don’t worry, we still have amazing views of the Bay Bridge! If you are in the Embarcadero area, let us know – we’d love to meet you in person at our new home. Here are some photos from our open house last week.

Pentaho's new San Francisco Headquarters at 353 Sacramento Street

Pentaho’s new San Francisco office at 353 Sacramento Street


It’s a small world after all

July 17, 2013

The other week Dave Henry, SVP of Enterprise Solutions at Pentaho had a great conversation with industry analyst Neil Raden, for his article in Forbes Magazine about Big Data trends. They discussed some of the ongoing roadblocks they see with companies tackling big data issues and how Pentaho helps to solve those issue. One concept that resonated with Neil was the comment, “It’s a small world after all and companies need to find a more optimal use of resources to gain efficiency.” Que the song

This idea and comment made its way to Forbes magazine this month in the article, Big Data, Small World. I encourage you to swing by newsstands in the next few weeks to pick up the latest issue of Forbes. You’ll be able to read more about Dave and Neil’s interview discussing crystal balls, the future of big data and Pentaho on page 120-121. Below is a glimpse of the article and Pentaho ad in this months issue. Head to your news stands today!

Top = Forbes cover (pick it up today) Bottom Left = article cover Bottom Right = Pentaho ad and interview with Neil Raden

Top = Forbes cover (pick it up today)
Bottom Left = article cover
Bottom Right = Pentaho ad and interview with Neil Raden

Rebecca Shomair
Director, Corporate Communications


Awards Are Rolling In for Pentaho

July 11, 2013

Awards CoverPentaho and its customers have been honored with a bevy of awards over the past few months, and I thought I’d share them here. We’re grateful to be recognized so widely as a leader in big data and software development.

Here’s a synopsis of the awards Pentaho has won in the past three months alone.

  • Red Herring 100 – Red Herring identified Pentaho as one of the 100 most promising private technology start-ups in North America for 2013. According to Red Herring, companies are judged based on “financial performance, technology innovation, quality of management, execution of strategy, and integration into their respective industries.”
  • CRN 2013 Big Data 100: Pentaho was named to CRN’s inaugural Big Data 100 list, which identifies “vendors that have demonstrated an ability to innovate in bringing to market products and technologies that help businesses manage big data.”
  • SD Times 100: ‘Best in Show’ in Software Development: This annual award recognizes top leaders and innovators in the software- development industry. Pentaho was included in the new Big Data and BI category, which names vendors the editors believe have “tackled the giant data problem with aplomb.”
  • 2013 SIIA Software CODiE Award Finalist for Best Big Data Solution: This award recognizes the vendors that best enable the “capture, analysis, and mining of large data sets.” Pentaho was the only full data integration and analytics provider among the five finalists.
  • IE Big Data Innovation Award Winner 2013: Pentaho won in the Big Data Technology Provider Category, and accepted the award at the Big Data Innovation Summit In San Francisco. The award is designed to recognize “organizations that have pioneered Big Data initiatives that have or will have an overall positive impact within the Big Data community.”
  • Database Trends and Applications DBTA 100: This inaugural award recognizes “the companies that matter most in data.”

Several of our customers were also honored for their creative deployment of Pentaho’s software to provide business solutions.

  • Apparel Magazine 2013 Top Innovator (Modcloth): Modcloth, an e-commerce retailer, was recognized as a “Top Innovator” for its Pentaho deployment. Modcloth “wanted to transition away from creating reports in Excel and migrate to a more robust platform with richer features,” Jessica Binns wrote in Apparel Magazine. “Pentaho’s reporting software is helping ModCloth to improve the speed and quality of the decisions it makes, [and] get a closer look at customer trends by collecting social media data such as loves, shares and reviews.”
  • Nucleus Research 2013 ROI Awards (STRATO): This award recognizes companies that have made the “best technology deployments.” STRATO, a storage and web hosting company, received the award for its use of Pentaho, which generated a 392% return on investment. According to Nucleus Research, using Pentaho helped STRATO to lower hardware and software costs, increase manager and employee productivity, put off capital investments, and increase revenue.
  • IT Europa’s Government Solution of the Year Finalist (BNova for Italy’s Chamber of Commerce): BNova, an Italian consulting firm, was recognized for a highly innovative application it designed based on Pentaho’s platform. The application will help Italy’s chambers of commerce to forecast economic trends and provide better local support to businesses.
  • Campus Technology Innovators Award 2013: Each year, the Campus Technology Innovators Awards recognize exemplary colleges and universities, their visionary technology project leadership, and their innovative technology vendor partners that have taken technology to extraordinary new heights to meet today’s challenges on campus. Marist College in Poughkeepsie, NY was selected for their Open Academic Analytics Initiative using Pentaho.

It means a lot to us that our work is being recognized by so many big names in the technology world, and across such a range of categories. 
The customer awards are especially meaningful to our team. They prove that Pentaho’s analytics and data integration technology can transform businesses, while delivering high-value ROI. So, don’t be shy. If you’re happy to brag about how transformative big data/business analytics is for your company and customers, send us a note. We’re happy to work together, shining the spotlight on any novel deployments.

Rebecca Shomair
Director of Corporate Communications


Rackspace brings ETL to the Cloud with Pentaho: Hadoop Summit Q&A

June 27, 2013

This week Pentaho has been meeting with the movers and shakers of the Apache Hadoop community in San Jose, at the 6th annual Hadoop Summit. Pentaho and Rackspace are drawing attention on this final day of the show with the announcement of a partnership that brings ETL to the cloud. We’re introducing Rackspace Big Data, a powerful enterprise grade Hadoop as a Service solution. As the industry leader in cost effective data integration for Hadoop, Pentaho is proud to team with Rackspace, the industry leader in enterprise IAAS, to deliver this new era of big data in the cloud.

photo.JPG

L) Eddie White, EVP business development, Pentaho | R) Sean Anderson product marketing manager for cloud big data solutions, Rackspace Hosting

To learn more about the news, we’re talking today with Pentaho’s Eddie White, executive vice president of business development.

Give us a quick overview of this Rackspace news, and how Pentaho is involved.

Rackspace Big Data is an exciting Hadoop as a Service offering with full enterprise features. This is the next evolution in the big data ecosystem, delivering the ongoing structure to allow enterprise customers to choose a variety of consumption models over time. Customers can choose managed dedicated servers, and public, private or hybrid cloud options. Pentaho was chosen as the only Hadoop ETL / Data integration partner for this Cloud Tools Hadoop offering.

So is this a solution for enterprise customers looking to grow their big data operations?

Yes, absolutely. Hadoop as a Service is an attractive alternative for customers that need enterprise-level infrastructure support. Pentaho gives Rackspace a partner with the skills and talent on-board to deliver big data for production environments, along with the support and stability that Rackspace customers demand from their service-level agreements. Enterprises are looking for a Cloud partner with an enterprise-grade infrastructure to support running their business; not just test and development efforts.

What makes up this Hadoop as a Service model?

Together, Rackspace, Hortonworks and Pentaho have jointly delivered an offering that facilitates ease of use and ease of adoption of Hadoop as a Service. Rackspace Big Data includes the HortonWorks Data Platform for Hadoop; Pentaho Business Analytics as the ETL / Big Data Integration partner; and Karmasphere providing Hadoop analytics.

Rackspace excels at the enterprise IaaS model, and now they’ve partnered with Hortonworks and Pentaho to introduce an easy-to-use, consume-as-you-scale Hadoop as a Service offering – so customers can get started today, confident their solution will scale along with their big data needs. Rackspace chose to partner with Pentaho because it is the industry-leading Hadoop ETL and Big Data Analytics platform. Rackspace Big Data offers a range of models to meet any organization’s changing needs, from dedicated to hybrid, and for private and public clouds. And the offering ensures the ability to bi-directionally move data in and out of enterprise clusters, with minimal technical effort and cost.

What does Pentaho Data Integration bring to Rackspace Big Data?

Rather than speak for our partner, I’ll let Sean Anderson, Rackspace Hosting’s product marketing manager for cloud big data solutions, answer that. He sums up what Pentaho brings to the partnership nicely:

“Pentaho Data Integration is all about easing adoption and enhancing utilization of Rackspace big data platforms, with native, easy-to-use data integration. Pentaho is leading the innovation of Hadoop Integration and Analytics, and the upcoming cloud offering with Rackspace reduces the barriers to instant success with Hadoop, so customers can adopt and deploy quickly, delivering faster ROI,” said Anderson.

“Pentaho’s powerful data integration engine serves as a platform, enabling delivery of that content right into an enterprise’s pre-existing business intelligence and analytics tools,” continued Anderson. “Rackspace Big Data customers who require multiple data stores can leverage the ease of operation inherent in their visual ETL tool Pentaho provides. Customers will be able to complement their platform offering by adding the validated Pentaho tool via the Cloud Tools Marketplace.”

A key takeaway is that Rackspace Big Data customers may choose to bridge to the Pentaho Business Analytics platform. As an example, Pentaho’s full suite can be used where a Rackspace customer wants to use both Hortonworks and ObjectRocket. We bring the data in both of these databases to life for the Rackspace customer.

Why is Pentaho excited about this announcement?

This is exciting news because it is Pentaho’s first strategic cloud partnership. As the big data market has matured, it’s now time for production workloads to be moved over to Big Data Service offerings. Rackspace is the recognized leader providing the enterprise with IaaS, with an enterprise-grade support model. We see Rackspace and a natural partner for us to make our move into this space. We are market leaders in our respective categories with proven experience that enterprises trust for service, reliability, scalability and support. As the market for Hadoop and Big Data is developing and maturing, we see Rackspace as the natural strategic partner for Pentaho to begin providing Big Data / Hadoop as a Service.

MarketplaceHow can organizations buy Rackspace Big Data?

For anyone looking to leverage Hadoop as a Service, Rackspace Big Data is available directly from Rackspace. For more information and pricing visit: www.rackspace.com/big-data. Pentaho will also be in the Rackspace Cloud Tools marketplace.


Beer + Pizza + Pentaho = Pentaho London User Group

June 11, 2013
Foto Oficial do Pentaho Day 2013 - Fortaleza - Brasil

Foto Oficial do Pentaho Day 2013 – Fortaleza – Brasil

Guest Post – Pedro Alves, Senior VP of Community, Pentaho

Hello everyone!

Exactly 2 months after the Pentaho Community event in Brazil, that had an all-time record of roughly 200 attendees (see photo on right), we’re hosting our next User Group meeting in London. Shared points between both events? Similar topics, amazing people willing to share their experiences and learn from others and the always fundamental beer and pizza! Unfortunately, the amazing Brazilian weather won’t be the same. Chances are it will be cold and wet – welcome to London :)

The event will be held at the Skills Matter Exchange in the Clerkenwell area of London on Thursday, June 20, 2013 at 6.00 pm. We’re targeting the Pentaho Community, which in my definition includes customers, users, developers, basically anyone that’s willing to spend some time helping to make the product better. It’s one of my main goals as Senior VP of Community to create conditions for that to happen, and I’m interested in hearing ideas and feedback that anyone is willing to share.

Here’s the current agenda:

  • Matt Casters, creator and architect of PDI / Kettle will lead a demo and discussion on how Pentaho supports Hadoop and big data analytics.
  • Dave Romano will lead a talk on how big data start-up Causata has been using Pentaho, specifically covering its use of a custom repository, step plug-ins and embedding Kettle
  • Pedro Alves will present CPK, the Community Plugin Kickstarter, a tool that allow non-developers to create Pentaho Plugins
  • Simon Raybould will describe the dashboard centric implementation at Found, heavily centered around Ctools and Mondrian

Please note that most of the presentations are technically oriented, mainly of interest to consultants and developers. We invite you to propose discussions, technical presentations, user stories and hosted Q&A sessions to educate and inspire other users.

There will be plenty of time before and after the meetup for informal networking. For more information and to register, please visit this link. On behalf of PLUG (Pentaho London User Group) organiser Dan Keeley and Pentaho, we hope to see you on June 20th.

Pedro Alves, Senior VP of Community, Pentaho


Pentaho Concierge Services – 5 FAQ

June 3, 2013
ft_director

Anthony DeShazor, VP of Enterprise Architecture and Principal Enterprise Architect at Pentaho

Last  month Pentaho announced Pentaho Concierge Support Services. We sat down with Anthony DeShazor, VP of Enterprise Architecture and Principal Enterprise Architect at Pentaho and asked him the top 5 frequently asked questions from our customers about this new service. Here is a summary of what we learned:

1. Pentaho recently announced Pentaho Concierge Services. What does this service offer clients?

The Concierge Services provide s a stronger partnership relationship with Pentaho. However, it is much deeper than that. Concierge Services allow customers greater access to Pentaho resources in order to maximize the return of investment in Pentaho. With almost on-demand access to an assigned solution architect and invitations to exclusive technical events, Concierge customers have ongoing access to technical and architectural expertise and to experience in implementing large and complex Pentaho solutions. The solution architect will be a valued partner in developing a long term vision and a plan of implementation. Each solution architect will only serve a few customers, allowing the architect to develop intimate knowledge of customer goals, implementation, and environment. Moreover, this knowledge to help other areas of Pentaho provide better service to Concierge customers.

2. Who is this geared towards? Only fortune 500 companies / smaller fast growing companies can also take advantage of this?

Concierge Services are targeted for the larger and more complex implementations of Pentaho. These implementations are usually for the larger customers. However, smaller fast growing companies can take advantage of the program. In particular, Concierge is a great fit for customers of all sizes who have the internal technical skill but need ongoing technical guidance.

3. Does this Concierge service help clients that are evaluating / building big data projects?

Absolutely! The solution architects who provide Concierge are some of our most experienced architects. The team has experience in implementing Pentaho in many environments—big data, SaaS, enterprise, etc. Concierge is a tailored to help customers develop a vision for their implementation that could include big data and predictive.

4. How is this different than technical support?

Concierge Services and Technical Support are complementary in that they work together to help customers through their implementation; however, they have a specific roles. Technical Support helps with questions related to product features and issues. Concierge provides assistance on strategic questions such as “What is the best way to implement my strict security requirements in Pentaho deployed in multi-tenant SaaS environment” or “Can you provide feedback on the architecture of my Pentaho-Hadoop solution?” These requests are not necessary related to product features but require an analysis of the requirements and experience in large implementations.

5. Is this a charged offering? What should one expect to pay?

Yes, this is a charged offering,  but there are two different levels of engagement that clients can take part depending on your needs – Concierge and Concierge with Strategic Solution Architect. I could go into details of the long list of what is included in each of the services or you can see an easy to read check list of all inclusions on our website at http://www.pentaho.com/services/concierge/.

Thanks Anthony!

Let us know if you have any question and answers you would like us to add. Leave your questions in the comments section below.

To learn more about Pentaho Concierge Services visit: http://www.pentaho.com/services/concierge/


Big Data Integration Webinar Series

May 6, 2013

line-chartDo you have a big data integration plan? Are you implementing big data? Big data, big data, big data. Did we say big data? EVERYONE is talking about big data…..but what are they really talking about? When you pull back the marketing curtains and look at the technology, what are the main elements and important true and tried trends that you should know?

Pentaho is hosting a four-part technical series on the key elements and trends surrounding big data. Each week of the series will bring a new, content-rich webinar helping organizations find the right track to understand, recognize value and cost-effectively deploy big data analytics.

All webinars will be held 8 am PT / 11 am ET / 16:00 GMT. To register follow the links below and for more information contact Rob Morrison at rmorrison at pentaho dot com.

1) Enterprise Data Warehouse Optimization with Hadoop Big Data

With exploding data volumes, increasing costs of the Enterprise Data Warehouse (EDW) and a raising demand for high-performance analytics, companies have no choice but to reduce the strain on their data warehouse and leverage Hadoop’s economies of scale for data processing. In the first webinar of the series, learn how using Hadoop to optimize the EDW gives IT professionals processing power, advanced archiving and the ability to easily add new data sources.

Date/Time:
Wednesday, May 8, 2013
8 am PT / 11 am ET / 16:00 GMT

Registration:
To register for the live webinar click here.
To receive the on-demand webinar click here.

2) Getting Started and Successful with Big Data

Sizing, designing and building your Hadoop cluster can sometimes be a challenge. To help our customers, Dell has developed: Hadoop Reference Architecture, a best practice documentation and open source tool called, Crowbar. Paul Brook, from Dell, will describe how customers can go from raw servers to Hadoop cluster in under two hours.

Date/Time:
Wednesday, May 15, 2013
8 am PT / 11 am ET / 16:00 GMT

Registration:
To register for the live webinar click here.
To receive the on-demand webinar click here.

3) Reducing the Implementation Efforts of Hadoop, NoSQL and Analytical Databases

It’s easy to put a working script together as part of an R&D project, but it’s not cost effective to maintain it throughout an ever building stream of user change requests, system and product updates.  Watch the third webinar in the series to learn how choosing the right technologies and tools can provide you the agility and flexibility to transform big data without coding.

Date/Time:
Wednesday, May 22, 2013
8 am PT / 11 am ET / 16:00 GMT

Registration:
To register for the live webinar click here.
To receive the on-demand webinar click here.
4)Reporting, Visualization and Predictive from Hadoop

While unlocking data trapped in large and semi-structured data is the first step of a project, the next step is to begin to analyze and proactively identify new opportunities that will grow your bottom-line. Watch the fourth webinar in the series to learn how to innovate with state-of-the-art technology and predictive algorithms.

Date/Time:
Wednesday, May 29, 2013
8 am PT / 11 am ET / 16:00 GMT

Registration:
To register for the live webinar click here.
To receive the on-demand webinar click here.

 


Make Your Voice Heard! – 2013 Wisdom of Crowds Business Intelligence Market Study

March 12, 2013

Make your voice heard!

Participate in the 2013 Wisdom of Crowds ® Business Intelligence Market Study and get a complimentary copy of the study findings. 

Dresner Advisory Services is inviting all Business Intelligence (BI) users to participate in its annual examination of the state of the BI marketplace focusing on BI usage, deployment trends, and products.

The 2013 report will build on previous years’ research and will expand to include questions on the latest and emerging trends such as Collaborative BI, BI in the Cloud, and Embedded BI. It will also rank vendors and products, providing an important tool for organizations seeking to invest in BI solutions.

BI users in all roles and throughout all industries are invited to contribute their insight, which should take approximately 15 minutes.  The final report is scheduled to be out in late Spring, and qualified survey participants will receive a complimentary copy.

Click here to start the survey today!


Is your Hadoop cluster big enough to hold your development team’s ego?

March 7, 2013
“There isn’t a cluster big enough to hold your ego!”

“There isn’t a cluster big enough to hold your ego!”

While Gartner describes a “trough of disillusionment” to describe the hangover that follows a period of commercial hype, on the IT side, I see a corresponding “mountain of ego”. Don’t get me wrong. This is not about a sales guy trying to go after the development community – one I proudly belonged to for many years and where I started my journey in this industry. But ask any developer how long it takes to code something and prepare to be amazed by how fast and easy it all is. Three budget cycles later and a couple of delay notifications and we all know better. Agile development tries to cope with this but it’s no silver bullet.

As companies plough ahead with big data initiatives, the relationship between IT and the business has never been more important.  IT and data integration specialists lead most of today’s big data initiatives; it’s uncharted territory, pioneering work and a place to shine a bright and powerful spotlight on IT’s capabilities and potential to add great value to the business. Challenged by the promise of crafting an algorithm that reads like poetry they dive in head first with the scripting language of choice: Python, Ruby, Pig, Perl, JavaScript…whatever you prefer, too bad there isn’t an Hadoop Assembler library available or we could take some real poetic license!

But here’s the problem. It’s one thing to develop beautiful algorithms and dazzling prototypes but what happens when the inevitable errors, exceptions and irregularities, or – worse – the continuous stream of user changes surface? These unglamorous, seemingly trivial inconveniences that seemed hardly worth factoring into the initial delivery estimates invariably wind up causing major headaches and delays. And fixing them is boring and unworthy of the self proclaimed and highly paid Data Scientist!

The reason I mention all this is that no group suffers the grave consequences of putting ego before pragmatism than IT itself. Business Intelligence 1.0 fell into disrepute for taking ages to implement, costing too much money, being inflexible and plagued with backlogs of IT requests. Our industry must avoid the situation in big data where searching for the Holy Grail of scripting nirvana gets in the way of delivering solutions on time to the business so we all can avoid a repeat failure.

When it comes to data analytics, sidestepping IT is reckless. If you thought ‘Excel Hell’ and ‘rogue spreadsheets’ led to inefficiency and poor decisions, just wait for the mayhem that ensues when ‘rogue analytics’ comes to town! Whether it knows it or not, the business needs IT to handle the data cleansing, warehousing, integration and assimilation that is vital to underpinning fast, meaningful, insightful analytics – especially when big data sets come into the picture.

All this means that IT and the business need to work together and find a rhythm instead of trying to get one over on each other. Since the business rarely gets blamed for bad IT decisions (even when it makes them) this rhythm will only happen when IT gets pragmatic and finds ways to work at the (rapid) pace of the business, especially when using big data sources. In addition to changes in culture and mentality, taking advantage of the promise of big data analytics will certainly involve IT using powerful data integration tools instead of coding scripts and hacking algorithms together. However, through these changes, IT stands to earn respect and even hero status, when the business is able to measure revenue gains and efficiency savings.

On the other hand if IT insists on putting ego before pragmatism, the business will find shortcuts, and believe you me they won’t be pretty! So don’t F* around…visit http://www.pentahobigdata.com to learn more.

Davy Nys

*Cartoon drawing by Pentaho’s own Steve Macfarlane


Follow

Get every new post delivered to your Inbox.

Join 97 other followers