Using Pentaho to Be Aware, Analyze, & Take Action

September 28, 2011

Be Aware

Denial of Service attacking (DoS), IP Spoofing, Comment Spamming and Malware programming… are malicious activities designed to disrupt services used by many people and organizations. If you are taking advantage of the internet to run your business, create awareness of a product or service or simply keep in touch with friends and family, your systems are at risk of becoming a target.

Successful internet “intrusions” can cost you money and even steal your identity. DoS attacks can prevent internet sites from running efficiently and in most cases can take them down. IP Spoofing, frequently used in DoS attacks, is a means to “forge” the IP address and make it appear that the internet request or “attack” is coming from some other machine or location. And Comment Spamming, oh brother…where programs or people flood your site with random nonsense comments and links with an attempt to raise their site’s search engine ranking or increase internet traffic to their sites:

“Nice informations for me. Your posts is been helpful. I wish to has valuable posts like yours in my blog. How do you find these posts? Check mind out [link here]”

Huh? – LOL

You may already have defensive measures in place to address some if not all of these things. There are programs, filters and services that you can use to look up, track and prevent this sort of activity. However, with the continuous stream of unique and newly produced malware, those programs and services are only as good as the latest “malicious” activity that is captured. No matter what, it will eventually cause headaches for many people and organizations around the globe. Being able to monitor when something is “just not right” is a great step in the right direction.

Analyze

In September of 2010, I introduced the Pentaho Evaluation Sandbox. It was designed as a tool to assist with Pentaho evaluations as well as showcase many examples of what Pentaho can do. There have been numerous unique visitors to this site, both legitimate and some as I soon discovered…not. Prior to the site’s launch, using Pentaho’s Reporting, Dashboard and Analysis capabilities, I created a simplistic Web Analytic Dashboard that would highlight metrics and dimensions of the Sandbox’s internet traffic. It was a great example to demonstrate Pentaho Web Analytics embedded in a hosted application. Upon my daily review of the Site Activity dashboard which includes a real-time visit strip chart monitor, I noticed an unusually large spike in page views that occurred within a 1 minute time-frame.

Now that spike can be normal, providing a number of different people are surfing the site at the same time. However it caught my attention as “unusual” due to what I knew was normal. The dashboard quickly alerted me of something I should possibly take action on. So I clicked on the point at the peak to drill-down into the page visit detail at that time. The detail report revealed that who or whatever was accessing the Sandbox was rapidly traversing the site’s page map and directories looking for holes in the system. I also notice that all the page views were accessed by the same IP address within under 1 minute. Hmmm, I thought. “That could be a shared IP, a person or even a bot ignoring my robots.txt rules.” But..as I scrolled down I further discovered there were attempts to access the .htaccess and passwd files that protect the site. I immediately clicked on the IP address data value in the detail report (in my admin version of the report) which linked me to an IP Address Blacklist look-up service. The Blacklist Look-up program informed me that the IP address has been previously reported and was listed as suspicious for malicious activity. BINGO! Goodbye whoever you are!

Take Action

I quickly took action on my findings by banning the IP address from the system to prevent any further attempts to access the site. I then began to think of some random questions I needed to ask of the data. I switched gears and turned to Pentaho Analysis. Upon further analysis of the site’s data using Pentaho Analyzer Report - I was able to see evidence of IP Spoofing and even Comment Spamming coming form certain IP address ranges. The action I took next was to block certain IP address ranges that have been accessing the site in this manner. In addition I created a contact page for those who may be accessing the site legitimately but may have gotten blocked if their IP falls in that range.

Wow, talk about taking action on your data huh?

It is not a question of if, but when an unwarranted attempt will occur on your systems. Make sure you take the appropriate steps to protect them by using the appropriate software and services that will make you aware of problems. My experience may be an oversimplification but it is a great example of how I used Pentaho to make me aware of a problem and take that raw data and turn it into actionable information.

Special thanks to Marc Batchelor, Chief Engineer and Co-Founder of Pentaho, for helping me explore the corrective actions to take to protect the Pentaho Evaluation Sandbox.

Regards,

Michael Tarallo
Director of Enterprise Solutions
Pentaho


Win Moneyball Tickets

September 26, 2011

Moneyball was a huge hit at the box office this weekend. Check out the trailer below. The movie is based on a true story about the Oakland Athletics baseball team and its general manager taking an analytical approach to assembling a team of high yield and low cost players.

Living and working in the Business Intelligence and Analytics world, we want to know how you are taking your BI skills and applying them to your personal sports analytics – this could be Fantasy baseball and football (like several employees here), tracking your running or favorite teams.

Post your story to our Facebook page. The best story will win 2 tickets to the movie Moneyball. Contest ends Friday at noon/est.


Pentaho SPOTLIGHT on BeyeNETWORK

September 23, 2011

The other day, Pentaho Founder and CEO, Richard Daley sat down with BeyeNETWORK Founder, Ron Powell for a SPOTLIGHT interview. Click here to read the Q&A transcript of Ron and Richard discussing the advantages of commercial open source business intelligence and user-driven BI.

A few of the questions include:

  • Why consider Open Source over traditional BI
  • Insight to the Pentaho community and the definition of community size
  • BI trends and what Pentaho is doing to make BI more user friendly
  • Inside look at Pentaho BI 4 and why data visualizations features are important
  • Richard looks into his crystal ball and predicts the future of BI

Check it out here and we encourage you to leave more questions to keep the conversation going or comments.


Pentaho Wins InfoWorld Bossie Award for Fourth Consecutive Year

September 21, 2011


Woohoo! We are very excited and honored to add another InfoWorld Bossie Award to our trophy case. Pentaho was selected for the fourth consecutive year as one of the industry’s best open source applications. The only Business Intelligence company to have won this recognition, the award accredits the improvements in Pentaho BI 4 to data discovery and analytics that make it easy for users to quickly connect to, optimize and analyze their data in an intuitive visual environment.

The InfoWorld Bossie Awards recognize each year the best business-oriented open source software products. Additional information on the 2011 Bossie Award winners can be found at on the InfoWorld website.

We can’t wait to add this to our trophy case including recent employee awards of Sheetz IT VIP Award and CRN Magazines Top Women in the Channel Award


The Right Tool For the Right Job – Part 1

September 20, 2011

 

All too Common

You have questions. How do you get your answers? The methods and the tools used to help get those answers to business questions will vary per organization. For those without established BI solutions; using desktop database query and spreadsheet tools are…all too common. And…If there is a BI tool in place, usage and its longevity are dependent on its capabilities, costs to maintain it and ease of use for both development staff and business users. Decreased BI tool adoption, due to rising costs, lack of functionality and complexity may increase dependencies on technical resources and other home grown solutions to get answers. IT departments have numerous responsibilities. Running queries and creating reports may be ancillary, which can result in information not getting out in a timely manner, questions going unanswered and decisions being delayed. Therefore, the organization may not be leveraging its BI investment for what it was originally designed to do…empower business user to create actionable information.

(Read the similar experiences of Pentaho customer Kiva.org here)

Six of One, Half a Dozen of the Other

The BI market is saturated with BI tools, from the well known proprietary vendors to the established commercial open source leaders and niche players. There are choices that include the “Cloud,” on premise, hosted (SaaS) and even embedded. Let’s face it and not complicate things…most, if not all, of the BI tools out there can do the same thing in some form or fashion. They are designed to access, optimize and visualize data that will aid in the answering of questions and tracking of business performance. Dashboards, Reporting and Analysis fall under a category I refer as “Content Delivery.” These methods of delivering information are the foundation of a typical BI solution. They provide the most common means for tracking performance and identifying problems that need attention. But..did you know, there is usually some sort of prep work to be done, before that chart or traffic light is displayed on your screen or printed in that report? That prep work can range from simple ETL scripting to provisioning more robust Data Warehouse and Metadata Repositories.

Data Integration

Content Delivery should begin first with some sort of Data Integration. In my 15 years in the BI space I have not seen one customer or prospect challenge me on this. They all have “data” in multiple silos. They all have a “need” to access it, consolidate it, extrapolate it and make it available for analysis and reporting applications. Whether they use it already as second-hand data, loaded into an Enterprise Data Warehouse for historical purposes, or produce Operational Data Stores, they are using Data Integration. Whether they are writing code to access and move the data, using a proprietary utility or even some ETL tool, they are using Data Integration. It is important to realize that not all data needs to be “optimized” out of the gate, as it is not only the data that is important. It is how it will be used in the day to day activities supporting the questions that will be asked. This requires careful planning and consideration of the overall objectives that the BI tools will be supporting.

Well, How do I know what tools to use? – Stay Tuned

With so many tools available, how will you know what is right for the organization? Thorough investigation of the tools through RFIs, RFPs, self evaluation and POCs are a good start. However, make sure you are selecting tools based on the ability to solve your specific current AND future needs and not solely because it looks cool and provides only the “sex and sizzle” the executives are after. The typical need is always reporting, analysis and dashboards. Little realize that there is a lot more to it than those three little words. In the next part of this article I will cover a few of the most common “BI Profiles” that are in almost every organization. In each profile I will cover the pains, symptoms and impacts that plague organizations today as well as the solution strategies and limitations you should be aware of when looking at Pentaho.

Stay tuned!

Regards,

Michael Tarallo
Director of Enterprise Solutions
Pentaho

This blog was originally posted on http://michaeltarallo.blogspot.com/ on September 19, 2011

Five Things to Consider When Evaluating a BI Solution

September 19, 2011

If you are at any phase in your evaluation of a Business Intelligence (BI) solution, please take the next minute to download a straightforward resource, The CFO’s Ultimate Guide to Buying Business Intelligence.

Not only will you learn what five things you must consider when evaluating a BI solution, you will also learn:

  • What most BI vendors don’t want you to know
  • Checklist of questions you must ask before you buy a BI solution

Here is the link to download the free guide: http://www.pentaho.com/cfo-ultimate-guide-to-buying-bi/

Pentaho’s COO and CFO, Doug Johnson, is leading an in-depth webinar on this topic on Thursday, September 22nd. You can register here: http://www.pentaho.com/events/20110922-CFO-buying-guide-webinar/

The agenda will cover:

  • Five areas of cost for a CFO to consider
  • How different types of BI compare
  • Customer success stories
  • Demo of Pentaho BI
  • Q&A

Pentaho Fun Week

September 15, 2011

Once a year, after a major release, the Engineering team takes a “Fun Week” break from our regularly scheduled Agile development process to focus on Pentaho topics that the Engineers find most interesting. This year we took the break immediately following the Pentaho BI 4.0.0 GA release. A number of great research projects and ideas were created by the Engineers.

One of the major highlights was the prototype work done for the iPad. Nick Baker updated our new Interactive Reporting client to work on the iPad, and Rob Fellows implemented an iPad compatible User Console interface.

At the end of the week, we held a BarCamp style meeting at the local Dave and Busters where everyone presented their work followed by some fun playing arcade games and air hockey. The most energetic presenter was Ezequiel Cuellar, who prototyped some great new functionality in Pentaho’s User Console, including bookmarks and history. As part of his demo, Ezequiel demonstrated how easy it would be to erase your Pentaho history when your pointy haired boss stopped by at your desk. With this feature, your boss won’t know if you ran that TPS report or not! I didn’t mention to Ezequiel that his boss could always check out the daily Pentaho Audit Logs :-).

Many of the prototypes developed in previous Fun Weeks have become headlining features. Taking a break to focus on fun has proven to be a great reward for the Engineering Team as well as a unique way to allow for product innovation within Pentaho!

Will Gorman
Vice President of Engineering
Pentaho


Follow

Get every new post delivered to your Inbox.

Join 102 other followers