Category Data Mining

New self-service features with Power BI for Office 365 #WPC13 #BI

chrystalextendedresults 1 Comment

Yesterday, we talked about the bundled and rebranded suite of Power BI Office 365 and Excel 2013 tools that was announced at the Microsoft Worldwide Partner Conference 2013. This suite provides self-service Business Intelligence in the Enterprise. Today, we’ll discuss some of the new features that further simplify self-service Data Mining.

The most exciting new Power BI feature is the new Q&A query functionality, which enables natural language processing.  Any non-technical, non-power user can ask simple questions and get immediate results in the form of easy-to-understand charts or graphs.  Non-technical users don’t have to learn structured SQL (T-SQL), chase down an analyst or data scientist, or even require knowledge of the great new Power BI tools, Power Query or Power Pivot to get their results. They can simply ask, “What was my sales volume this year?”  or, “How did my sales do last quarter?” A table or chart pops up showing the data they need to see to get their questions answered.

Power BI

Mobile is all the RAGE!!  Another new Power BI feature that we’re really excited about is the Connected (Mobile) BI experience.  Until now, Power View created very dynamic and rich visualizations using Silverlight, which was only available on Intel-based Windows and Mac OS platforms. With Power BI on Office 365 and SharePoint, Power View now renders charts, graphs and visualizations with HTML5. That means where ever you go, as long as you have a smartphone, tablet or any of the variety of other mobile devices, you can get to your enterprise KPIs.  On vacation in Maui without your laptop? Not a problem with Power BI!

Source: http://blogs.office.com/b/office-news/archive/2013/07/08/announcing-power-bi-for-office-365.aspx

And last, but not least, featuring Business Intelligence in the cloud!  Power BI Sites is a new Business Intelligence collaboration platform in the cloud, which utilizes Office 365 and SharePoint.  With Power BI Sites, you’ll get workspaces in Office 365 that enable sharing and collaboration of data and insights with your customers, colleagues and employees.  Easily keep your team up to date with the latest charts and graphs as data refreshes from multiple data sources.

To learn more about Power BI for Office 365, register for the preview at Microsoft: http://www.office.com/powerbi

Japan Earthquake Analysis–Insight into the numbers that make up the catastrophe

Patrick Husting 7 Comments

updated data through: Fri, 18 Mar 2011 03:15:39 GMT

updated with Twitter data and chart: Fri, 18 Mar 2011

updated with Nikkei and DOW financials to # of earthquakes on Wed, 16 Mar 2011

special note: It has been interesting to hear some of the comments coming back after people reading this blog. I have received, “great job”, “interesting view of the earthquake data”, to “you suck”, “you are trying to capitalize on it” and others. Funny, I’m not selling a thing below. All we did was bring in data from the USGS site to see the data in a different perspective so we could get a better understanding of the quakes. We explained the tools we used so if you wanted to try it yourself you could. The power of the Internet, everyone can have a comment.  BTW, my favorite comment was why didn’t we do analysis to predict the event so they could have taken steps to mitigate it.  Smile   Statistics are important.  If you don’t understand the numbers, you probably don’t understand.

I just wrote a blog about how we mashed up data from the USGS web site so we could get an understanding of the Japan catastrophe.   We wanted to do it because we thought it would be a good training exercise in using Microsoft Excel and PowerPivot.  But when we got the data ready, we were not prepared for the insight we would find.  Below is my best ability to provide some analysis into the earthquake data we have to date.

The following screen shot is the Excel 2010 spreadsheet that we created which you can download the Excel spreadsheet here.   If you want to use the slicing features, you need to have Microsoft PowerPivot installed.  Or you can go to the online report which can be found here, the slicers do not work, but you can see it live.

image

document, excel, file icon

View Live Report

2007, excel, microsoft, office icon

Download Spreadsheet

Ok, onto the analysis

I believe the interesting information is in raw counts.  These numbers were taken at 3/15/2011 at 2pm Pacific Time.  The numbers are staggering and are basically telling you the area is shaking constantly.

  • Total Earth Quakes:  496 534 555 597 Quakes!   Only 600 719 776 847 total world wide in our data set…
  • Average Quakes per day: 83 79.9 77.6 77.0 
  • Average Quakes per hour: 27.7 30.0  32.3 35.3 
  • Average Quake Depth (miles): 16.2 16.5
    Then we moved onto looking at the magnitude of the quakes.   You can see the sheer volume of quakes are in the 4.x and the 5.x range.  All very significant.  But they had 40 in the 6.x range.   Even in the 6.x range, those are enough to cause major damage.   40 is basically saying you are experiencing all day over 4 days!

image

Now we will look at the total number of quakes per day.   You will see they were still experiencing some quakes leading up to March 11 when the big quake occurred.  But the volume of quakes following the next 4 days was amazing.   On March 14, 3 days after the big quake, they had 60 earthquakes!!

image

Now onto the depth of the quakes.  You can see between 15 and 16 miles down, there was some major action going on.  Something big is happening down there and it makes you wonder what it looks like at that depth.

image

My final bit of analysis reveals something very interesting.   The quakes really start in the morning and go most of the day and then fall off during the evening hours.   So what is going on during the day that would cause the earthquakes to pickup?  I’m sure there is some scientist out there that can tell us, they just haven’t.  Surprised smile

image

We just added another chart that puts the financial markets overlaid onto the number of earthquakes per day to give you cause and effect view.  You see a flat line of the Nikkei (gray line) on the 12th and 13th because that was a weekend.  But when the markets opened on Monday, you see the Nikkei start taking a dive.   The green line is the DOW index in the U.S.

image

We started pulling Twitter Tweets on the event.  We used keywords, Japan #nuclear, Japan #earthquakes, Japan #tsunami.  Interesting to see all the tweets on the different days.  During the last several days, I’m surprised that #nuclear is not higher.  The day of the big quake is when you see the most tweets.  I wish we had data for which country the tweets came from.  That would be very interesting to see.

image

We will continue to update the data for another week.  I think the statistics are VERY interesting and really show the constant shaking Japan is experiencing every day.  Please download the spreadsheet and continue to add your data and send it back if you find something interesting.

We are all praying for the people of Japan

Regards

Patrick

Google Ngram Database Tracks Popularity Of 500 Billion Words

Patrick Husting Leave a comment

Google has quietly released Ngram, a free tool that allows users to sift through 500 billion words contained in nearly 5.2 million books published between 1500 and 2008 in English, French, Spanish, German, Chinese and Russian. The tool can compare words and phrases and provide and year-by-year breakdown of when and how often they appeared in print. The entire dataset that powers the tool is also available for download.

Source: http://www.neowin.net/news/google-ngram-database-tracks-popularity-of-500-billion-words

image

Business Intelligence using Satellites

Patrick Husting Leave a comment

You just have to love big investment firms that can invest millions to return billions.   Read this article about how hedge funds and wall street firms are using satellites to view car populations in Wal-Mart parking lots!

As part of a growing trend among hedge funds and Wall Street firms, Cold War-style satellite surveillance is being used to gather market-moving information.  The surveillance pictures are often provided by private- sector companies like DigitalGlobe in Colorado and GeoEye in Virginia, which build and launch satellites and take pictures for US government intelligence agency clients and private-sector satellite analysis firms.  

That means there are two links in the chain before the satellite data gets to Wall Street—a satellite firm takes the pictures and sells them to an analysis firm, which scrutinizes the images and sells the aggregated data to hedge funds and Wall Street analysts.

More at source: http://www.cnbc.com/id/38722872

Other Link: http://rsmetrics.com/

World Cup Social Media Analytics – How We Did It #worldcup #analytics @extendedresults @microsoft

Patrick Husting Leave a comment

A good colleague of mine over at Microsoft, Bruno Aziza, gave me a call on a Sunday night to discuss Extended Results partnering with Microsoft on the development of a World Cup Social Media Analytics web page.   Bruno chose us because we have technologies to capture the social conversation going on along with the advanced visualizations we do in Microsoft SilverLight.  I told Bruno, where do we sign up!  Check out www.extendedresults.com/worldcup for the demo.

We started the project about 7 days ago with Microsoft using our Social Media Server solution to provide some insight into World Cup 2010.   Within just a couple days using our existing Social Media Server solution, we were able to capture over 800,000+ World Cup mentions in just 7 days!  Amazing…

There are many social media software solutions on the market basically all competing for your marketing dollars.   We took a completely different approach to the space in that we believe all this information needs to be captured and stored within a companies enterprise database and tied to other ERP solutions (CRM, Financial, Marketing, Competitive, etc).   Having the information stored within a SQL database, we can create laser sharp focused results out of the social media data, whenever we want, and tie it to business performance objectives.  Powerful…

The below diagrams will give you some insight into the process of capturing World Cup social mentions and providing the insight on our web site using our and Microsoft’s technologies.

 

GRAB THE DATA

  • We started by identifying all the World Cup teams and Players that we wanted to track and entered those keywords into our Social Media Server admin tool.   Our server tool then continuously pulls from the major social media web sites and a predetermined set of blogs and matches those keywords and pulls a rich subset of that conversation and posts it to our SQL database.

soccer

DATA MINE FOR INSIGHTS

  • Once we started capturing all the data, we started writing queries to summarize the results into summary tables.  The reason we did this was because we are adding about 100,000+ new World Cup mentions a day.  We actually used Microsoft Access for a bunch of ad hoc querying because it was so easy to use.  You didn’t have to be a SQL developer to write complex scripts, a business user could use MS Access.
  • Once we got a summary tables completed, we wrote a data web service using Windows Server 2008 R2 and Microsoft Visual Studio 2010 (C#).   We created a set of web service methods that our SilverLight client could call to get its data.

 

VISUALLY RICH DISPLAY

  • Now the best part.   For the past couple of years, we have been using Microsoft SilverLight for many of our business intelligence projects because we can provide a deeper insights with graphical representations of the data.   We used Microsoft Expressions Blend to built out our graphical objects and then passed the XAML to the C#/SilverLight developers (Matt and Steve) and watched them crank out the specific report views below.  Took them all of one day to build it out including the web data service!  They are the Best!

 

We have discovered all kinds of interesting social media facts around World Cup 2010 and will have some later blog posts on them.   Just think of what you could do bringing technologies like these into your company.  They can provide very valuable insights in what your customers are saying about your company and enhance business performance.

You can find out more about our Social Media Analytics solution at www.extendedresults.com