Data Visualization Golden Rules

Last week I posted about a friend of mine, John Brookmyre, who blogged about getting started with cloud computing. It's a great post and John's blog is something that's definately worth keeping an eye on, because he understands and can communicate business intelligence very well in addition to being able to deliver technically in some 20+ platforms.



John's latest post, is about business intelligence data visualization, something that I've been very interested in for the 2 years, and have written about my experiences in the following;

• User Centred Design and Business Intelligence
• Business Intelligence Dashboards

Interestingly, Johns post has been commented on by Tableau Software, who are niche players in the BI market, but among the industry leaders in BI data visualisation. You can trial the software, which I did about 9 months ago when I was looking for alternatives to Microsoft Performance Point Server.



On that occasion the client purchased PPS instead, which we deployed via a customised sharepoint front end. We got round the limitations of both PPS and sharepoint with some clever visualisation tricks and the result was stunning. I'd been reading Stephen Few's blog for a while, (he's a modern day Edward Tufte) and also had some recent experience of heuristic evaluation, which meant I was well up to speed on usability and visualization. If your interested in getting into the subject then these are good places to start.



Back to Tableau though. It's a very interesting product and I wasn't able to get through it all in the 28 days. I managed some very pleasing visualisations involving multiple graphs in a grid formation. The advantages of this being you waste very little space on title's and legends as you benefit from a write once, use many policy. I managed to get 25 graphs on screen (5x5) in a layout that wasn't off-putting, and I could see it working for business users. I've pasted in below an example of the tableau website, that demonstrates the type of visualisation I'm describing. (I like to think that mine was better though!)

Corporate Social Responsibility Reporting

Corporate Social Responsibility (CSR) is becoming ever more important. Some recent headlines;

Tesco is to test putting "carbon labels" on its own-brand products next month in a move to enable consumers to choose products which are less damaging to the environment. Guardian

Marks and Spencer is to begin charging its food shoppers for carrier bags. BBC

Of course Corporate Social Responsibility reporting isn't new in itself, it's the increased focus and level of detail that is changing. As a result, companies will report in ever more detail on these issues and refine their definition of Corporate Social Responsibility. CSR is an increasingly important part of a company’s annual report and CEO’s are making ethics and sustainability claims that need to be backed up. The trend therefore is that CSR data is being treated like financial data which requires greater accountability and accuracy in the reporting of Corporate Social Responsibility objectives, increased levels of corporate governance, and stronger internal control documentation across the enterprise. On the back of increased CSR reporting will come increased auditing requirements as mandated by good corporate governance.

Several governing bodies have arisen to provide guidance and best practice such as the Global Reporting Initiative (GRI) and the World Business Council for Sustainable Development (WBCSD). Each of these has their own protocol for calculating and reporting environmental KPI's such as carbon emissions. Invariably these rely on excel spreadsheets that are manually entered, in disparate locations around the field of operation. This is going to become less and less acceptable.

Corporate Social Responsibility data will need to be audited and we'll see security, workflow and audit trail issues seeing increased focus in the capture and reporting of CSR data. Here at Retail Monster Consulting we can help you deliver CSR reporting in a robust and usable manner through Business Intelligence (BI) technology. Our CSR dashboard and data model experience, can help you fast track a CSR implementation.

We are actively blogging about CSR related topics such as Green Credentials and Carbon Labelling.

If any of this interests you then please talk to us

Queue theory can help manage retail queues

Queues are bad. Really bad. They literally cost you money. One of the principles of queue theory is that once a queue forms it has to be managed, which requires resources, then you need over capacity at some point, in order to reduce the length of the queue. Often the resource to manage the queue comes from the very thing serving the queue. This means the queue gets processed even more slowly than before the queue started and the queue gets longer. It doesn't start to get shorter until you have over capacity.

Retail queues cost money in all kinds of ways and often those costs are hidden, or at least not immediately apparent. Long queues lead to customer dissatisfaction and ultimately to lost customers. Losing customers obviously isn't a good thing and will cost you money in the long run, and it's in this respect that most people associate the cost of queueing. (Queueing, incidentally, is the only word in the English language to contain 5 consecutive vowels)

Some less obvious ways that queues cost money involve

o In a supermarket, the space kept clear between the checkouts and the start of the aisles is valuable space that is being used to manage queues instead of being used as shelf space.
o Two delivery lorries arriving at a store/depot at the same time cost money in waiting time.
o Or a more trivial example, taking a ticket at the meat counter involves the cost of the little yellow tickets and the dispenser

Understandably therefore, minimising queues is top priority, and as something that has big impact on customer dissatisfaction, it's something of a PR tool in the supermarket wars. The trade off is balancing the cost of resources used to prevent queues, vs the cost of the queue itself. Having all checkouts open all the time would go a long way to easing congestion in the store, but that has to weighed up against the retail wage budget, (the single biggest expense for a large retailer), used to pay all of those checkout operators required to ensure no queues.

Retailers understand this trade off and have developed complex systems to help them manage queues, something that the average shopper trying gauge whether aisle 19 (short queue, but inefficient operator) is better than aisle 20 (longer queue but smaller basket size) probably doesn't appreciate. (Fuzzy logic might help here!) Space in store is critical and the retail wage budget is massive, so what looks like large investment can be easily offset by the cost of real savings in either of those areas.

You might like to ponder this next time your stuck in a checkout queue, It's something I can't help myself doing... (My personal favourite is the Wilkinson's dot matrix receipt printer, which takes something like 15 seconds to print your receipt. Whilst laser printer are more expensive, the retail wage budget saving, at roughly a quarter to half a million pounds per second, I'm sure would massively outweigh the investment)

You can apply a queue theory equation in loads of different ways (another favourite of mine is NHS waiting lists). In a later post I'll talk about how I've used queue theory to revamp a large data warehouse batch schedule..

Green credentials : altruism or commercial interest?

I recently facilitated a Retail Business Intelligence Seminar, which was attended by many of the UK's leading retailers. The format of the day was a number of presentations and some round table discussions on topics that had been selected by the attendees in advance by way of a questionnaire. The Butler group presented on the leading topics of the day within business intelligence and we heard from Conchango's David Ellis, on web performance management.

Corporate Social Responsibility reporting, colloquially referred to as 'Green' reporting, was one of the latter topics of the day for the round table discussion. Green credentials are obviously top of the agenda at C-level of the top retailers and are filtering their way down the organisational hierarchy. IT departments haven't yet got to grips with this in any serious way although departments like Marketing are in full swing, which is why we're all familiar with stories about Marks and Spencer, who are soon to start charging for carrier bags as part of their Plan A initiative, but in the meantime have been giving away free 'Bags for Life'. Sainsbury's have dipped their toe in the 'free Bag for Life' water too. Tesco of course are in on the act with the Future Store initiative, check it out next time you're in Wick (!), and have installed, reputedly, the worlds largest solar panel roof at their Fresh and Easy distribution centre in the US.

The discussion at the table rounded very squarely at one point on 'Are they doing it for money or because they care for the planet?'. Unanimously, instantly, 'for the money!' was the response. Was I the only person slightly disappointed by this response? Or have I bought into the marketing message and believe they do care after all? Who knows? but later when I reflected back on this, I thought, 'If they are in it for the money, in what directions could they take it?'

If you shop with a Tesco, and use your clubcard, then Tesco will know every item you buy. Tesco could therefore work out an individuals customers green credentials from the products they buy, assuming they work out individual product greenness, which surely is a first step. They'll already use some Fuzzy definition of greenness probably linked to a measure of ethicalness for segmentation purposes, but this will allow a whole new measure of granularity.

Doing this would of course give rise to opportunities to up-sell people to more 'green' products, that as well as espousing their green credentials, probably cost more and have a greater margin. This has profit implications but also huge customer satisfaction potential. Imagine how powerful Tesco online shopping could be if you could be presented with greener alternatives to those Kenyan strawberries, right there on the page? This has the added advantage of being applicable to all customers for as we have all heard, there are those vocal customer groups who shun the mighty clubcard for big brother reasons. For the many who carry one though, (13.5 Million), I wouldn't be surprised if they actually wanted Tesco to go this far. Why try and work out your own carbon footprint when someone else can do it for you?

As I stated above, 'Green' is big at C-level and has slowly started filtering down, at some point in the near future green credentials will filter it's way into mainstream BI. Expect more blogs on this in the future....

Selling houses with fuzzy logic

Isn't searching for a house online addictive?

You spend hours and hours at the computer, changing sites, varying search parameters, taking virtual tours etc. If your seriously moving then you do this every day so you can catch the latest properties. Is it addictive? Or does it just take hours because traditional house hunting search engines like Right Move, just don't work like we need them too? In my experience, In order to find that dream house, you need to enter parameters so wide that hundreds of properties are returned, almost all unsuitable and your left to trawl through each one looking for the one or two possibilities. In this day and age shouldn't the computer be doing that? Yes!

It's my belief that house hunting search engines are fundamentally flawed because they use boolean logic. Things are either true or false, 1 or 0. A house either has 4 bedrooms or it doesn't. It's priced under £400K or it isn't. But searching for a house isn't a true or false search. You may want 4 bedrooms, but a 3 bed house with room for an extension might be ok if the other factors were also strong. Boolean logic can't cope with this. Things get much worse when you add more and more search parameters in i.e. detached, 4 beds, 400K, Drive, Garage, study. Let's look at two approaches.

If you AND these factors together then only houses that have all 6 will be returned, unlikely.

If you OR these together and rank on number of matches (like a search engine) then you typically get hundreds of houses returned.

I think you'd get much better results if your search applied fuzzy logic. Fuzzy logic deals with degrees of truth, rather than absolutes. Rather than a yes or no, it's under 400K or not, yes or no. Its a 'yes, nearly, not quite, not really, no' type of thing. Rather than looking at criteria as pass or fail, 1 or 0, Fuzzy Logic scores them and come up with a holistic view. You score the search against the parameters and include near misses.

Estate agents would argue they been doing this for years, and given the increasing trend to do away with estate agents, they'd be wise to press this angle. If your estate agent is good you might get a call like this?

"Hello Mr Hanlon, its Laura from the Estate Agents. We've got a new property on the market that I think you may be interested in. It hasn't got the 4 beds that you wanted but it's in a really great location, large back garden, drive for 5 cars and the possibility to extend. I thought you might be interested. Give me a ring back when you can".

The estate agent here used fuzzy logic to judge that whilst this property didn't tick the boxes exactly, it scored highly across a number of categories to warrant looking at. How many people have ever given something 9.5 out of ten? or used a rule of thumb? All of these are examples of fuzzy logic in action. Our brains see things as shades of grey, rather than black or white. So how does this all relate to Business Intelligence? Well it's one reason why dashboard reporting can miss the mark.

OK but what do we learn from this.?

Estate agents could re-learn from retailers that's it's important to listen to your customer and give them what they want. They have the edge over search engines and they should work hard to exploit it.

Fuzzy logic is the way all things in the future will work. More of this later...

PS A friend of mine, Jamie Thompson, can also see the benefits of Fuzzy Logic, why not read some of his posts in his blog, SSIS Junkie.

User Centred Design and Business Intelligence

Traditional Business Intelligence was all about numbers, charts and reports and it's been replaced by super whizzy dashboards that promise the earth but may or may not deliver value.

Business Intelligence applications, along with email, are probably the only system that senior executives of companies actually use. As such they have visibility that's unrivalled from other IT areas and therefore why not make them a showcase of IT. A User Centred Design ( UCD ) approach can deliver a look and feel that exceeds peoples expectations, whilst avoiding the temptation to play with technology and overload on 'gimmicky' sliders and gauges.

Taking the time to sit down to map and understand the user journey can change the whole design of an application and deliver something that's really usable and adds value.

But isn't that just capturing the business requirements?

No, it goes beyond that. Understanding what the next move needs to be at each point creates new opportunities to link in deeper analysis, link to operational systems to action items as they are identified, memo issues so that they can be referred to later or trigger workflow tasks on the fly. Then have all of that designed by someone with years of experience in graphic design so that it looks fantastic. If you thought about it , why would you let a programmer design a front end that goes in front of the CEO?

Fantastic, I'll take two. Well, it's not that easy. Working with BI tools isn't the blank canvas that you have with traditional uses of User Centred Design such as web design. The tools have bounds and limits that you can't go beyond, and varying degrees of customisation. At the top end you could re-skin app's published in Sharepoint and have breathtaking state of the art look and feel. At the other end of the spectrum, you can make seemingly simple layout and design changes that transform mediocre reports into clean, professional, visually appealing documents that speak quality.

We use highly skilled IT professionals to build Business Intelligence applications, shouldn't we use highly skilled Creative professionals to design them?

Business Intelligence Dashboards : Are they all they're cracked up to be? Yes and No...!

Traditionally Business Intelligence is about numbers, charts and reports right? Big mainframe reporting systems using Focus or SAS, or even COBOL (those were the days, PIC S9(9) COMP-3, happy memories!!). Or early implementations of Microstrategy or Business Objects that looked good in their day, but seem out of place now.

The trend with technology companies for a few years now has been to make this front end graphical and 'fancy', and to sell the concept of a dashboard. Tools like Business Objects Excelsius and Microsoft Performance Point Server two examples of the leaders. One can't but help get the feeling though that technology has got ahead of the business requirements. Sure, we've all come to expect fancy graphics, but if these don't add any business value then what you end up with is 'gimmicky', 'fun' tools, that don't really add anything above charts and reams of numbers, or worse, deliver less. Charts after all did do the job for decades.

People like the dashboard concept because it promises summarised, easily understandable, relevant, actionable information. It's too easy though to under deliver on this promise. The idea that we can define rules that highlight exceptions, so that time spent analysing numbers is minimised and people are presented with the answer, is an attractive one but not one based on reality. 'Retail is Detail' so the saying goes and reading a retailers trading reports is not a simple business, if it was, buyers would have been replaced with Quants developing algorithms to manage the entire buy/supply process, and I'm not aware of any retailer doing this, or even remotely near it.

But let's not lose heart. That doesn't mean we can't get dashboards working. We need to concentrate on the business requirements and the business value and avoid the temptation to 'play' with technology. We can and should make the applications look good, processes like User Centred Design can add very significant value. Business Intelligence applications, along with email, are probably the only system that senior executives of companies actually use. As such they have visibility that's unrivalled from other IT areas and therefore why not make them a showcase of IT. Charts have their place and we need to play on their strengths and build in functionality that enhances them rather than supercedes them.

Too often the temptation with dashboards is to hide charts behind petrol gauges and sliders, why not have the chart of numbers as the centre piece and use it as the entry point into graphs, charts and deeper analysis.

Corporate Performance Management

A healthy and stable business finds the balance between, often competing, stakeholders. A point I try and make in a humorous way in my previous post about my local Off-License. Corporate Performance Management is the new name for what was referred to as the 'balanced scorecard' approach to executive management. This approach aims to balance the competing stakeholders such as employees and shareholders, customers and suppliers.

Limited companies need to provide a good return for their shareholders to do that they need the best staff, who like to be paid well. The more you pay your staff, the less profit you have to distribute amongst your shareholders, or the more you need to charge your customers for goods in order to make the profit, or the harder deal you need to negotiate with suppliers in order to minimise costs, etc etc.... So we have many stakeholders who all have individual needs, which often compete against each other, but to create a sustainable business they all need to be managed well.

Corporate Performance Management, or CPM for short, is about setting upper and lower boundaries for each of these stakeholders and their interests and then monitoring the performance of each to keep within these boundaries. It's CPM that we can blame for staff survey's, who want to know if 'My contribution is recognized by my manager', 'I have an opportunity for career progression' and 'the pay I get for my job is fair'. It's also behind Retailer/Supplier code of ethics and for the rise in Corporate Social Responsibility.

Whilst mostly CPM is used at the executive level, the principal applies equally well within departments. Shops need to balance the wishes of the customer in not having to queue, with the impact on the retail wage budget in having excessive staff manning the checkouts. An area where retailers invest heavily to create the right balance, as discussed in an earlier post on queueing theory. Inventory levels are another key area where the balance between stock and availability needs to managed closely, to minimise the costs of out of stocks and the lost interest of having money tied up in stock instead of in the bank.Sustainable business isn't "all about the numbers", it's a balancing act.

Off-License 2.0

I've introduced my local off-license and it's over friendly owner, Pete, in my previous post about the impact that merchandising has on replenishment. Being a small shopkeeper, Pete only has limited capital to invest in the business, so it needs to be highly targeted to where it adds most value. Pete still uses a pricing gun and an old fashioned till. It's obvious that this costs him a lot in time and effort, but it's his own time and he likes doing it so he carries on, the benefit isn't worth the investment.

This might seem a little old fashioned but in another way Pete is on the cutting edge of off-license retailing. Pete has just introduced the concept of off-license 2.0, social network retailing in beers, wines, spirits and confectionary. I mentioned before that the real purpose of the shop is to engage customers in conversation and Pete has now taken this to the next step by installing sofa's in his shop, near the checkout, to extend the social interaction both to customers and non-customers. Regular customers can now come in for a sit down and a chat whether they are buying anything or not.

It's hard to work out the impact on trade that this will have. I'm sure that the regulars now become even more regular and will often be tempted into impulse purchases when they'd no intention of buying. For others,me included, I'm less impressed with the innovation. Before the checkout process used to take 5 minutes and was a necessary evil. Now of course this can be extended as other people can chip into the conversation. Maybe the two balance out.

The benefit to Pete though, of this investment, is huge. Now he can hold conversations whether there are customers in the shop or not, as there tends to be a steady stream of loafers willing to come in and sit down. It's not all about trade and numbers, there are other factors, often competing that contribute to a healthy and stable business. Larger retailers are trying to balance shareholders vs employees, or customers vs suppliers. Off license 2.0 might not have had a huge impact on trade but it's certainly improved some of Pete's other KPI's and therefore worth every penny.