Retail Monster

The Seven Habits of Highly Effective DBA's

2009-11-20T18:36:00.009Z

I was just reading my usual daily digest from SQLServerCentral and it included an article on an interview with a scary DBA.

Now I've worked in IT long enough to have crossed swords with many a DBA. I even had the pleasure once of firing one, (and whilst no-one wants to see someone out of a job, I really do mean pleasure, what a ******* **** that guy was, and I hope he gets all that's coming to him.)

The interview rambles a bit, a one point asking which historical figures would have made good DBA's (!!!), but Grant Fritchey sums up very well the characteristics of a highly effective DBA. You can read the article here, but I quote Grant below;

The first one that comes to mind is grace under pressure. At some point in your career, you'll have a substantial number of managers, probably going very high up the chain of command, standing inside your cubicle. Typing will be difficult because they'll be jostling your elbows to get close enough to read what you're typing, even though they don't understand a word of it. It's all because, for some reason, one of the most important databases in the company is offline or inaccessible, or in an unknown state. You need to fix it quickly and competently, all the while explaining what you're doing in plain English, no techno-babble allowed, and suggesting ways to prevent the problem, whatever it is, recurring in the future. If you can do all this at the same time with those managers in your cube, then you're on your way to becoming an exceptional DBA.

I think in addition to grace under fire, Grant is also explaining several other traits;
Communication - they key to doing well in IT is to be able to communicate simply to non-IT people. So many people get this wrong, but I would almost put my entire success down to this one skill
Humble - Being a DBA involves power, Sysadmin, sysdba etc. People who get off on power make poor dba's. Senior managers are in your cubicle to get the problem fixed. It's your job, your not the worlds greatest man, your just a man who knows a password and some syntax. Remember that and you'll do well!

Next is a willingness to learn. Picking up the new TSQL syntax is only a small, although important part of that. The larger part is an understanding that things change, and a willingness to adapt as necessary. You might need to learn how to work with an agile development team, or implement a third party software package that requires 'sa' privileges, or move your database out into the cloud. It's a willingness to embrace the change, to learn, not just new processes, but new paradigms that will make you more successful as a DBA.

Flexibility is key. Standards, for example, are a means to an end, not the end in itself. If your standards say that apps cannot have 'sa' priviledges, then your business colleagues buy a product that requires them, then remember that they bought it for a reason. To make money for the company that pays your wages, so find a way of dealing with it!

Finally, I think a good DBA has to have a strong backbone. You need to be able to say no. You need to be able to say that the choice made on this database, if implemented, will cause problems, explain why the problems are going to occur, and offer a more-suitable alternative. You will often be under pressure to accept the solution "as is", but if you simply roll over, you're going to pay the price. It won't just be you though; it'll be your database and possibly your company.
There are certain issues on which a DBA must never compromise. For example, there's a good reason why you should do backups before a major modification of the database, and just because someone within the organization thinks it's a silly waste of time to run one, and doesn't want to wait the extra 45 minutes, tough. It's your charge as a DBA to ensure the integrity and protection of the databases under your control.
If someone asks for something silly, say no. You may have to do it anyway but get out there and make sure people understand why you said no. And it shouldn't just be because you want to be able to say "I told you so." If that's why you're doing it, stop being a DBA. You do it so that maybe, the next time, they'll listen.

SUITABLE ALTERNATIVE!! As one of my best colleagues once said, good design is the result of furious debate. If you don't like something, suggest an alternative THAT WORKS. I couldn't count on both hands the number of times I've wanted to physically hurt DBA's because of their 'veto everything that different' attitude that contributes nothing.

If your a DBA, and people hate you, then perhaps you should try incorporating some of these ways of working into your day. Start adding value to the business, and above all, remember that your just a someone who knows a password and some syntax!

SQL Server Include Columns

2009-11-19T19:58:00.018Z

I've just been refreshing my memory on the basics of indexes after my daily email from SQL Server Central arrived, advertising this excellent beginners guide to SQL Server Indexes from Gail Shaw.

I think it's a great article, covering all the essentials in a clear and precis manner. My only criticism was that she skipped over the value of INCLUDE COLUMNS, without introducing them in the first place.

I'm a well travelled Database professional having started on DB2, moved to IMS, then Teradata before arriving at SQL Server. I'm fortunate to have been able to pick-up SQL Server very quickly through my good friends James Rowland-Jones and Jamie Thomson (aka SSIS Junkie), both SQL Server MVP's. (and if your in the UK, James is speaking at SQLBits, check him out)

I recognised INCLUDE COLUMNS from my Teradata days, and whilst maybe not exactly the same, the Teradata Join Index and even more impressive Aggregated Join Index are similar in that they allow you to specify columns as part odf the index (not the index key), that allows the query to be satisfied from the index alone, without resort to the base table.

Because Gail didn't really cover Include Columns, I did some research on the web and was a bit surprised that the majority of the SQL Server community considered Include columns as a way of getting around the 16 column and 900 byte limitation on a sql server non-clustered index. Despite the fact that almost everyone agreed that these limits are highly unlikely to be reached in normal usage, something that with my 15 years of experience, I'd agree with.

The real benefit of include columns is being able to elevate a few select columns from the table to the index, such that the index read alone can satisfay the query. I work in retail (retailmonster get it!) so for example you may have some individual basket data, ie everyitem an individual customer buys in a single trip. This is retail gold dust but also incredibaly space and CPU heavy. You might split this into a basket header record, with an index of storeid/calendardate, but since a lot of people might want the total retail value of the basket, or the number of items in the basket, if you specify totalbasketretailvalue and basketitemcount fields as include columns, then a query asking for the index columns and only these fields, would be satisfied by the index read and not have to go to the table.

James Rowland-Jones has just authored a new book, SQL Server 2008 Internals, in which he covers the benfits of include columns for SQL server.

The Effect of Company Culture on Application Development

2009-11-04T20:14:00.004Z

How the underlying culture of a company affects design.

My early professional career was all in large, well establsihed (FTSE100) companies, and therefore my early experiences of IT were shaped in what I like to consider a very rigourous and disciplined way. I like to think of myself as old School and proud of it.

However since becoming a consultant I've worked at varying sizes of companies and have therefore become exposed to other ways of working. What strikes me most is how much the underlying culture of the company pervades every way of working. Like some secret predicate that is present on every decision.

Two examples.

I worked for one of the largest retailers in the world for over a decade, where the underlying culture was global expansion, scalable, repeatable etc. This affected every decision in application design and no-one would ever dare not take a decision that was pragmatic or tactical, however appropriate that could have been.

More recently I've worked in a startup organisation with less than 20 employees, where money is tight, the revenue stream is far from garuanteed and it's the corporate equivalent of a hand to mouth existence. This means that every IT decision is pragmatic, tactical, low cost, here and now.

Now in both instances, it was often not an explicit part of the requirements that the solution needed to meet these underlying requirments. It's just that everyone conformed. Further more, the culture wasn't necessarily purposefully dictated. ie the culture formed on it's own, regardless of whether it was in the best interests of the company.

So beware of the underlying culture of an organisation as it might lead to architectural and design decisions that aren't appropriate, just because every does it that way...

Maximum Value Safety Nets

2009-11-03T16:25:00.002Z

Related to my previous post about parameters in application development, I've recently been incorporating safety nets into my applications to catch those unusual data occurrences that can result in very abnormal results of calculations. Results far outside what was expected and that can have a dramatic effect on the final output of the system.

I've built in safety nets that will trap any outliers and limit them to a maximum, or minimum value. As you've guessed, these safety nets are defined at the grain of that fact to give the maximum level of control of the system.

Hindsight is a wonderful thing and I wish I'd put these in at the start as although the number of these issues that have surfaced have been small. The pain they've caused me has far outwieghed the effort to have designed them into the system in the first place.

The biggest pain has the credibility issue that these instances have caused. We'll all be familiar with the newspaper headlines about people being charged thousands of pounds by mistake (in fact there was one today.. man loses driving license for doing 383mph on a motorbike). These isolated issues, which may be 1 defect in millions of transactions, cause people to lose faith in the system. These can easily get blown out of all proportion by anyone with an axe to grind against a new system.

'If this numbers wrong, then maybe they're all wrong...'

Clearly not the case, but how many you out there have faced this situation.. Safety Nets..!

Parameters in Application Development

2009-11-03T15:54:00.002Z

This is my first blog post in months and been spurred on by my good friend JRJ blogging here about his latest consultancy jolly to Seattle for PASS09.

I've been getting my fingers dirty for the last few months in hardcore application development. A bit of a blast from the past for me as I've mostly been management for the 5/6 years, but you'll be pleased to know, I've still got it when it comes to application development.

This bueaty of a project is pretty hardcore. A complex statistical database app crunching hourly data for a retailer (no surprises there). Given my background in repeatable application deployment, the application is massively configurable by parameters. However where we've ended up is very different from where we originally started, and has caused me to re-consider my whole approach to defining parameters for applications.

To cut the story short, we originally had parameters defined at higher level, ie system wide parameters. But in almost all instances have now dropped the granularity of the parameters, to match the grain of the fact. This means we have a lot of parameters! and you don't want to be typing them in by hand (or adjust them manually), so along with the increase in granularity, comes along a little sql engine that we use to set and adjust the parameters. If you use SQL to set the parameters, then there's little difference in overhead between a single row where parameter x = 12, or 25000 rows where the value of x = 12 ie

Update Parameter
set x = 12;

You could consider this an overhead, but if you build it in from the start, you never know any different and you build the system to work this way. Then if you need to start setting seperate values, you have the functionality and can go do it. I'm sure this is less effort than changing your mind during the build, or worse, after go -live.

I'm so sure in fact, that this is now my new default approach to designing parameters into applications, and I'll be testing it out in anger as I've now taken on a another apps dev project where I can again flex my system designing muscle.

Reduce, Re-use, Recycle

2009-07-09T10:02:00.003+01:00

I have never thought of myself as a horder; it’s just I never throw anything away. This causes a few problems, since we haven’t got a lot of space at home to store many things, so these days my stash of useful of oddities is limited by the size of the shed. As a result, and like any good shed, it’s a work of organisational art ferreting away little bits and bobs here, there and everywhere.

Obviously DIY knick knacks rule the roost. I’ve got rawl plugs in every conceivable size, shape and colour, nuts without bolts, and more flat pack furniture alan keys than you can shake a stick at. I keep all these, knowing more than hoping, that one day they’ll come in useful. My latest re-use of which I’m particularly happy is the cutlery basket from our old dishwasher, which I now use as a beer holder, to ferry up to 8 beers from the shed to the fridge (warm), or fridge to patio (cold). Anyone who can remember their last time in a nightclub will know what I’m talking about.

Over the last couple of years it’s my cycling collection that’s grown the most though, and even though I’m a beginner by most people in the clubs standards, I’ve already amassed an enviable collection.

At the last count I had 7 spare cycle pumps. Spare that is. I already have one on each bike (4 bikes). 4 spare saddles. A surprising 5 sets of spare peddles (2 clipless (SPD), 1 cage, 1 flattie BMX and 1 normal). 11 Inner tubes, OK these aren’t strictly spare, I will throw a tube away once I’ve patched it more than 10 times, and with 4 bikes plus kids it’s probably only 2 or 3 in each size. (See there I go justifying it). Before Christmas though I was able to help an old friend out who was getting his first clipless shoes as a present. Out came the spare pair of shoes, cleats, pedals, a pump. “what’s that, you don’t find you saddle comfortable?, take your pick..!”, so I was right all along to keep them.

Far from being a hindrance, my growing collection means I’m able to use my DIY spares to help store it all! One of my bikes hangs from the ceiling on hooks made out of some thick wire from a wardrobe, wrapped in foam from the sleeves that my wheels came in, (after strengthening the roof with some battens left over from a previous job). But the ‘piece de resistance’ in re-use, is a pair of habitat beech veneer CD holders about 2' by 4', together, in a grid formation designed to hold say 6 or 7 CD’s. I salvaged them after my wife decided they ‘didn’t go’ anymore after we decorated. These little beauties are perfect for storing all of my kit. Each grid is just big enough to stuff a few useful items in. I have my overshoes in one (2 pairs), gloves in another (4 pairs –winter, normal, fingerless * 2), lights, tubes, tools, lubes etc fill the others. It actually does a better of job that it ever did with CD’s.

I’m sure many of you reading this have been able to relate to this, perhaps you can share a few tips. If you’ve never tried it, come on in, you don’t know what you’re missing. Next time you see that old broom handle leaning against the dustbin for collection, put it away in safe place, you never know when it might come in handy...

P.S. that reminds me of the time I had to move a metal shed from one end of the garden to the other, I'd had these two broom handles I’d been saving for years, so I lifted the shed up and put them underneath and rolled it down the garden... happy days...

Understanding Confidence in Teradata Explain Plans

2009-06-12T11:23:00.004+01:00

I'm a big fan of Carrie Ballinger's Tech Support column in the Teradata Magazine. Carrie is a Teradata Certified Master and works in the Teradata Performance Centre in Los Angeles.

Her latest article came as a revelation to me because I've spent many years working on Teradata systems and I thought I new about Confidence until I read the article. The following is an edited summary, the full article is available here.

Confidence levels indicate the degree to which Statistics are (or are not) available for a given query step. Confidence is driven by statistics and the more complete the statistics, the higher the confidence level will be. Both high and low confidence are perfectly acceptable - both are treated the same for optimizer costing purposes.

Confidence takes the form of four discrete values

High Confidence

High confidence will appear on a step that performs database access if a single predicate exists in the query and statistics exist on the predicate AND the primary index.

Multiple columns in the predicate are allowed, as long as multi-column statistics cover them.

Low Confidence

You get Low Confidence if usable statistics are present (either from collected statistics or random sampling).

You'll also get low confidence if;

Multiple predicates appear in the query for the table being accessed.
No PI statistics are collected
The step is a JOIN step

Index Join Confidence

Index Join Confidence will appear in instances when an index is present on one side of the join, but no index or statistics are on the other side.

No Confidence

No Confidence is assigned when no usable statistics exist for a step. No confidence trickles down to subsequent dependant steps, even if they have usable statistics. Subsequent steps may therefore display as No Confidence when in fact they are making use of available statistics.

Thanks to Carrie for shedding light on this.

Chilled Distribution Centres

2009-06-08T13:32:00.002+01:00

I was lucky enough today to be taken on a tour of a chilled and frozen distribution centre. I've worked in retail for more than 15 years and been on a few DC tours in my time, but I was particularly excited at the prospect of this tour, as it was my first trip round a chilled DC.

Depots really are amazing places. I was particularly pleased to see pick by line in operation, since traditional warehousing usually involves pick by store, which I've seen a number of times. Pick by line works by taking a cage of a single product, and delivering it to the cages of a large number of stores. This type of picking is used in DC's that don't have any racking and where the goods that come in, go out the same day.

The technology has moved on a bit since my last trip too. Of course the picking is paperless, I expected that, but gone are the arm-mounted terminals, it's all done via voice activated headsets.

Voice picking, reponds to the instructions given by the picker and delivers the instructions to him/her. Working down a line of store cages, the picker speaks the name/number of the store and is told the number of units to pick for that store. It attaches quite nicely the hard hat baseball cap which is another nice touch.

This particular depot also had a large frozen section. Operating at about -25C, warm clothing is a must. The guys who work in here 8 hours per day are all clad in salopettes, big jackets, hats, scarves, gloves etc. This section is like a traditional ambient depot except for the temperature. Standard procedures of goods in, put away, letdown and pick by store take place.

A fascinating experience...

Queuing at Disneyland Paris

2009-06-04T16:01:00.003+01:00

I'm fascinated by queuing and how organisations can manage them better, or avoid them completely. I've just got back from a weekend at Disneyland Paris, an experience that gave me plenty of food for thought on the queuing front.

I assume that Disneyland Paris is the busiest amusement park in Europe (I haven't checked) and therefore there isn't huge scope to avoid queues in this scenario. It's very much about how best to manage them, and Disney have thought long and hard about this and come up with a number of tricks.

Queue Time – All rides have a queue time sign at the start and for the most part this was very accurate. This allows you to make an informed decision before you join the queue on how long you’re going to be in it. In a subtle way, this shifts the responsibility onto yourself. You can’t complain about queuing for an hour, if you knew in advance that it was going to take that long.

The long and winding queue – For the most part the queues are unnecessarily windy. This means that you can’t normally see the route of the queue. Even if it’s in one big block, which means you are never quite sure how long the queue is. Even if you know it’s a long queue, it somehow doesn’t feel as bad when you can’t see it all laid out in front of you.

The double queue – This one’s a bit sneaky and gives rise to a few grumbles. You queue outdoors for a period of time and then when you get to the doors and go inside, thinking you’re at the front, you find another queue inside just as long and you realise your only half way. Personally I hate this. They use this in NHS hospitals quite a lot too. You turn up for a 2:30pm appointment and take a seat in the waiting room. At 2:30 they call your name and you start to think ‘Hey this is good’. They then take you round the corner to a smaller waiting room, (with more people in it than the larger one you just left, work that one out ) where you spend the next hour or two.

Making the queue part of the ride – This is a good distraction technique which ranges all the way from the scenery in the queuing environment, through video’s, sound animatronics etc all the way up to the pre-ride briefing.

Pre-ride Briefing – The designers might explain that the pre-ride briefing is about creating mood, telling a story, explaining vital safety information. Don’t be fooled. The pre-ride briefing is almost certainly followed by another queue. They’ve distracted you then hit you with the double queue strategy and you probably haven’t even noticed.

Virtual Queuing – Also called FastPass (Qbot at Legoland), is the latest development on the queuing front. You take a ticket that allows you to turn up at a ride and walk straight to the front of the queue. You’re usually only allowed one of these at a time. This allows you to queue for the ride without actually queuing for it (virtual queuing in fact), which isn’t really queuing at all in my book. This is a great idea in principle. In practice it’s a bit complicated which means there are winners and losers, those who can understand and milk the system and those who can’t. In one of my previous posts on queuing I’ve highlighted before that queues cost money. Fastpass requires two queues and the management of them and so takes up twice as much money (in simple terms). Plus it also generates bad feeling in those who have waited those glorious 90 minutes in the queue. No matter how well you understand and accept the system, I defy anyone to be happy about someone walking to the front and taking your space. You can’t get rid of the feeling that you’ve been cheated, a feeling that lingers because of the generally flimsy nature of the fastpass queue management. It’s easy to slip into Fastpass without a ticket, either at the start when no-one is watching, or later when you can usually just hop over a rail.

Introduction to RFID Inventory Management in Retail

2009-02-10T14:24:00.008Z

Radio Frequency Identification - RFID is an established data-carrying and automatic identification technology used throughout industry, and in the retail sector, has long been touted as the Holy Grail of Inventory Management. I can remember a conversation I had with one of the IT Directors of Tesco who said to me '..whichever Retailer cracks RFID first, wins. Period'. Dramatic words indeed.

Within Retail, think of RFID as an Intelligent barcode. Intelligent because it not only identifies the product, but it uniquely identifies the product. ie I'm not just a 300g tin of beans, I'm 300g tin of beans number 54167. It can do this because data relating to the specific item is stored on the RFID tag which is attached to the item. Like a bar code, a tag is a data carrier. A bar code carries data in a visible symbol and is read by a bar code scanner using optical or infrared wavelengths. An RFID tag carries data programmed into a small computer chip and operates at a wide range of radio frequencies.

The tag is activated by radio waves emitted from an RFID reader. The reader communicates wirelessly with the tag across what is known as the air-interface. Once activated, the tag sends data stored in its memory relating to the item back to the reader.

The RFID readers vary in the range at which they can read the RFID tags. This starts from the tap and go type readers which operate at the 0 - 1cm range, think TFL Oyster card, where you tap the card on to a reader (Interestingly such system are called contactless, despite the need to touch them to the reader). This area of RFID isn't really suitable for Inventory Management and is being explored more as a quick payment method.

Long range RFID scanners can pick up tags at range's up to 200m, and its these long and medium range scanners that open up the opportunity within warehouses and store backrooms for automatic inventory counting, goods in scanning etc. Imagine being able to take in a delivery from a supplier and automatically know each individual product that is on the pallet.

Aside from the infrastructure and setup challenges associated with an RFID solution, is the challenge of what to do with all that data. The increase in data volumes associated with a change in supply chain management from pallets to individual items is huge. I've worked at 4 out of the top 5 UK retailers and they all have enterprise datawarehouses measured in the 10's of Terabytes, driven by holding data mostly at SKU level. (Some of the data held will be at transaction level, which is almost individual item level, but the volumes of this typically range in the 0 - 5% of total space utilisation). To change the granularity to be at individual RFID rather than SKU is to scale that volume by a rough factor of 10,000. (based on 1000 stores and 10 incidences of each item per store).

The data challenge for RFID Inventory Management therefore becomes how to cope with a new level granularity, which systems need to use it, how they talk to other systems, how to cope with the increased network and storage requirements.

Grapes are a vines babies

2009-01-29T08:59:00.005Z

A thought occurred to me yesterday while I was working with my operations team on a reasonably complex data issue. We were waiting for the results of a program to come back and I was nibbling at the grapes in the company fruit bowl. For those of you that don't have office based lives, the latest must have accessory is company provided fruit. Fresh fruit is delivered to the office every other day, and staff get to munch for free. It's a HR thing aimed at us all leading healthier lives. I'm all for it.

So I'm sitting there eating grapes, big fat juicy seedless grapes, the best sort. I felt a bit guilty that they'd come in from Kenya via Air freight, not so environmentally friendly, and I felt my carbon footprint flex ever so slightly.

I'm eating grapes and suddenly the thought occurs to me, that these grapes are the vines babies. Grapes are the equivalent of eggs surely. The plant-world equivalent of an unborn foetus.

I'm not a vegetarian. I eat meat and dairy, so really this shouldn't matter should it. Milk is not the innocent white-coloured water we pour over cereals, but the fluid used by a mother cow to feed baby calves. Eggs are unborn chicks.

If vegetarians don't eat meat and vegans don't eat meat or dairy. What do you call someone who doesn't eat meat, dairy or fruit and vegetables??

Answers on a postcard...

The True Cost of Cheap Food

2009-01-22T20:10:00.015Z

A dispatches investigation into how the supermarkets are cashing in on the credit crunch by encouraging the trading down to own label value lines.

I find myself drawn naturally to programs that slag supermarkets off (we all know it's fashionable to be anti-supermarket). I've worked in retail for more than 15 years, including more than 11 years working for an international retailer, so as a retail expert, you could say it's part of my job to keep an eye on programmes like this. I find myself usually both agreeing and disagreeing to the various points in the show, and the degree to which this happens depends greatly on the quality of the show.

This particular program is better than many other 'supermarkets are really bad' type shows. It was full of stats, many of which I captured and I've documented at the end of the post, and it's main point was that cheap food can be improved very easily and very cheaply, usually for less than 1 pence per sausage/apple pie/cheese slice.

It suggested that supermarkets should improve the quality and take the hit on their own margin. I'm not sure I agree with that and given that we're only talking about increasing the price of a pack of 8 value sausages by 8 pence, why should they. However, since that 8 pence produces a significantly better product, I rather think the show should have stressed the point that supermarkets should wherever possible, provide the best quality product. They're allowed to make a profit out of it.

I'm toying with the idea of doing a Hugh Fearnley-Whittingstall and raising a motion at the Tesco AGM to force them to improve the quality of their sausages. If enough people comment on this post in favour of this action I'll do it. Pass this on to your friends...

Now for the stats

Beef Burgers
How much protein is their in premium vs economy burgers?? grams protein per 100g

Asda - premium 23, economy 20
Tesco - premium 23, economy 19
Iceland - premium 25, economy 18

Tomatoes
How much of the anti-oxidant lycopene in premium vs economy tomatoes??

Tesco - premium 5.5, economy 3.4
Asda - premium 6.5, economy 3.6
Sainsburys - premium 5.2, economy 3.6
Morrisons - premium 5.1, economy 4.1
Aldi - premium 5.1, economy 3.4
Lidl - economy 3.8
Iceland - premium 5.5, economy 3.3

Chicken breasts
The percentage of meat in chicken breasts.

Asda - 82%
Iceland - 85%
Morrisons - 88%
Tesco - 84%

The remaining percentage is made up of varying combinations of the following water, salt, stabilisers, dextrose, polyphosphate and liquid glucose.

Sausages
The percentage of protein in a sausage. (protein is an indication of meat content)

Tesco - premium 15.8, economy 11.2
Asda - n/a
Morrisons - premium14.3, economy 9.0
Sainsburys - premium 15.3, economy 8.3
Iceland - premium 12.0, economy 11.6
Aldi - n/a
Lidl - premium 15.3, economy 12.9

Data Visualization Golden Rules

2009-01-09T19:36:00.004Z

Last week I posted about a friend of mine, John Brookmyre, who blogged about getting started with cloud computing. It's a great post and John's blog is something that's definately worth keeping an eye on, because he understands and can communicate business intelligence very well in addition to being able to deliver technically in some 20+ platforms.

John's latest post, is about business intelligence data visualization, something that I've been very interested in for the 2 years, and have written about my experiences in the following;

Interestingly, Johns post has been commented on by Tableau Software, who are niche players in the BI market, but among the industry leaders in BI data visualisation. You can trial the software, which I did about 9 months ago when I was looking for alternatives to Microsoft Performance Point Server.

On that occasion the client purchased PPS instead, which we deployed via a customised sharepoint front end. We got round the limitations of both PPS and sharepoint with some clever visualisation tricks and the result was stunning. I'd been reading Stephen Few's blog for a while, (he's a modern day Edward Tufte) and also had some recent experience of heuristic evaluation, which meant I was well up to speed on usability and visualization. If your interested in getting into the subject then these are good places to start.

Back to Tableau though. It's a very interesting product and I wasn't able to get through it all in the 28 days. I managed some very pleasing visualisations involving multiple graphs in a grid formation. The advantages of this being you waste very little space on title's and legends as you benefit from a write once, use many policy. I managed to get 25 graphs on screen (5x5) in a layout that wasn't off-putting, and I could see it working for business users. I've pasted in below an example of the tableau website, that demonstrates the type of visualisation I'm describing. (I like to think that mine was better though!)

Compostable Plastic vs Recycling

2008-12-16T17:43:00.002Z

Composting is a form of recycling. Plant based packaging materials (environmental packaging) are recycled into compost which can then be used to grow more plants. Similarly recycling plastics is a resource efficient process where energy can be adequately recovered- there is no need to adopt an either or approach!

As the realisation dawns that we cannot keep consuming without due consideration, it would seem prudent to explore as many options and strategies for reducing our environmental impact as possible. There is no one solution or silver bullet. Different forms of recycling may be appropriate for different situations. For example where collection of plastics is possible and the technology and machinery exists within short distances to reprocess it, this may present an appropriate option.

There are around 50 grades of plastic and currently the UK has facilities to reprocess two of them- HDPE (milk bottles) and PET (fizzy drinks bottles). Often recycling schemes may accommodate the collection of just these two types of bottles to avoid contamination from different grades of plastic leaving a vast array of oil based packaging that does not even have the opportunity of being recycled.

Similarly recycling of paper and card has proved an effective and efficient way of recovering energy. However paper used in food packaging can often be unsuitable for these schemes as the paper will firstly be lined with a plastic and secondly may contain food residue that might contaminate the recycled batch of paper.

Compostable packaging can provide an effective solution and since they are all compostable, the paper, bioplastic and bagasse products can all be recycled together. Furthermore the food residues do not have to be separated either as they are compostable too and will not contaminate the end recyclate. This can prove particularly useful in acting as a tool for the diversion of biodegradable waste away from landfill.

Getting started with Cloud Computing

2008-12-15T19:00:00.003Z

A friend of mine, John Brookmyre, has recently posted about Amazon's Web Service Database and his first tetntative steps into the world of Cloud Computing.

I'm a bit jealous really as I've been thinking a lot about the applications of cloud computing within the retail sector, but haven't actually taken anything further than initial thoughts. My own fault really, so hats of to you John.

I can think of several ways in which Live Mesh can be useful. I struggled in tha past building an e-catalogue for clothing buyers to take with them on buying trips to the South East. The catalogue functionality was straight forward, but updating the dataset that needed to sit locally on the machine (access to the internet was less than robust in those days) turned out to be a can full of worms.

Back then I could only dream of having a Live Mesh folder synchronised for the buyers so that all they needed to do was be connected to the internet, to receive the latest update.

I haven't really got into the detailed techy bits of cloud computing such as SSDS but another friend of mine, Jamie Thomson has got several examples of SQL Server Data Services he's built. Read about them in his blog, SSIS Junkie.

Biodegradable Plastic

2008-12-04T14:31:00.002Z

Anyone whose followed my series of posts on Carbon Labelling, Plastic Bags and anything relating to green credentials, will know that I'm more than a little interested in the environment. More interested observer than environmental activist, I would say.

I like to do my bit aswell and the various bins outside the backdoor are testament to our residential recycling effort. Not everything's recycleable though and last week after getting a take-away, I ended up throwing about 7 - 10 little plastic food containers away. It didn't feel right somehow. Not only was I thinking guilty thoughts about the environmental impact but I was worried about how much room would be left in the wheelie bin now we're on fortnightly refuse collections.

"Surely someone can do better than this" I murmed, so I hit google to see what it had to offer.

Interestingly I came across a company offering fully biodegradable 'Plastic' food containers, the advantage being that you can throw them away with the food scraps and it all composts together. Nice. Who hasn't just thrown away a recycleable item before because it's too dirty (or your too lazy) to put it in the bin. Which reminds me of my pet hate about recycling, but more that later...

An extract from their website below

Camden Primary Care Trust and Packaging Environmental have collaborated in an initiative to replace all of the disposable food and drinks packaging at the Trust’s restaurant with sustainable alternatives. Whilst everything has been replaced with high quality plant based materials, the scheme has also involved the collection of all the restaurants packaging, and its disposal into the local commercial composting site. From burger boxes to coffee cups to salad containers, everything is being recycled back into compost and ultimately diverted away from landfill.
The compostable nature of the packaging means that the associated food scraps do not need to be separated out either, but can be collected together with the packaging and similarly recycled into compost.

Corporate Social Responsibility and the economic downturn

2008-12-04T13:35:00.003Z

Given the unstable nature of the world economy at the moment, how will the latest corporate trend, Corporate Social Responsibility, fair as companies tighten their belts and look to cut costs?

All major retailers have Corporate Social Responsibility (CSR) statements proudly displayed on their web sites, and being green and being seen to be green, is high up the agenda in all PR and Marketing departments. Retailers are fighting to have sector leading green credentials with the humble carrier bag thrust into the limelight, centre stage. Corporate Social Responsibility is not just about being green though, it's about promoting sustainability and one of the leading bodies to define standards for business sustainability, the Global Reporting Initiative, defines five areas where business needs to focus;

o Economic
o Environmental
o Human Rights
o Labour
o Product Responsibility

With the price of energy increasing all the time, it's easy to see that focusing effort on reducing your Carbon Footprint, is good business sense, as well as providing good PR material. The green movement has successfully persuaded companies that being green makes commercial sense. However being green is only a part of a companies CSR strategy, so as times get harder will we see a relaxing of the commitments made across the other key areas.

With sales weakening across the high street many retailers will be asking their supply chain to bear the costs of poor trade. Changing payments terms for suppliers and asking suppliers for extra discount are some of the ways retailers can put the squeeze on their suppliers. For suppliers though, struggling with their own increased energy costs, this additional pressure will be most unwelcome and for many, could push them close to the edge.

This doesn't strike me as a sustainable business strategy. Sure, your suppliers will need to bear some of the costs, but the sustainable way out of the downturn would be to work together. How sustainable can it be to destroy your supplier base?

Managing Client Expectations: Deliver and Delight

2008-12-04T08:46:00.001Z

In my experience of many years of working with clients in the retail industry, I've almost naturally settled on the principle of 'Deliver and Delight'.

Many of the projects and solutions that I've delivered for clients over the years, they could have done themselves, and at a much cheaper rate. I like to think even though the cost in the short term may have seemed more, the client has got value for money, and clearly this must be true to a greater extent or I wouldn't have been successful. So why, when a client can deliver these solutions themselves are they paying external consultants and more importantly, why do they feel they got value for money.

When I say that clients could deliver the projects themselves, there's nothing stopping them of course, but it isn't quite as simple as that.

- They'd need to recruit the right team with a good mix of technical and business skills.

- They'd need to create a culture that allowed ideas to flow and encouraged experimentation.

- They'd need to learn from their mistakes, for they would certainly make a few along the way.

- They'd need to do all this in the timeframe dictated by the opportunity.

Not looking so clear cut now is it....

Explaining the benefits and winning a contract means nothing without delivery though. Delivery stands at the core of consultancy or service provision. My reputation stands on the strength of delivering against the promises that I make. My technical and industry experience allows me to make those promises. Having built the best team allows me to deliver against those promises.

But as I explained in an earlier post on business requirements, to raise yourself above the rest, you need to do more than just deliver. Deliver and delight.

Delight means going further than the deliverables, it means making those years of experience count by doing more. It's delivering the basics and then some. Creating strategic relationships, helping the client shape their vision, bringing in wider industry knowledge, bringing in ideas... I could go on but you get my point.

If your interested in working with an organisation that goes that bit further than everyone else, then get in touch.

Business Requirements: Delivering Unspoken Requirements

2008-09-24T12:38:00.000+01:00

If you've read one of my previous posts about Fuzzy Logic, then you'll know that I was thinking about moving house earlier this year. Without boring you too much about my personal life, we're not going down that route anymore, instead we've been talking to architects and looking at doing a home extension. We live in a Victorian house full of original features (I sound like an estate agent !) and as a result we have a downstairs bathroom. One of our requirements is to achieve an upstairs bathroom. According to the beenster, an en-suite bathroom is one of the top 5 requirements requested by home buyers. Our architect, in trying to maximise re-sale value, is suggesting that we create a bathroom with two doors, one opening onto the landing and one opening into the master bedroom. This is called a Jack and Jill arrangement. Technically this means we'll have an en-suite, a tick in the box when selling a house.

Now I know the architect is trying to do us a favour, but I can't help but feel that whilst this technically meets the description of an en-suite, it doesn't meet the 'spirit' of an en-suite. (ie it's not an en-suite)

I can draw parallels with my job in IT. We often have critical success factors on projects that are hard to articulate. In my experience sometimes these can go unspoken, but everyone understands what they are. 'Look like this', 'behave like this', 'be better than that', 'delight me'. These are just a couple of examples in my head at the time of writing this. We often try to capture these and call them non-functional requirements. Sometimes we capture them well, other times less so and sometimes we don't capture them at all (I like to call this flexibility). Even when we capture them, we can fail to deliver on them, and when we don't, we can try and justify ourselves by claiming we've met the 'letter' of the requirement, when we know we haven't delivered in the 'spirit' of the requirement.

'If people don't use the system it's there own issue, it does everything they asked for....'

I've had a recent experience of this in a project where time was the most critical factor, ahead of cost and quality/scope. We had the ability to flex the scope in order to meet the date, but this soon got to the point where we knew we'd compromised the 'spirit' of the requirements, but we were trying to kid ourselves that we'd met the 'letter' of the requirements, so it was alright.

When I was a younger, less experienced project manager / business analyst, I would have carried on regardless, feeling like being unable to deliver the 'spirit' of the requirements was a personal failure and something to be covered up and ignored. Having been burnt several times by this approach ( I got away with it a couple of times too), I now know that raising these project issues early always leads to a better outcome in the long run. It can of course mean some short term pain from management and stakeholders as they realise they're expectations aren't going to be met fully, but the benefit of raising this early is two fold

Stakeholders get a chance to be involved in a resolution plan, so you get good stakeholder involvement and buy-in to the resolution. (When issues are raised late, your usually very limited on the resolution.)
Stakeholders expectations get managed gradually, so each incremental change is less painful than a big realisation at the end of a project

So with the issue over requirements raised, I'm probably in for a period of short term pain. I've learnt it's better this way though than to carry on regardless...

Corporate Social Responsibility Reporting

2008-09-24T12:33:00.000+01:00

Corporate Social Responsibility (CSR) is becoming ever more important. Some recent headlines;

Tesco is to test putting "carbon labels" on its own-brand products next month in a move to enable consumers to choose products which are less damaging to the environment. Guardian

Marks and Spencer is to begin charging its food shoppers for carrier bags. BBC

Of course Corporate Social Responsibility reporting isn't new in itself, it's the increased focus and level of detail that is changing. As a result, companies will report in ever more detail on these issues and refine their definition of Corporate Social Responsibility. CSR is an increasingly important part of a company’s annual report and CEO’s are making ethics and sustainability claims that need to be backed up. The trend therefore is that CSR data is being treated like financial data which requires greater accountability and accuracy in the reporting of Corporate Social Responsibility objectives, increased levels of corporate governance, and stronger internal control documentation across the enterprise. On the back of increased CSR reporting will come increased auditing requirements as mandated by good corporate governance.

Several governing bodies have arisen to provide guidance and best practice such as the Global Reporting Initiative (GRI) and the World Business Council for Sustainable Development (WBCSD). Each of these has their own protocol for calculating and reporting environmental KPI's such as carbon emissions. Invariably these rely on excel spreadsheets that are manually entered, in disparate locations around the field of operation. This is going to become less and less acceptable.

Corporate Social Responsibility data will need to be audited and we'll see security, workflow and audit trail issues seeing increased focus in the capture and reporting of CSR data. Here at Retail Monster Consulting we can help you deliver CSR reporting in a robust and usable manner through Business Intelligence (BI) technology. Our CSR dashboard and data model experience, can help you fast track a CSR implementation.

We are actively blogging about CSR related topics such as Green Credentials and Carbon Labelling.

If any of this interests you then please talk to us

Queue theory can help manage retail queues

2008-09-24T12:28:00.000+01:00

Queues are bad. Really bad. They literally cost you money. One of the principles of queue theory is that once a queue forms it has to be managed, which requires resources, then you need over capacity at some point, in order to reduce the length of the queue. Often the resource to manage the queue comes from the very thing serving the queue. This means the queue gets processed even more slowly than before the queue started and the queue gets longer. It doesn't start to get shorter until you have over capacity.

Retail queues cost money in all kinds of ways and often those costs are hidden, or at least not immediately apparent. Long queues lead to customer dissatisfaction and ultimately to lost customers. Losing customers obviously isn't a good thing and will cost you money in the long run, and it's in this respect that most people associate the cost of queueing. (Queueing, incidentally, is the only word in the English language to contain 5 consecutive vowels)

Some less obvious ways that queues cost money involve

o In a supermarket, the space kept clear between the checkouts and the start of the aisles is valuable space that is being used to manage queues instead of being used as shelf space.
o Two delivery lorries arriving at a store/depot at the same time cost money in waiting time.
o Or a more trivial example, taking a ticket at the meat counter involves the cost of the little yellow tickets and the dispenser

Understandably therefore, minimising queues is top priority, and as something that has big impact on customer dissatisfaction, it's something of a PR tool in the supermarket wars. The trade off is balancing the cost of resources used to prevent queues, vs the cost of the queue itself. Having all checkouts open all the time would go a long way to easing congestion in the store, but that has to weighed up against the retail wage budget, (the single biggest expense for a large retailer), used to pay all of those checkout operators required to ensure no queues.

Retailers understand this trade off and have developed complex systems to help them manage queues, something that the average shopper trying gauge whether aisle 19 (short queue, but inefficient operator) is better than aisle 20 (longer queue but smaller basket size) probably doesn't appreciate. (Fuzzy logic might help here!) Space in store is critical and the retail wage budget is massive, so what looks like large investment can be easily offset by the cost of real savings in either of those areas.

You might like to ponder this next time your stuck in a checkout queue, It's something I can't help myself doing... (My personal favourite is the Wilkinson's dot matrix receipt printer, which takes something like 15 seconds to print your receipt. Whilst laser printer are more expensive, the retail wage budget saving, at roughly a quarter to half a million pounds per second, I'm sure would massively outweigh the investment)

You can apply a queue theory equation in loads of different ways (another favourite of mine is NHS waiting lists). In a later post I'll talk about how I've used queue theory to revamp a large data warehouse batch schedule..

Collaboration between client and supplier lead to project success

2008-09-24T11:31:00.000+01:00

I've been frustrated recently by being surrounded by some 'Old Skool' supplier management techniques. I've always worked in Retail so I'm used to being in environments with very strong purchasing backgrounds, and suppliers being driven hard on deals. However, recent events have left me exasperated to the point of writing this.

The recent examples involves insuring yourself against failure to deliver by a supplier, which I think everyone would agree is a good thing. You can do this in a number of ways and permutations. Two approaches at opposite ends of the spectrum are, having bullet proof contracts that claw back money or services in the event of failure, at one end, or being pro-active and putting processes and resources in place to prevent the failure ever happening, at the other.

The former appears to cost you less, you get money back for any issues and you don't pay for anyone in a QA type role etc. and therefore seems quite attractive. I've also seen this create the 'big man effect' in those people who suddenly find themselves in a position to throw some weight around, which adds to the attractiveness for some. I feel this approach is short sighted and tends to focus on cash rather than value. It ignores the opportunity cost, or cost of failure.

Organisations who take this approach would do well to remember why they embarked on those projects in the first place. Generally (I know there are some exceptions) business undertakes new projects to add value, get a return on investment, call it what you will, spend some money, get more money back (lots more hopefully!). Or in the case of service, spend on support, to prevent issues happening that will cost you lots more.

So, if we're doing projects because we want to make more money, then any failure or delay, not only costs the additional sum to rectify the problem, but also costs the lost opportunity from delivering the project. In today's rapidly changing technology world, these opportunity costs can be enormous. Here at Retail Monster we've had the opportunity to work on cutting edge projects that have delivered massive returns on investment. You can read some of our case studies here.

My experience is to avoid this type of approach like the plague and take a more collaborative approach, more partner than supplier. Shared vision, shared objectives, collaborative teams, Agile approach, which is also how Retail Monster Consulting like to work. It's one of the reasons I formed the company. If you feel the same way, then maybe you'd like to Join us too.

Organisations that take the former approach, hiding behind contracts in the event of failure, can create a blame culture, where everyone is covering their back. Avoiding blame and responsibility becomes the primary driver. Organisations who embrace the partnership philosophy set themselves up for success, the culture of the organisation becomes based on success, delivery becomes key, and everyone internally, and 3rd parties, are focused on this. Because they're focused on the value of delivery. failure is just not an option....

Carbon labelling comes to a supermarket near you...

2008-09-24T11:20:00.000+01:00

Tesco opens up another front on the green supermarket war today, by announcing that products will now be labelled with their individual carbon footprint. Tesco are hot on the heels of Marks and Spencer who stole a march on their rivals to lead the sector on green credentials, by being the first to charge for carrier bags. A move that sounds as good for business as it does for the environment.

Tesco is to become the world’s first supermarket group to launch a major trial of carbon labelling on its own-brand products. The Carbon Reduction Label developed by the Carbon Trust will soon appear on 20 Tesco products in four different categories: laundry detergent, orange juice, potatoes and light bulbs.... Read the full press release here.

Product level carbon labelling was bound to happen sooner or later, and I don't take any credit for predicting it here some weeks ago. Whether this takes off or not, and leads to a genuine reduction in the production of CO2, only time will tell. It's easy to think that people will see this and think twice about which product to buy, searching and substituting for greener alternatives. That might certainly be true of hardened environmentalists but for the ordinary shopper this additional dimension of choice may not come in to play very much at all.

Today's ordinary shopper has a vast array of choice in picking between variants of a similar product. On top of the traditional hard-core options of brand and price, we can opt in for healthy eating (Be good to yourself), quality (Finest, Taste the Difference), ethics (Fairtrade), organic (inc non GM) to name but a few. How 'carbon' as an option competes amongst these existing choices will be interesting to see. i.e. High Carbon but Fairtrade vs Low Carbon non Fairtrade etc.

Whilst I might be questioning the effectiveness of carbon labelling in driving consumer choice, what I think is more certain is that labelling of products will drive down the carbon footprints via the suppliers. Some suppliers will want to be seen at the cutting edge of green manufacturing and will lead the reduction in Carbon values. For those that don't want to lead from the front, they won't want to get at the back of the back either. Once product level carbon measurement becomes more common, it won't be long before the media start running column inches and documentaries exposing high polluting suppliers.

Whilst consumers might find it hard to differentiate products, on the basis of their green credentials, at the shelf edge. They might find it easier to do from the comfort of their living room...

M&S start charging for food carrier bags

2008-09-24T10:59:00.000+01:00

5p for Carrier bags, good for the environment and good for business...

Today, Marks & Spencer begin charging 5p for food carrier bags. That's good for the environment because alongside the reduction in plastic that will be sent to landfill, the profit from the sale of the bags will go to Groundwork, an environmental charity which creates and improves green spaces across the UK. In preparation for this, M&S have been giving away free Bags for Life over the last month, with 15 Million given away so far. Customers will now be able to choose either a 5p food carrier or a Bag for Life costing 10p, to take their shopping home in. This is all part of Marks and Spencer's Plan A initiative which commits to, amongst others, reducing carrier bag usage by a third and to sending no waste to landfill by 2012.

With those free Bags for life costing £1.5M, how can that be good for business? Of course the key word in the above paragraph is profit. 'All of the profit from the sale of the bags'. Officially, that means 1.85p from the sale of single use food carrier bags will be donated to Groundwork. That's a nice 3.15p recouped per bag for M&S, which previously was bearing the costs of the plastic bags. So aside from the benefits of all the positive publicity, column inches, air time etc. On top of the competitive advantage gained by making the first move and being seen to set the standard for sustainability within the retail sector. You get to wipe out a very sizeable revenue charge in one fell swoop and get sector leading green credentials at the same time.

Now that's good business!

Green credentials : altruism or commercial interest?

2008-09-24T10:54:00.000+01:00

I recently facilitated a Retail Business Intelligence Seminar, which was attended by many of the UK's leading retailers. The format of the day was a number of presentations and some round table discussions on topics that had been selected by the attendees in advance by way of a questionnaire. The Butler group presented on the leading topics of the day within business intelligence and we heard from Conchango's David Ellis, on web performance management.

Corporate Social Responsibility reporting, colloquially referred to as 'Green' reporting, was one of the latter topics of the day for the round table discussion. Green credentials are obviously top of the agenda at C-level of the top retailers and are filtering their way down the organisational hierarchy. IT departments haven't yet got to grips with this in any serious way although departments like Marketing are in full swing, which is why we're all familiar with stories about Marks and Spencer, who are soon to start charging for carrier bags as part of their Plan A initiative, but in the meantime have been giving away free 'Bags for Life'. Sainsbury's have dipped their toe in the 'free Bag for Life' water too. Tesco of course are in on the act with the Future Store initiative, check it out next time you're in Wick (!), and have installed, reputedly, the worlds largest solar panel roof at their Fresh and Easy distribution centre in the US.

The discussion at the table rounded very squarely at one point on 'Are they doing it for money or because they care for the planet?'. Unanimously, instantly, 'for the money!' was the response. Was I the only person slightly disappointed by this response? Or have I bought into the marketing message and believe they do care after all? Who knows? but later when I reflected back on this, I thought, 'If they are in it for the money, in what directions could they take it?'

If you shop with a Tesco, and use your clubcard, then Tesco will know every item you buy. Tesco could therefore work out an individuals customers green credentials from the products they buy, assuming they work out individual product greenness, which surely is a first step. They'll already use some Fuzzy definition of greenness probably linked to a measure of ethicalness for segmentation purposes, but this will allow a whole new measure of granularity.

Doing this would of course give rise to opportunities to up-sell people to more 'green' products, that as well as espousing their green credentials, probably cost more and have a greater margin. This has profit implications but also huge customer satisfaction potential. Imagine how powerful Tesco online shopping could be if you could be presented with greener alternatives to those Kenyan strawberries, right there on the page? This has the added advantage of being applicable to all customers for as we have all heard, there are those vocal customer groups who shun the mighty clubcard for big brother reasons. For the many who carry one though, (13.5 Million), I wouldn't be surprised if they actually wanted Tesco to go this far. Why try and work out your own carbon footprint when someone else can do it for you?

As I stated above, 'Green' is big at C-level and has slowly started filtering down, at some point in the near future green credentials will filter it's way into mainstream BI. Expect more blogs on this in the future....