Saturday, November 22, 2008

The Problem with Assumptions

I just finished reading an article by Michael Lieberman in the November Issue of Quirk's Marketing Research Review. The article is titled Measuring and maximizing the ROI of a loyalty program. Needless to say, it caught my attention.

First let me say that Mr. Lieberman has presented an excellent case for showing how an ROI can be computed for a given loyalty program. He uses a technique referred to as Monte Carlo Simulation. This technique uses a series of random trials (events) to see how a system that is not deterministic will respond by going through the process a large number of times. By using some statistical distributions to represent the different ways elements of the system (customers) could act, the output provides a picture of those many trials. This technique has been used for a long time and has been well tested and accepted.

There are two very important aspects to performing a Monte Carlo simulation. The first is to understand the assumptions used and second is to compare the simulation results to actual results. Unfortunately, the magazine did not give Mr. Leiberman the space to fully demonstrate the power of Monte Carlo simulation and answer these two aspects.

The first aspect which was inadequately discussed was a clear and complete discussion of the assumptions in his model. The second aspect was to compare the simulation results to the real world that he was simulating. There was no there, there. The results of his simulation were not compared to actual data.

Some of the assumptions that should have been explained in the article might include:
1. the relationship between the dollar value of the purchases and the incentives given. Is the relationship linear, non-linear or ???
2. the variation of margins for different product mixes. Was this considered a constant?
3. the percentage of loyalty points that will be redeemed (if the data is categorized then the assumptions should be noted by group unless it is further assumed that the groups are all the same).
4. the belief that a customer's spending behavior will change because their spending has changed.

The second aspect is that results must ALWAYS be compared with the real world. Yes, computers can produce a lot of output that seems to represent the real world, but until it is tested and verified in the real world, it is just computer output. The fact is that the computer output may be 100 percent accurate or 10 percent accurate, but you will never know it unless it is checked against the actual system in place.

For example, when I worked at Xerox, we were investigating the idea of going to team service. One of the variables we needed to understand was the statistical distribution of the time to complete a service call. The assumption was that the service completion time was statistically distributed according to the negative exponential distribution. Before we ran our simulation model we built an analytical queuing model to get an idea what the optimum team size should be. We could not easily include meetings, lunches and parts chasing into our model so the only way to get an accurate assessment of the optimum team size was to build the simulation model to account for the vagaries that were not easily modeled analytically. As it turned out the service completion time distribution was not a simple negative distribution, it was a significantly different variant of the negative exponential distribution and one that made a difference in our results. While it did not change the optimum team size, it did impact the value of the team.

The point here is that you must carefully describe the model and ALL its assumptions so that you are not misled by the computer results. A friend of mine once said that GIGO does not stand for "Garbage - In Garbage Out." Rather it stands for "Garbage In - Gospel Out."

The bottom line is that Monte Carlo Simulations provide an excellent tool for pre-testing business policies. I believe that Michael Lieberman did do the work and managed the assumptions. Unfortunately, his article doesn't reflect that. Sometimes an editorial pen can kill you.

65 comments:

Anonymous said...

Hello there! I could have sworn I've visited this blog before but after going through some of the articles I realized it's new to me.
Anyhow, I'm certainly delighted I discovered it and I'll be
book-marking it and checking back frequently!
Feel free to visit my webpage - Buy twitter Followers cheap

Anonymous said...

I gave each person who won a gift card to buy i - Tunes songs.

In this way, there will be no shortage of people at
the party. (Just don't forget to take some Tylenol for the migraine headache that might result.

My website :: 18th birthday party ideas

Anonymous said...

Hey! I know this is kinda off topic but I was wondering if you knew
where I could locate a captcha plugin for my comment form? I'm using the same blog platform as yours and I'm having difficulty finding
one? Thanks a lot!

My homepage ... 10th birthday party ideas

Anonymous said...

Douching and unique scented bathroom items can upset
the pure stability of bacteria in the vagina and make yeast infections much more possible.
The most common feature of yeast infection are-. Candida bacteria feed on other weakened
friendly bacteria in the human body.

Feel free to visit my page - recurring yeast infection after period

Anonymous said...

DJ spins out on Saturday nights. * Team answer sheets
- Basically a grid lined A4 type sheet with answer write in numbered boxes and a line on top for the team name.

The decline of the East side blues scene was disheartening, but,
it also gave rise to the need for a fresh start, which came in
the form of the next blues-only venue, Antone's, founded by the late Clifford Antone, during the summer of 1975.

Here is my web page pub quiz anagrams

Anonymous said...

DJ spins out on Saturday nights. Brazenhead is a great place to go with
your family or for a business lunch or dinner, but if
you are looking for a party atmosphere, this isn't it. Her father, Bruce Paltrow, produced the critically acclaimed TV series that is considered the precursor to many medical shows today, St.

Review my web site; good pub quiz names

Anonymous said...

For those who are unaware, Vistaprint is a website that has been around for
years, and it offers a variety of personalized products that can be used for businesses, gifts, weddings,
and more. This is the perfect chance to send a wedding
announcement. So even though it's been annoying at times that I've had to do most
of the planning of this wedding myself, it's been good for the same person to have an understanding of everything that's going on.


Here is my website :: wedding website album

Anonymous said...

The kids will have fantastic fun playing this game, which is really a favorite out of
quite a few on the more popular kids birthday party games.
So if you plan one, here are few more birthday
games ideas for different themes of birthday parties. Traditional board games like musical
chairs relay, dancing paper, and many others.

Also visit my website; 10th Birthday party ideas

Anonymous said...

This is because they have a high number of satisfied clients
who have purchased likes and fans in bulk and are reaping the immense benefits.
For those who do not have much experience, there are many sites that offer
a free book on how to better sell their products through Facebook fans, so
do not wait, build your site and buy some fans now. The more Facebook fans you can make, the more followers
you can get, which eventually add to your growing clientele base.


my site http://socialbears.com/facebook-fans/

Anonymous said...

This is usually considered a fun and effective
learning method in general. -- Notes: Bb D F. When you're acquainted with these core 7 keys you can start to learn about the black keys (flats and sharps).

my homepage Kailangan kita piano chords and notes
Also see my web page: learning piano chords and scales

Anonymous said...

One quick and important note: make sure you know the requirements and can get them
all accomplished for your out-of-state or out-of-country wedding.
(US). If you don't have time, think about hiring someone for Best wedding website.

my web page - our wedding website about us
My site: wedding website about us

Anonymous said...

can be included. The diet program is generally known as the
caveman diet or perhaps the Paleolithic Diet. Consider the following statistic.


Feel free to surf to my blog post :: paleo diet acne rosacea

Anonymous said...

G# or Ab - (lower) so#, (middle) do, (middle)
re# G#, C, D# keys. As an alternative, it will most likely be far more powerful for you to learn about audio and
chord idea. ][.

Also visit my web blog: easy piano chords Amazing grace

Anonymous said...

If your tween is into paint-ball you can take everyone to play paint-ball.

Even Halloween parties with the very best Halloween costumes can get dry after everyone's seen the costumes to be seen and passed out the candy to the beggars. Now, one of my co-workers was applying to the Secret Service and I knew that I was down as a reference for her.

Feel free to visit my blog post; www americanexpress com

Anonymous said...

This is usually considered a fun and effective learning method in general.
As an alternative, it will most likely be far more powerful for you to learn about audio and chord
idea. When you're acquainted with these core 7 keys you can start to learn about the black keys (flats and sharps).

Also visit my web-site - free piano chords and keys

Anonymous said...

In looking for the proper collar for the pets, things that you need to consider are safety, the training process as well as your convenience.
This will help make navigating the weaving poles a habit for you dog.
It is wise to take the classical conditioning method first because it is easier to do.



My web blog: agility dog training austin
my site: protection dog training atlanta

Anonymous said...

In other words, they go against the grain of the careers their parents had.
At the end of each round read out the cumulative scores. 8.


Here is my website; brainstormer pub Quiz answers

Anonymous said...

D Minor:. Now among the advice of your educational materials, shoring jar discover all the things of the
substitute chords. When you're acquainted with these core 7 keys you can start to learn about the black keys (flats and sharps).

Here is my site; jazz piano chords and scales

Anonymous said...

He will know when he’s done something bad or
good. In the event you employ strategies
that may be employed in agreement together with your dog's normal urge and instincts, you'll be able to improve its character and provide restrictions that can maintain your dog satisfied.
Dog Training Using Praise.

Feel free to surf to my webpage ... sfspca dog training academy

Anonymous said...

Password protection is an ideal way to protect sensitive information you prefer only those authorized have access to.
Instead of sending your scanned invitation, it would be great if you create your own wedding website and invite your friends and relatives.
So even though it's been annoying at times that I've had to do most of the planning
of this wedding myself, it's been good for the same person to have an understanding of everything that's going on.



my site :: wedding website album

Anonymous said...

Password protection is an ideal way to protect sensitive information you prefer
only those authorized have access to. (US). If necessary, place a polite reminder on your sign-in page that this is
not an informal website, and that distasteful comments will
be removed.

my site :: Wedding website and invitations
my website: wedding website affiliate program

Anonymous said...

Natural remedies are deemed highly efficient, cost effective,
and significantly safe to use. Are you looking to get rid of
troublesome yeast infections. If you apply Aloe Vera
gel on the affected areas it can help reduce the itching and burning caused by the infection.


Also visit my website symptoms of yeast infection and bleeding

Anonymous said...

The owner decided after the event to donate half of the
proceeds back to the students. This was zero cost because the students went
to local stores and asked for either donations and gift cards to purchase these items.

Now, one of my co-workers was applying to the Secret Service and I knew
that I was down as a reference for her.

Feel free to surf to my weblog: Mailing lists

Anonymous said...

Some were practical, of course, but others were psychological and emotional.
*Rounds table format. The decline of the East side
blues scene was disheartening, but, it also
gave rise to the need for a fresh start, which came in the form of the next blues-only venue, Antone's, founded by the late Clifford Antone, during the summer of 1975.

my web site: pub quiz answers and questions

Anonymous said...

Some were practical, of course, but others were psychological and emotional.
Brazenhead is a great place to go with your family or for a business lunch
or dinner, but if you are looking for a party atmosphere, this isn't it. Her father, Bruce Paltrow, produced the critically acclaimed TV series that is considered the precursor to many medical shows today, St.

Also visit my web site :: best pub quiz names

Anonymous said...

Narrow blood vessels lie alongside the intestines of the
earthworm and they absorb the nutrients from the alimentary canal feeding the rest of the body.

*Rounds table format. The buccal cavity is a small
cavity that has neither jaws nor teeth.

My website ... pub quiz austin

Anonymous said...

The decor on the inside of the restaurant is
absolutely beautiful. Brazenhead is a great place to go
with your family or for a business lunch or dinner, but if you
are looking for a party atmosphere, this isn't it. The decline of the East side blues scene was disheartening, but, it also gave rise to the need for a fresh start, which came in the form of the next blues-only venue, Antone's, founded by the late Clifford Antone, during the summer of 1975.



Here is my homepage; pub quiz aberdeen

Anonymous said...

It was developed by Walter L. Obesity is an epidemic in our society.
Always check your physician before making any changes
to your daily habits.

Here is my web-site paleo diet and cholesterol levels

Anonymous said...

In other words, they go against the grain of the careers their parents had.

At the end of each round read out the cumulative scores.
28.

Feel free to surf to my web-site free pub quiz and answers

Anonymous said...

(Thank you rounds are always welcome, of course. Anna had been looking through my phone while
I was naked. The buccal cavity is a small cavity that has neither jaws nor teeth.



Feel free to visit my web-site; pub quiz archive

Anonymous said...

In that case, this kind of question should not be taken
from any show telecasted in specific country. * Team
answer sheets - Basically a grid lined A4 type
sheet with answer write in numbered boxes and a line on top for the team name.
28.

Here is my page; free pub quiz answers

Anonymous said...

In other words, they go against the grain of the careers their parents had.

Anna had been looking through my phone while I was
naked. The decline of the East side blues scene was disheartening, but, it also gave rise to the need
for a fresh start, which came in the form of
the next blues-only venue, Antone's, founded by the late Clifford Antone, during the summer of 1975.

my site free pub Quiz And answers

Anonymous said...

If all players are wrong the murderer gets away and everyone loses.
At the end of the party all of the papers write down who
they think was the prohibition agent. You Divide everyone at the party into
two groups, or tribes.

my web site :: weight watchers

Anonymous said...

A result of the immense size and strength that these animals can grow to means pets should be leash trained from a young age.
Another tip is to start with the basic skills. We turn off the TV.


Visit my web page ... guard Dog training at home

Anonymous said...

If your tween is into paint-ball you can
take everyone to play paint-ball. At the
end of the party all of the papers write down who they think
was the prohibition agent. Spice things up by including these Halloween-themes recipes:.


Here is my web site - self defense

Anonymous said...

there are lots of family dog web pages which provide totally
free information on several unique instruction troubles.
Citrus juices help in decreasing the chance of bacteria to remain in your dog.
It is wise to take the classical conditioning method first because
it is easier to do.

Feel free to visit my weblog: electronic dog training aids

Anonymous said...

He will know when he’s done something bad or good.
Citrus juices help in decreasing the chance of bacteria to remain
in your dog. With the Easy Walk Harness, if your dog attempts
to pull forward the straps across the chest and shoulder blades
begin to tighten.

Here is my web blog; paw law Dog Training academy

Anonymous said...

If all players are wrong the murderer gets
away and everyone loses. You make different challenges like scavenger hunts, food challenges,
mazes and races. The game would begin with them watching the collapse of society and
the outbreak of magic around the world.

my webpage ... online poker

Anonymous said...

Simple and Easy to Create Recipes with clear step by step instructions
you will be able to produce paleo friendly meals that get rave reviews from friends and family
every- time. Anti Nutrients:. - Lecithin.

my homepage :: paleo diet and weight loss

Anonymous said...

So you are looking to rid your girl of these unwanted stalkers when you are drawn to Willow Creek's Black Mirror Castle. At the end of the party all of the papers write down who they think was the prohibition agent. I hope you have fun planing a birthday party for your tween, I hope my birthday party ideas for tween's helped you.


Feel free to visit my website: virgin atlantic

Anonymous said...

To get the best product type never make your choice with first product or store you
see, but you should check several fishing kayaks for sale and try to get appropriate offer.
You will also have to decide whether you want a tandem kayak or single kayak.
Best of all, a kayak will cost a fraction of the price of
the average fishing boat.

My web page: columbus hotels

Anonymous said...

Nothing can compare to the more expensive tools which are made from high-grade tempered steel.
You might even find yourself flying over top of other riders,
so keep your limbs as close to your tube as possible. A really great way to make this work -- if
the store allows it -- is to couple sale
items with coupons, for even greater savings.

Look at my web-site :: water shoes

Anonymous said...

Its peak season, June through August, is smack in the
middle of New Braunfels' hottest, sunniest weather season. Dig your feet into the hot sand and leave your wrist watch at home. Our six year old was there too and instead of telling daddy Gerrin was climbing on the cabin roof she was helping him get up there my husband found out.

Feel free to visit my site; Water shoes

Anonymous said...

The only hurdle to accessing this is of course human limitations and the fact that
the brain does not function solely as a learning tool for the human being.
Anna had been looking through my phone while I was naked.

28.

Also visit my web site: free pub quiz and answers

Anonymous said...

Okay so I'm to write a creative writing essay on my own personal utopia and what i would like it to be like and all that. I'm having issues starting though. I can't seem to come up with a good introduction that would lead to my thesis---which i don't even have..
[url=http://bonus-2013-france.blogspot.com/]bwin bonus[/url] [url=http://bonus365sk.blogspot.com/]bet365 bonus[/url]
[url=http://www.bapostas.com/]bwin bonus[/url] [url=http://bonus-2013-italy.blogspot.com/]bwin[/url]

Anonymous said...

After completing your program at the swimming pool, you must wash the
feet cleanly and dry the feet are very much necessary.
You can wear this unique toe shoe like they were water shoes, while doing some barefoot running, while hiking,
or even while doing yoga. Just be aware that the sun doesn't always love too well in return.

Anonymous said...

Food has become the most overused drug and the most underused form of nutrition.

The fact is, most grains are quite high in glycemic index, meaning that they make your blood sugar go up rapidly,
and send you crashing down soon thereafter. - Lecithin.

Visit my web-site :: the paleo diet and acne

Anonymous said...

I’m not thаt much of а online reaԁer to be honest but yοuг blogs rеally niсe,
keep it up! I'll go ahead and bookmark your website to come back down the road. All the best

My web page - diamondlinks review

Anonymous said...

The only hurdle to accessing this is of course human limitations and the fact that
the brain does not function solely as a learning tool for the human being.
Below are listed a few popular pubs where one can chill in Mumbai.
The Bull's Head Pub, Bangkok.

Also visit my site; pub quiz names

Anonymous said...

In that case, this kind of question should not be taken from any show telecasted in specific country.
Brazenhead is a great place to go with your family or for a business lunch or dinner, but if you are looking for
a party atmosphere, this isn't it. The Bull's Head Pub, Bangkok.


my page pub quiz anagrams

Anonymous said...

The decor on the inside of the restaurant is absolutely beautiful.
Local country clubs are offering trivia fun for as little as
forty dollars per couple. The decline of the East side blues scene was disheartening, but, it also gave rise to the
need for a fresh start, which came in the form
of the next blues-only venue, Antone's, founded by the late Clifford Antone, during the summer of 1975.

my homepage :: pub quiz and answers

Anonymous said...

DJ spins out on Saturday nights. Local country clubs are offering trivia fun for as little as
forty dollars per couple. They feature almost nightly drink specials and some form of entertainment every night of the week--DJ's, live music, trivia, you name it.

Feel free to visit my website pub quiz aberdeen

Anonymous said...

On Sunday nights Erin Jaimes hosts a blues jam where anyone from Alan Haynes to Gary Clark,
Jr. *Rounds table format. The decline of the East side blues scene was disheartening, but,
it also gave rise to the need for a fresh start, which came in the form of
the next blues-only venue, Antone's, founded by the late Clifford Antone, during the summer of 1975.

Have a look at my web-site - brainstormer pub quiz answers

Anonymous said...

In that case, this kind of question should not be taken from any show telecasted in specific country.
You must definitely be planning to make it special
and memorable by keeping a good theme, ordering the best food
and choosing the best games. They feature almost nightly drink specials and some form of entertainment
every night of the week--DJ's, live music, trivia, you name it.

Check out my blog; funny pub quiz names

Anonymous said...

Some were practical, of course, but others were psychological
and emotional. *Rounds table format. Her father, Bruce Paltrow,
produced the critically acclaimed TV series that is considered the
precursor to many medical shows today, St.


Here is my web site - brainstormer pub quiz answers

Anonymous said...

The decor on the inside of the restaurant is absolutely
beautiful. Brazenhead is a great place to go with your
family or for a business lunch or dinner, but if you are
looking for a party atmosphere, this isn't it. Her father, Bruce Paltrow, produced the critically acclaimed TV series that is considered the precursor to many medical shows today, St.

Here is my blog ... funny pub quiz names

Anonymous said...

In that case, this kind of question should not be taken from any show telecasted in specific country.
At the end of each round read out the cumulative scores.
The decline of the East side blues scene was disheartening, but, it also gave rise to the need for a fresh start, which came in the
form of the next blues-only venue, Antone's, founded by the late Clifford Antone, during the summer of 1975.

Look at my web site: Easy pub Quiz and Answers

Anonymous said...

(Thank you rounds are always welcome, of course. You must definitely
be planning to make it special and memorable by keeping a good theme, ordering the best food and choosing the best games.
28.

Also visit my weblog - pub quiz archive

Anonymous said...

DJ spins out on Saturday nights. * Team answer sheets - Basically a grid lined A4 type sheet with answer write
in numbered boxes and a line on top for the team name. 28.

Here is my web blog :: pub quiz answers and questions

Anonymous said...

DJ spins out on Saturday nights. You must definitely be planning to make it special and
memorable by keeping a good theme, ordering the best food and choosing
the best games. Theme Format: It is almost like standard format of the
pub quiz.

Here is my homepage :: pub quiz and answers

Anonymous said...

In other words, they go against the grain of the careers their parents had.
Popular prizes include sports tickets, cash and vouchers for drinks, food - and dollars off of tabs.
The food is decent and the drink specials on Tuesdays include $2.


Here is my site: clever Pub quiz Names

Anonymous said...

' Satanic Ritual - Decorate the space with human sacrifice, goblets of blood, pentagrams, and things draped in black. Unfortunately for me as an addict, I need my crime fix. You Divide everyone at the party into two groups, or tribes.

My web site free insurance quotes

Anonymous said...

By dividing up the party at several peoples houses you make it easy on yourself.

You make different challenges like scavenger hunts, food challenges,
mazes and races. The game would begin with them watching the
collapse of society and the outbreak of magic around the world.


Check out my homepage :: used cars ()

Anonymous said...

Wow, that's what I was seeking for, what a material! present here at this weblog, thanks admin of this site.

Feel free to visit my web-site :: side business ideas

Anonymous said...

This website was... how do I say it? Relevant!! Finally I have found something that helped me.
Thanks a lot!

My web site - working from home ideas - -

 

web visitor stats
OptiPlex 755 Desktops