## Friday, November 30, 2012

### Poisson Distribution Facts - Some Really Interesting Stuff

The Poisson Distribution is a probability distribution.  A probability distribution assigns the probability of an event occurring in a random experiment.  In fact, it assigns the probability of every possible event occurring.  That's what makes it a distribution.

Understanding probability distributions empowers us to estimate how likely an event is to occur.   Different distribution models fit different types of data.  In 1898, Ladislaus Bortkiewicz published the book The Law of Small Numbers.  He discussed an interesting question, "How many Prussian cavalrymen are killed by horse kicks."

You may have no interest in the number of deaths per horse kick per year in a cavalry, but you probably do have your own questions.  You can use a Poisson probability distribution model if your data meets these criteria:

1.  The likelihood of the event you want to measure is small (thus the title "The Law of Small Numbers").
2.  The events of interest are independent.  For instance, when you flip a coin more than once, the second toss does not depend on the outcome of the first toss.
3.  The event can be counted in whole numbers, like how many times a salesman rings your doorbell.
4.  You would not ask how many times something did not happen, like how many times a salesman did not ring your doorbell.

The first step in constructing a Poisson distribution is to collect data and determine how often the event really does occur in the data.  But that is a topic for another day.

Don't be afraid to ask questions and construct your own data sets!

Happy experimenting.

## Tuesday, November 27, 2012

### An Exciting Freebie from Qualtrics

Get "Marketing Research 101: How to Get Insights that will Grow Your Business" from Qualtrics.  Go to http://www.qualtrics.com/blog/marketing-research-101 , and sign-up for their e-mail list.  If Qualtrics published it, you know it is solid statitics and anlaytics!

Here are more books by other authors...

### Examples of the Anchoring Effect

Are you thinking about buying a new car?  What about a house?  The salesperson is, no doubt, trained in using the anchoring effect to bring your expectation of the price up to their sales goal.

In a negotiation, the anchoring effect works by making sure your "opponent" has a higher or lower number in mind than what you really expect them to agree to.

The application to sales is pretty easy to see.  The case that surprised me was jail sentences.  Research has shown (see previous blog for references) that judges award longer sentences when the prosecuting attorneys ask for them.  This is being attributed to the anchoring effect.

I have wondered why students that harangue instructors win arguments for better grades.  Maybe the anchoring effect makes a difference in those discussions that go something like, "I deserve a 98, but you gave me a 90."

Now that you are aware of the anchoring effect, watch for it.  You may be surprised at the many places it appears.  However, be warned that even after receiving instruction on the anchoring effect, test subjects were still affected by it!

Happy negotiating!

## Monday, November 26, 2012

### This Post Could Change Your Life - The Anchoring Effect

Researchers Amos Tversky and Daniel Kahneman hypothesized that initially presented, irrelevant numbers influence human judgment.  They called their hypothesis the “anchoring effect” and argued that “…people make estimates by starting from an initial value that is adjusted to yield the final answer…Adjustments are typically insufficient.  That is, different starting points yield different estimates, which are biased toward the initial value” (Tversky & Kahneman, 1974, p. 1129).  The outcome of their research has application to all aspects of daily life that includes numbers, such as the length of criminal sentences, medical diagnoses, and purchase decisions.

The phenomenon of insufficient adjustment of initial values was tested with an experiment wherein subjects were asked to estimate various quantities in percentages.  After having been presented with a question, subjects spun a wheel of fortune.  They were then instructed to state whether the answer to the question was higher or lower than the number that came up on the wheel.  Finally, they estimated the answer to the question by moving up or down from the number the wheel produced.

In one particular trial, subjects were asked how many African countries belonged to the United Nations.  They were shown the numbers 10 and 65.  The median estimates for these respondents were 25% and 45% respectively.  Based on the fact that approximately 80% of all African countries belonged to the United Nations at the time, the majority of the respondents were estimating and not citing facts from memory.

The anchoring effect has many practical applications.  It changes outcomes in legal proceedings; medical diagnoses; and purchasing decisions, including major purchases such as real estate, as well as more minor consumer purchases (Smith, R., 2011, p. 110).  Besides proving the reality of the anchoring effect, the Tversky and Kahneman study suggests the impact of the anchoring effect is practically significant and warrants further investigation.  Specifically, the awareness of and ability to mitigate the anchoring effect can influence the outcome of daily events in profound ways.

Smith, A. (2011). Exploring the relationship between knowledge and anchoring effects: is the type of      knowledge important? (Doctoral Dissertation).  Retrieved from University of Iowa, Iowa Research Online (http://ir.uiowa.edu/etd/1264).

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1130.

Example to come...

## Sunday, November 25, 2012

### Quick Note on How I Choose Advertisers

1.  Do they offer a quality product?
2.  Do they offer a relevant product?
3.  Do they offer a unique product?

For the most part, I am a customer of these advertisers.  Occasionally I choose an advertiser I have not done business with, but I would be willing to try.

Please let me know of your experience with these advertisers.  If I receive any complaints, I will remove the offending advertiser from the site in order to maintain an excellent user experience!

Happy shopping!

## Saturday, November 24, 2012

### What does "robust" mean to process, software, and product developers and engineers?

In a nutshell, if a design (process, product, etc.) is "robust," you can't break it easily. Of course, everything is breakable. But making it robust means it is very much harder to break.

I will give you an example of robust development. On a recent job, I was asked to write a reporting tool in Microsoft's VBA for Excel. (VBA is Visual Basic for Applications--meaning for Excel and other Microsoft programs.) There was a very close deadline, so I estimated the amount of data that would be loaded into the spreadsheet for display. I used that estimate to write the upper bound of records to include in the report.

This was sloppy (not my norm), and not robust. Should the report user add just one more data set than I accounted for, the report would crash. On the next report, I wrote a subroutine to count the actual data sets loaded into the report. That report was very difficult to break. No matter how much data was included, the program could handle it.

Robust design is critical in product development. Robust products, processes, and services increase customer satisfaction, and in turn, the return on investment.

Data can provide information to designers about how robust their end product is. Look for patterns of complaints. If complaints or failures recur, look for ways to make the area of objection more robust.

Happy designing!

## Friday, November 23, 2012

### Process Improvement Starts with Thinking in Terms of Processes

Many times analytics are used to improve processes. These efforts will fail without one important key thought--when something goes wrong, it is the fault of the process, not the process user.

Here is an example. A few years ago a traffic circle was installed in my brother's neighborhood. I commented about the advantages of traffic circles. He said, "They may be good in other places, but people around her just don't understand them. We keep having accidents."

I was surprised by that statement. The next time I was in the area, I drove through the traffic circle, again. I noticed it was much more narrow than other traffic circles I had seen. Shortly after that, the local transportation agency removed the traffic circle and replaced it with a traditional intersection.

Did the people of the area really misunderstand how to use a traffic circle? It is possible there is a learning curve. However, traffic circles work in other neighborhoods not far away. I think it is more likely that the traffic circle (process) had inherent flaws. It appeared the road was too narrow, and the circle entrances did not have any lead area.

To improve processes using analytics, base your decisions on the quality of the process, not the capability of the users.

## Thursday, November 22, 2012

### Why Do We Randomize in Experiments?

Curious people experiment.  We want answers we can't find on Google.  Unfortunately, without some training, experimental results may not be valid.

One of the first things to know about experimenting is the need to randomize. First, randomization helps avoid the pitfalls of bias. Bias is the tendency for the data to all move one direction due to something other than what we are interested in investigating.

Second, statistical analysis makes some assumptions. We assume all errors will gather about an average and approximately cancel to zero. Without that assumption, many statistical tools are useless. So we randomize to justify making that assumption.

Design of experiments is a complex subject. But learning some simple principles will help you get started with everyday experiments that answer YOUR questions.

Happy experimenting!

123112

### Book Review -- Probability: An Introduction

Probability is the basis of not only statistics, but of almost all science, including mathematics, physics, chemistry, biology, engineering, marketing, etc.

In general, we rely on a number of principles of randomness.  Entropy is a measure of disorder or randomness.  An example of entropy is our reliance on air molecules spreading themselves randomly.  This is critically important to our ability to breathe.  If air typically arranged itself in a pattern that excluded part of a room, some people would suffocate.  Fortunately, nature is somewhat random.

This foundational subject is covered in a nice introduction by Samuel Goldberg.  Probability:  An Introduction was originally published in 1960, but has been republished by Dover since 1986.  The current cover price is only \$16.95.  You get a lot of bang for your buck with this book.

Although the style is stodgy, the information is solid.  The book contains five chapters, which cover sets, probability in finite sample spaces, sophisticated counting (which is combinations and permutations), random variables (which are functions), and binomial distributions.

After reading this book, you will understand the theoretical basis of an introduction to statistics.  The book covers the theory behind expected values (means), variance and standard deviations, z values in a normal distribution, as well as much more.  This is the best explanation I have ever received on combinations and permutations.  Overall, reading the book has been very satisfying.

If you can get past the style, this book is worth the time and money.

## Wednesday, November 21, 2012

### Keep Your Own Medical Records - Part 2

If the only thing stopping you from collecting and keeping you own medical data is knowing where to start, read on...

Obtain a permanently bound notebook. A regular composition notebook will do. Write the date somewhere in each entry, preferably the top. Record the basic facts of each event along with any special notes. Include any advice you receive with names of those present.

If you decide to make control charts, you can create simple time line charts in your notebook. More involved charts will probably require software.

One website you will want to look at is when evaluating your medical record history is http://www.hhs.gov/ocr/privacy/hipaa/understanding/consumers/index.html.

113012

### Do Yourself A Favor - Keep Your Own Medical Records

No matter how mundane your doctor visit may seem, keeping your own record of each visit is a good idea.  When you think about it, you are paying the doctor to do three things for you: 1. Evaluate your condition. 2. Make a record of your condition. 3. Treat you when necessary.

Even though you are paying for a record to be made, you do not have control over that record. The doctor can note whatever they want, even if it is not really what happened. You do have the right to dispute your record, but without a solid contradictory record, you will have a hard time doing that.

Besides being able to write whatever they care to write in your record, the doctor can redact information at a later date. So the record is subject to change, again, outside of your control.

Why should you care about what the doctor does with the record they make, own, and keep outside of your control? Sometimes what happened previously really matters. Some possibilities are insurance disputes, malpractice events (which you think will never happen to you, but may), recommendation of procedures requiring a second opinion, and ability to donate blood or organs.

Perhaps the best reason to keep your own record of your medical care is your health. You spend more time with yourself than anyone else. You have the opportunity to track data related to medical visits as well as everyday events.

One extreme example of tracking everyday events is diabetic care. Diabetic people must continually evaluate blood sugar levels. Relying on a doctor to do that is not a realistic option for maintaining good health. In this case, use of control charts would provide valuable data on which events/foods/etc. affect blood sugar levels the most. Lifestyle adjustments could be made based on the data collected.

Another example of event tracking would be control charts for weight. Exercise and diet could be included for additional decision making ability.

Just tracking your own immunizations could keep you out of the office for the occasional tetanus shot that you can't remember if you need or not.

Control your own data. Control you own life.

### Analytic Heros - You'll Be Suprised!

What do Sherlock Holmes, Adrian Monk, and Shawn Spencer have in common?  Well, yes, they are all fictional detectives, but they are also analytic experts.

Thinking about the definition of analytics (the discovery and communication of meaningful patterns in data), it makes sense that detectives are at the top of the hero list.

My favorite quote from Sherlock Holmes is "...when you have eliminated the impossible, whatever remains, however improbable, must be the truth."  Here Sherlock demonstrates the depth of his understanding of analytics.  The heart of the matter is believing the data will lead you to correct conclusions.

Adrian Monk also puts profound faith in data by way of evidence.  In one TV episode, he proclaims the murderer is a man who has been in a coma.  While everyone one around him disbelieves due to improbability, Monk follows the data.

Finally, Shawn Spencer uses analytics to promote his psychic powers.  He boldly states a dinosaur has killed a victim despite the improbability of that event.  In the end, the data proved him correct.

Do you find you believe the data (evidence) in front of your senses?  In general, we tend to ignore the improbable, despite the evidence.  The very act of estimating improbability makes us all statisticians.  It takes conscience training to overcome our skepticism of improbable data and believe the logical conclusions before us.

Happy sleuthing!

### Who Needs Analytics?

Everyone uses analytics, whether they know it or not.  Wikipedia says "Analytics is the discovery and communication of meaningful patterns in data." (http://en.wikipedia.org/wiki/Analytics)

Toddlers discover patterns in sound data to learn language, identify food vs. non-food object data to eat, find repetitive patterns in behavior data to play games, and generally categorize action data in order to interface with other people.

Learning to recognize analytics in everyday activities empowers.  Government agencies, medical providers, retail stores and their marketing agents, as well as other organizations collect and analyze data about us around the clock.  When we collect and analyze our own data, we level the playing field.  When we act on that analysis, we're ahead of the game.