Expert: Josh - 8/29/2005

Question
Hi Josh (sorry if it seems I'm always asking you, you're the only person I can find on here who covers this area);

There are 3 people, Kate, Pam, Wendy with 3 names for each person in a hat (making 9 names e.g. K1,K2,K3,P1 etc.). 3 names are being
drawn each time.

I was told if I showed this in a 9x9x9 graph it was not independent, which I agreed with but they said;

'I could put the 9*8*7 (equally likely) outcomes in the 21x24 chart however I want, and the row event will be independent of the column event, but I will likely not be able to come up with a meaningful description of the rows and columns
(they're just random).'

I don't understand how the 9x8x7 or the 21x24 can be independent just because they have equally likely outcomes.

[Difficulty]
9*8*7 is baisically the same as drawing one name, then another, then another. So if this is independent, that would mean the drawing of 3
names is independent and that cannot be..!

Like say if you drew one name, that name could not appear on the axis with 8 names on because it has already been drawn and the same
for the second draw. For each name drawn first it would affect the axis and would have to keep changing for each name that was drawn first/second and it's not possible, thus meaning not independent!

Would you say the 9*8*7 and 21*24 chart is independent or not because I really don't know!?

Thanks!:)


Answer
Hi Alastair,

This is not an issue...except I probably won't be around much of the time over the next three weeks, due to work commitments. Other than that, you are most welcome to ask me any question you like, whenever I'm around.

Regarding your question, we should take care in understanding what the term "independent". What does it refer to?
In probability (when we are dealing with random processes), the term "independence" means that each selection has no bearing on the outcome of subsequent events. Loosely speaking, knowing the outcome of the first selection, for instance, does not give you anymore information on what is going to happen next. Statistically, the likelihood of an event happening is not influenced by past events. Relating back to your example, we can ask ourselves the following questions: (1) Is the second selection independent from the first selection; (2) Is the third selection independent of both the first and second selection.

It is important to realize what the term "independence" means in the context of linear algebra. Or perhaps, you might be more comfortable with 3D geometry. Imagine creating a coordinate system from the edges of a cube. Let us refer to the three axes as "x", "y" and "z" respectively. The 3 axes are said to be "mutually orthogonal" because they are 90 degrees apart. Using the notation (x,y,z), we can then describe objects in three dimensional space. These three axes are said to "span a three dimensional space", because we cannot completely locate the position of an object if we omit one of the (x-,y- or z-) measurements. In this sense, each axis points in an independent direction.

So, how do these two seemingly different ideas fit together? Well,... we are using each axis (X, Y and Z) to represent a random selection (pick 1,2 and 3, respectively). For each selection, X, Y and Z, we enumerate all the possible outcomes {K1,K2,K3,P1,P2,P3,W1,W2 and W3} as the set of values {0,1,2,3,4,5,6,7 and 8}. In this way, we can visualize each position in the X-Y-Z coordinate system as being representative of a particular outcome.

Now, if our first pick was to be, say, X=P2, we would have assigned a value of X=4 to denote this outcome. The significance is that this equation, X=4, removes one degree of freedom from us. It effectively restricts us to a 2D probability space, described by (X=4,Y,Z); where the random values for the second and third selection (Y and Z) are yet to be chosen. In this example, if we had picked X=P2, we would be forced to sit in the two-dimensional YZ plane, which cuts through X=4 at right angle.

If we continue with our hypothetical discussion, and suppose that we picked K2 in our second random selection, we would represent this as Y=1 (this follows from the fact that I have labeled K1 as 0, K2 as 1 and so forth). Now, the situation is described by (X=4,Y=1,Z).

So, how did we arrive at this point? Can we still occupy any point (realize any outcome) in the 3D space at this stage? No. The third selection is not independent of the first two selections. Instead, our final selection is conditional upon our history, X=P2 and Y=K2 (in this example, anyway).

Of course, we are free to assign the 9*8*7 possible outcomes to 21*24 slots, as we please. But, then, we lose this very nice analogy with geometry, where each axis spans in a linearly independent direction. (The X,Y,Z axes are orthogonal, remember? i.e., at 90 degrees with each other. For example, X cannot be described by combining Y and Z. Similarly, if you imagine yourself flying a plane, Z cannot be described as a combination of X and Y movements. If Z represents "altitude", your aircraft cannot climb vertically, if you simply maneouver it laterally, in the X-Y directions.)

IMPORTANT: The loss of independence has nothing to do with the events being equally probable or not. It is due to the fact that we insist on thinking of three consecutive selections as a group. The question is constructed so that it gets a point across. We haven't done anything rigorous to show that the random variables X,Y and Z are statistically independent (or NOT).

If objects are selected without replacement, subsequent events CANNOT be independent of its history. Since, you cut your odds knowing which element you have just eliminated.