This article is not about probabilistic algorithms, which give the right answer with high probability but not with certainty, nor about Monte Carlo methods, which are simulations relying on pseudo-randomness.
The probabilistic method is a non-constructive method in combinatorics pioneered by Paul Erdös, for proving the existence of a prescribed kind of mathematical object. It works by showing that if one randomly chooses objects from a specified class, the probability that the result is of the prescribed kind is more than zero. Although the proof uses probability, the final conclusion is determined for certain, without any possible error.
One way of doing this is by considering a randomly selected thing from a finite-sized universe. If the probability that the random thing satifies certain properties is greater than zero, then this proves the existence of a thing that satisfies the properties. It doesn't matter if the probability is astronomically small; any probability strictly greater than zero will do. (Also, showing that the probability is zero can be used to prove the non-existence of such an object).
Another way to use the probabilistic method is by calculating the expected value of some random variable. If it can be shown that the random variable can take on a value less than the expected value, this proves that the random variable can also take on some value greater than the expected value.
Example:
One example of how we can use this method is to create a lower bound on the Ramsey number R(r,r;2). Suppose we have a complete graph on n vertices. We wish to show (for small enough values of n) that it is possible to color the edges of the graph in two colors (say red and blue) so that there is no complete subgraph on r vertices which is monochromatic (every edge colored the same color)
To do so, we color the graph randomly! Color each edge independently with probability 1/2 of being red and 1/2 of being blue. We calculate the expected number of monochromatic subgraphs on r vertices as follows:
For any set S of r vertices from our graph, define the variable X(S) to be 1 if every edge amongst the r vertices is the same color, and 0 otherwise. Note that the number of monochromatic r-subgraphs is the sum of X(S) over all possible subsets. For any S, the expected value of X(S) is simply the probability that all of the r(r-1)/2 edges in S are the same color, 2*2(r(r-1)/2) (the factor of 2 comes because there are two possible colors).
This holds true for any of the C(n,r) possible subsets we could have chosen, so we have that the sum of E[X(S)] over all S is
2*C(n,r)*2(-r(r-1)/2).
The sum of an expectation is the expectation of the sum (regardless of whether the variables are independent), so the expectation of the sum (the expected number of monochromatic r-subgraphs) is 2*C(n,r)*2(-r(r-1)/2).
Consider what happens if this value is less than 1. The number of monochromatic r-subgraphs in our random coloring will always be an integer, so must for at least one coloring be less than the expected value. But the only integer which satisfies this criterion is 0! Thus if C(n,r)<2(r(r-1)/2-1), some coloring fits our desired criterion, so be definition R(r,r;2) must be bigger than n. In particular, R(r,r;2) must grow at least exponentially with r.
Note that although the probabilistic method tells us that such a coloring exists, it does not tell us anything at all about how to construct it!
