Sign in Subscribe
Machine Learning

Artificial Intelligence: Uncertainty

Artificial Intelligence: Uncertainty
There is no way we can predict with 100% accuracy, but we can create AI that makes optimal decisions given limited information and uncertainty.

Can we be certain about anything, really?

Hopefully for us humans out there in the world we can draw conclusions based on previous knowledge or topic. However, when introducing AI, often times we can only provide partial knowledge of the world which in turn leaves opportunity for uncertainty within the system. Much like it would be impossible to predict with 100% accuracy natural disasters or market trends, by providing these machines information we might reach an optimal decision given their uncertainty.

Probability

Probability might be the best way to measure the likelihood or chance of a particular event occurring. It is expressed as a number between 0 and 1, where 0 indicates that the event is impossible, 1 indicates that the event is certain, and values between 0 and 1 represent degrees of likelihood.

Imagine rolling two standard dice. One red, one blue. There are 36 possible worlds we can roll. We can measure this event 0 < P(ω) < 1. We can see the highlight 7 has the highest probable chance of being rolled since it spans across 6 different iterations, and 2 & 12 only have one possible chance. Two 1's or two 6's.

A modern day example of this type of unconditional probability is (The Settlers of) Catan board game. In the image we see dots indicating the total number of possibilities of the dice rolling that specific number. The number 6 & 8 are both colored red and presented with 5 possible worlds of entry for the probability you might roll such a number. What is not indicated is the number 7, which serves as a "robber" function in the game whereby opponents have the opportunity to block and steal resources by placing a pawn object on a numbered tile of their choosing when they roll. The result of rolling dice is not dependent on the previous state, but together we can derive a probability nonetheless.

(The Settlers of) Catan on Board Game Arena. There are 18 number tokens, two each of 3, 4, 5, 6, 8, 9, 10, 11 and one each of 2 and 12. A valid arrangement of these tokens on the board (19 hexes in a side-3 hexagon) has each token placed in exactly one hex, leaving one empty hex (the desert), and none of the 6 and 8 tokens adjacent. So a 6 cannot be adjacent to the other 6, an 8 cannot be adjacent to the other 8 and a 6 cannot be adjacent to an 8.

Conditional Probability

The probability of a "given" b is equal to the probability of a and b being true, divided by the probability of b.

Conversely, to call something conditionally probable there must be a relationship between the evidence provided. We can begin to ask questions like what is the probability of a stock market surge today given that it surged in the previous trading session P(surge today | surge yesterday), or what is the probability of an incoming email being spam given that the previous email from the same sender was marked as spam P(spam today | spam yesterday).

Variables

In probability theory, a random variable is a variable whose possible values are outcomes of a random phenomenon. It represents a numerical quantity determined by the outcome of a random experiment or process. Random variables can be classified into two main types: discrete and continuous.

  1. Discrete Random Variables:
    • Take on a finite or countably infinite set of distinct values: dice rolling
  2. Continuous Random Variables:
    • Can take any value within a given range.
    • Described by probability density functions (pdf) rather than probability mass functions.
    • Examples include the height of a person, the time it takes for a computer to process a task, or the temperature in a given location.

For both types of random variables, probability distributions are used to describe the likelihood of different outcomes. The distribution function of a random variable specifies the probability of the variable taking on a specific value or falling within a certain range.

Understanding random variables is fundamental in analyzing and modeling uncertain or random phenomena, making them a key concept in probability theory and statistics.

Joint probability is a concept in probability theory that deals with the likelihood of two or more events occurring simultaneously. Identify the events you are interested in. For Catan, these could be things like getting a specific combination of resources (e.g., wheat and ore), rolling a certain combination on dice, or both you and an opponent achieving specific objectives in a turn.

Probability Rules

Negation:

  • The negation of an event A is the probability of A not happening, P(¬a) = 1 - P(a)

If event A is rolling a 4 on the dice, then the negation is the probability of not rolling a 4. It helps you consider alternative outcomes.

Inclusion-Exclusion:

  • This rule is used to calculate the probability of the union of multiple events (A or B or both). P(a ∨ b) = P(a) + P(b) - P(a ∧ b)

Events A and B represent different ways to achieve a specific goal (e.g., gaining resources through trading or rolling the dice), the inclusion-exclusion rule helps calculate the overall probability of success considering both possibilities.

Marginalization:

  • Marginalization involves obtaining the probability distribution of one or more variables from the joint probability distribution of all variables. P(a) = P(a, b) + P(a, ¬b)

In Catan, if you are interested in the probability of a specific resource (e.g., wheat) regardless of other factors (e.g., dice rolls), marginalization helps focus on the probability of getting wheat by summing up the joint probabilities across all possible dice outcomes.

Ultimately, in the face of uncertain events, the principles of probability provide a structured framework to navigate ambiguity and make informed decisions.

[1]: Brian Yu, David J. Malan; Harvard University CS50's Introduction to Artificial Intelligence with Python

[2]: Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor; An Introduction to Statistical Learning with Applications in Python

Read next