DEV Community
Follow
Mathematics for AI — Ch.03: Probability [EN]
Probability offers a mathematical way to describe uncertainty and measure the likelihood of outcomes, crucial for AI due to real-world unpredictability. It starts with random experiments, processes with uncertain but definable outcomes, like rolling a die. A sample space (Ω) lists all possible outcomes, which can be finite, countably infinite, or uncountably infinite, depending on the experiment's definition. The structure of the sample space dictates how experimental results are represented, with AI often using them for system states or environmental conditions.An event is a specific collection of outcomes within the sample space, expressed as a subset of Ω. Events can be combined using set operations like union, intersection, and complement to describe complex conditions. Mutually exclusive events cannot occur simultaneously, an important distinction for accurate probability calculations. The probability of an event, P(A), is a numerical measure of its likelihood, ranging from zero to one.Valid probability measures adhere to three axioms: non-negative probabilities, a sample space probability of one, and additivity for disjoint events. For equally likely outcomes in a finite sample space, probability is calculated by dividing favorable outcomes by total outcomes. When outcomes have unequal likelihoods, individual probability weights are assigned and summed for events. The complement rule (P(Aᶜ) = 1 − P(A)) simplifies calculations for opposite events.The addition rule, P(A ∪ B) = P(A) + P(B) − P(A ∩ B), describes the likelihood of at least one of two events occurring. Probability can also be understood as the long-run relative frequency of an event in repeated experiments, consistent with the law of large numbers. Empirical probability uses observed data to estimate likelihoods, while computational methods combine formulas with simulations.Python and NumPy facilitate computational probability analysis, allowing simulations of experiments and comparison of empirical estimates with theoretical probabilities. Monte Carlo methods extend this by using repeated random sampling to estimate probabilities, especially useful for high-dimensional or complex problems. Careful attention to numerical precision and model assumptions is critical in computational probability.In machine learning, probability models provide uncertain predictions, expressing confidence rather than just single labels, and are vital for evaluating model calibration. Robotics and physical AI leverage probability to handle sensory noise and environmental variability, enabling robots to assign likelihoods to locations or objects based on incomplete data. A solid grasp of sample spaces, events, and probability computation forms the bedrock for advanced probabilistic AI models.