linear algebra
essential for machine learning algorithms
Basic Terminology
Matrix - an array of numbers, symbols or expressions, made up of rows and columns
usually denoted by a capital letter in bold; matrix A
a matrix with m rows and n columns is called m x n (m by n) matrix; e.g. a 2 x 3 (two by three) matrix means it has 2 rows and 3 columns
you can refer to a specific element by using [row, column] index, e.g. A[1, 2] is referring to the element in the 1st row and 2nd column of Matrix A
Vector - a matrix with one column
usually denoted in bold or with arrow on top of letter(s)
a row vector is a matrix with one row
you can refer to a specific element by using [row] index, e.g. v[2] refers to the 2nd element in vector v
Element of Set (∈) - ∈ is the symbol for membership set; it means “is an element of”; e.g. x∈A = x is an element of set A
Linear Combination - obtained by multiplying matrices by scalars (vector), and then adding them together
Inverse of a Matrix - when AC = I or AA^-1 = I where “I” is the identity matrix
Like real numbers, where the inverse of 3 is 3^-1 (or 1/3) and the product is 1, the inverse of matrix A is A^-1
An n x n (square) matrix A is invertible if n x n matrix C satisfies the equation AC = I or CA = I
If A and C are invertible matrices, then the product AC is also invertible by (AC)^-1 = (C^-1)(A^-1)
We can’t divide matrices, but you can take the inverse of a matrix and multiple it by the other matrix; e.g. A/C = error but A x C^-1 = value
Not every matrix has an inverse
Identity Matrix (I)- a square matrix with 1’s on the diagonal and 0’s elsewhere
Determinant (|A|) - denoted by two vertical lines; matrix must be a square; used when calculating identity matrix; formula for 2x2 matrix is |A| = ad - bc
Orthogonal Matrix - orthogonal means perpendicular; a square matrix whose columns and rows are orthogonal unit vectors; an invertible matrix where product of matrix A x its transpose equals the identity matrix
Eigenvector - or “characteristic” vector; a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it; think of rotating, reflecting, stretching/compressing an X-Y line chart without changing direction
Eigenvalue (λ) - the coefficient (factor/scalar) by which the eigenvectors are scaled
Trace of Matrix - tr(A); the sum of elements on the main diagonal of A
defined for a square matrix
sum of its eigenvalues
