A transformation in linear algebra can be likened to functions in the sense that they both take an input and produce an output. In the context of linear algebra, both the input and output are vectors.
To understand the transformation as a whole, we need to consider how every possible input vector maps to the corresponding output vector
Since all possible input vectors span the entire space, observing how they transform is equivalent to understanding the transformation of the entire space itself
Arbitrary transformations can get very complicated and hard to predict, so we will limit ourselves to linear transformations. A transformation is linear if it satisfies two main properties:
Additivity: The transformation of the sum of two vectors is equal to the sum of their transformations.
Homogeneity (or Scalar Multiplication): The transformation of a scalar multiple of a vector is equal to the scalar multiple of the transformation of that vector.
In simple terms, a linear transformation preserves vector addition and scalar multiplication, the two fundamental properties of vectors.
Just like functions, we can describe a linear transformation numerically:
Trans(V)=Vtransformed
Our objective is to find a mathematical representation that can take any appropriate vector as input and produce the corresponding output.
The complexity of our problem is greatly reduced when we realize we can understand the entire transformation by observing its effect on a set of basis vectors.
Recall that any vector can be represented as a linear combination of the basis vectors.
V=v1 basis 1 +v2 basis 2 +v3 basis 3 +⋯
When the transformation is linear, we can exploit the additivity and homogeneity properties
Geometrically, this means that during a linear transformation, all vectors, including the basis vectors, experience the same type of change. Therefore, any output vector can be obtained by applying the same linear combination to the transformed basis vectors.
The fact that the set of components of the transformed vector is identical to that of the original vector means that the effect of a linear transformation can be fully described by how it transforms the basis vectors
In other words, if we know how the basis vectors transform, we can easily determine how any vector in the space will change. Having said that, by assembling all the transformed basis vectors together, this array encompasses all the information required to describe the linear transformation
an array containing all necessary information[transformedbasis 1transformedbasis 2transformedbasis 3⋯]
We refer to this array as a matrix.
A matrix can be viewed as a collection of transformed basis vectors, encapsulating all the necessary information to perform a linear transformation.
In this sense, a matrix acts as the operator for linear transformations.
We can readily derive a general expression for how a matrix transforms a vector, a process known as matrix-vector multiplication. To execute this operation, it's essential to know the components of the transformed basis vectors and the input vector in the original basis.
Therefore the computation of matrix-vector multiplication can be reduced to scalar multiplication, something we are already familiar with
Each entry is a scalar that scales the corresponding basis vector basis i .
The number of entries in the column vector is equal to the number of basis vectors required to span the space. Therefore, the number of entries directly reflects the dimension of the vector space.
The dimension of the transformed vector thus depends on the dimension of the transformed basis vectors. Suppose the basis vectors are represented by columns of m entries, then the transformed vector will be of dimension m as well.
Linear transformations can change the dimension of vectors, meaning the dimension of the input vector and the dimension of the output vector can differ. This behaviour depends on the dimensions of the transformation matrix, which determines how vectors are transformed from one space to another.
More Columns than Rows :
Dimension of Input Space is>Dimension of Output Space
In other words, the transformation squishes space into a lower dimension
More Rows than Columns :
Dimension of Input Space is<Dimension of Output Space
In other words, the transformation elevates the vector space into a higher dimension
Equal Number of Columns and Rows :
Dimension of Input Space is=Dimension of Output Space
In other words, the transformation preserves the dimensionality of space
Dimension Compatibility for Matrix-Vector Multiplication
Once we understand how the shape of a matrix relates to the dimensionality of the input and output spaces, it becomes clear that there are constraints on matrix-vector multiplication.
For matrix-vector multiplication to be possible, the dimension of the input vector must match the number of columns in the matrix
Matrix Notation
A matrix with m rows and n columns is denoted as an m×n matrix. Each element in the matrix is represented by Ωij, where i is the row index and j is the column index.
In many situations, we encounter the need to perform a series of successive linear transformations. The fundamental principle remains constant: the final transformation can be determined by tracking the set of basis vectors.
The traditional approach to compute the final position of a vector involves multiplying the vector by the corresponding matrices one by one, in a nested fashion:
However, this method can become tedious and cumbersome, especially when dealing with numerous transformation steps.
A more efficient approach is to find the overall effect of the successive transformations using a composite matrix.
The matrix responsible for directly transforming the input vector to the final position is known as the composite matrix.
Composite MatrixV=Vtransformed(overall)
Since multiplying the vector by the composite matrix has the same effect as multiplying it by a series of individual matrices, we can establish the following equality:
An interesting aspect to note is that matrix multiplication is performed from right to left in this context, which is analogous to the notation used for composite functions.
To understand matrix multiplication, let's delve into the transformation of basis vectors in each step
second transformation⎣⎢⎢⎢⎡😭11😭21😭31:😭12😭22😭32:😭13😭23😭33:⋯⋯⋯⎦⎥⎥⎥⎤first transformation⎣⎢⎢⎢⎡🙏11🙏21🙏31:🙏12🙏22🙏32:🙏13🙏23🙏33:⋯⋯⋯⎦⎥⎥⎥⎤=overall transformation⎣⎢⎢⎢⎡💀11💀21💀31:💀12💀22💀32:💀13💀23💀33:⋯⋯⋯⎦⎥⎥⎥⎤
Recall that the columns of a matrix represent the transformed basis vectors. Therefore, the columns of the composite matrix represent the final transformed basis vectors
Thus, to find the final transformed basis vectors, we determine how the basis vectors change after each transformation. This can be expressed through matrix-vector multiplication by applying the second transformation to the initially transformed basis vectors.
Therefore the computation of matrix-matrix multiplication can be reduced to matrix-vector multiplication, something we are already familiar with
Take out your pen and paper and re-derive the expression for matrix-matrix multiplication provided above. Understanding the derivation will clarify the process, so there’s no need to memorize it.
The sequence of transformations determines the final result because each step modifies the space or vectors, affecting subsequent steps.
Each transformation changes the vectors it is applied to, and subsequent transformations operate on these altered vectors.
The intermediate state produced by the first transformation becomes the input for the next, making the final result sensitive to the order of operations.
In other words, matrix-matrix multiplication is non-commutative ( i.e. ♣^♠^=♠^♣^)
Not all matrix multiplications are allowed due to specific dimensional constraints imposed by the matrices involved.
Matrix multiplication can be viewed as a series of matrix-vector multiplications. Therefore, the constraints that apply to matrix-vector multiplication also apply to matrix multiplication.
Consequently, Matrix-Matrix multiplication is only possible when the number of columns in the matrix on the left is equal to the number of rows in the matrix on the right
We can predict the dimension of a composite matrix by examining the dimensions of the matrices being multiplied.
The composite matrix will have the same number of columns as the dimension of the input vector and the same number of rows as the dimension of the output vector.
Specifically, the matrix on the right dictates the number of columns in the composite matrix, while the matrix on the left determines the number of rows.
To determine if matrix multiplication is allowed and to predict the dimensions of the resulting composite matrix, use the following mnemonic:
😭row×😭columnMatrix 😭×🙏row×🙏columnMatrix 🙏
If 😭columns=🙏rows, then matrix multiplication is permitted.
When matrix multiplication is allowed, the composite matrix will have dimensions 😭rows×🙏columns.
