Matrix multiplication – Serlo

In this article, you learn how to multiply matrices. We will see that matrix multiplication is equivalent to the composition of linear maps. We will also prove some properties of the matrix multiplication.

Introduction

Bearbeiten

How can we multiply matrices?

Bearbeiten

In the article on matrices of linear maps, we learned how we can use matrices to describe linear maps   between finite-dimensional vector spaces   and  . This requires fixing a basis   of   and a basis   of  , with respect to which we can define the mapping matrix  . In a plane of coordinates, this matrix descibes what the linear mapping   does with a vector  :

 

where   is the coordinate mapping with respect to  , which maps a vector   to the coordinate vector   with respect to  . Similarly,   is the coordinate mapping with respect to  .

We can concatenate linear maps   and   by executing them one after the other, which results in a linear map  . Can we define a suitable "concatenation" of matrices? By suitable, we mean that the "concatenation" of the matrices corresponding to   and   should become the matrix of the map  . We will call this "concatenation" of the matrices also the matrix product since it will turn out to behave almost like a product of numbers.

For example, let's consider two matrices   and   with the corresponding linear mas

 

and

 

given by matrix-vector-multiplication. Then   is the matrix of   (with respect to the standard bases in   and  ), and   is the matrix of   (with respect to the standard bases in   and  ). The product   of   and   should then be the matrix of  .

However, in order to be able to execute the maps   and   one after the other, the target space of   must be equal to the space on which   is defined. This means that  , i.e.  . Therefore, the number of columns of   must be equal to the number of rows of  , otherwise we cannot define the product matrix  .

Computing the product matrix

Bearbeiten

What is the product   of   and   corresponding to the map  ? To compute it, we need to calculate the images of the standard basis vectors   under the map  . They will form the columns of the matrix of  , that is, the matrix  .

We denote the entries of   by   and those of   by  , i.e.   and  . We also denote the desired matrix of   by  .

For   and  , the entry   is given by the definition of the matrix representing   by the  -th entry of the vector  . We can easily calculate it using the definition of   and   using the definition of matrix-vector-multiplication:

 

This defines all entries of the matrix   and we conclude

 

This is exactly the product   of the two matrices   and  .

Definition and rule of thumb

Bearbeiten

Mathematically, we can also understand matrix multiplication as an operation (just as the multiplication of real numbers).

Definition (Matrix multiplication)

The matrix multiplication is an operation

 .

It sends two matrices   and   to the matrix  , given by

 

for   and  .

However, there is an important difference to the multiplication of real numbers: With matrices, we have to make sure that the dimensions of the matrices we want to multiply match.

Hint

The two matrices do not have to be of the same size, but the number of columns of the matrix   must be equal to the number of rows of the matrix  . The result then has the number of rows of the left-hand matrix   and the number of columns of the right-hand matrix  . This means that two matrices   can only be multiplied if  .

Warning

The two matrices   with   can never be multiplied.

 
To calculate the matrix product, we use the scheme "row times column".

Rule of thumb: row times column

According to the definition, each entry in the product   is the sum of the component-wise multiplication of the elements of the  -th row of   with the  -th column of  . This procedure can be remembered as row times column, as shown in the figure on the right.

Concrete example

Bearbeiten

Example 1

Bearbeiten

We consider the following two matrices   and  :

 

We are looking for the matrix product  . This matrix has the form

 

We have to calculate the individual entries  . We will do this here in detail for the entry  . The calculation of the other entries works analogously.

According to the formula

 

This calculation can also be seen as the "multiplication" of the 2nd row of   with the 3rd column of  . To illustrate this, we mark the entries from the sum in the matrices. We have the sum

 

These are the following entries in the matrices:

 

In this way, we can also determine the other entries of   and obtain

 

Example 2

Bearbeiten

We consider the following matrices   and  :

 

In this case, we can calculate both   and  . Let  . Then   is a  -matrix  . We calculate its only entry:

 

Thus,  .

Let  . Then   is a  -matrix. We can calculate the entries of   by the scheme "row times column". For example, the first entry of   is the first row of   times the first column of  , i.e.  . If we do this with each entry, we get

 

Example 3

Bearbeiten

In this example, we want to illustrate that the matrix multiplication really corresponds to "concatenating two matrices". That means, if we have two matrices   and   that we apply to a vector  , then we always have  . As an example, let   and   be the following matrices with entries in  :

 

Let further  . We check that  . To do so, we first calculate the matrix product  :

 

Now we multiply this matrix with  :

 

Next, we compute  .

 

We now apply   to this vector:

 

Indeed, here we have  .

Properties of matrix multiplication

Bearbeiten

We now collect a few properties of the matrix multiplication.

Shortening rule for matrices representing linear maps

Bearbeiten

The following theorem shows that matrix multiplication actually reflects the composition of linear mappings.

Theorem (Shortening rule for matrices representing linear maps)

Let   and   be linear maps between finite-dimensional vector spaces. Furthermore, let   be a basis of  , let   be a basis of   and   a basis of  . Then we can "shorten the  ":

 

Proof (Shortening rule for matrices representing linear maps)

We set   and  . Further, the matrices of   and   are given by   and  .

By definition of the matrix of a linear map, we know that the   are the unique scalars with

 

for all  . In oprder to prove  , we need to verify that

 

And indeed,

 

By uniqueness of the coordinates in the linear combination of  , we conclude  .

Warning

For the shortening rule, it is important that the same ordered basis   of   is chosen in both cases for the matrices representing   and  . If   is taken with respect to a different basis   of  , then the shortening rule is no longer true: The following is in general a false statement:

 

As matrices representing linear maps depend on the order of the basis vectors, the shortening rule also becomes false if   is a rearrangement of  .

Associativity of matrix multiplication

Bearbeiten

Theorem (Associativity of matrix multiplication)

For   we have

 

Proof (Associativity of matrix multiplication)

First, we check that the sizes of the matrices that we want to multiply are compatible. This is directly visible for the products   and  . Now   and  , so the products on both sides of the equation are well-defined: they are both in  .

Now we look at the individual components of the matrices to verify the equality. Let  .

 

Compatibility with scalar multiplication

Bearbeiten

Theorem (Compatibility with scalar multiplication)

Let   and  . Then:

 

Note that " " refers to both scalar multiplication ("scalar times matrix") and matrix multiplication ("matrix times matrix").

Proof (Compatibility with scalar multiplication)

 

Distributivity of matrix multiplication

Bearbeiten

Here we must be careful that the sizes of the matrices are compatible.

Theorem (First distributive law)

For   we have

 

Proof (First distributive law)

 

Theorem (Second distributive law)

For   we have

 

Proof (Second distributive law)

 

Left and right neutral element of matrix multiplication

Bearbeiten

We denote the entries of the unit matrix with  , i.e.  . Then

 

Theorem (The unit matrix is a left- and right-neutral element of the matrix multiplication)

Let  . Then

 

Proof (The unit matrix is a left- and right-neutral element of the matrix multiplication)

Proof step:  

We prove this equality by direct multiplication. The following holds for all   and for all  :

 

For the last equality, we used the fact that   if   and  . Since each entry of   matches the entry of   at the same position, the two matrices are equal.

Proof step:  

We proceed as in the first proof step. For all   and for all   we have:

 

This proves equality of both sides.

In other words, the unit matrix (of the correct size) is the left- or right-neutral element with respect to matrix multiplication.

Non-commutativity

Bearbeiten

Example (Non-commutativity of  -matrices)

For   matrices, we can see that commutativity fails within the following example: On the one hand

 

and on the other hand

 

So the order of the matrix multiplication matters!

Warning

In general,  , so the matrix product is not commutative.

The commutative law only applies in a few special cases (e.g. products of diagonal matrices).

As the number of rows and columns of the matrices must match, it is even possible that one of the two products is not even defined! For example, for   the product   is defined, but the product   is not defined.

Further reading

Bearbeiten

Hint

If we multiply two  -matrices, the result is again an  -matrix. We now know two inner operations on the set  : the addition of matrices

 

and the matrix multiplication

 

From the article on the vector space structure on matrices, we already know that   is an Abelian group. It follows from the properties of matrix multiplication that   is even a unital ring (i.e., a ring that has a unit element): The multiplication   is associative, there is a neutral element   and the distributive laws apply.

However, the ring of matrices is generally not commutative, as we have seen above. Also note that we only have such a ring structure for square matrices, as otherwise the multiplication of two elements is not defined.