Section 17 Linear Dependence
In this activity, we will explore linear dependence and independence in the context of solving nonhomogeneous \(A x = b\) and homogeneous equations \(A x = 0\). Download this Rmd file.
Remember that we will use the pracma package to get the rref function, so we first load it in:
17.1 Example 1: a 7x9 integer matrix
Here is a 7 x 9 coeefficient matrix that we will use. These commands define it and echo it back.
A = cbind(
c(3, 0, 0, 1, -2, -4, 1),
c(5, -5, 0, 3, 3, 1, 4),
c(3, 5, -1, 1, -3, -3, 5),
c(4, -1, -2, 0, -1, 2, -3),
c(0, 17, 3, 0, -17, -29, 8),
c(-4, -1, -5, -2, -1, -4, 3),
c(5, 3, -4, -5, -2, -3, -1),
c(0, 5, -3, -2, -1, -5, 0),
c(37, -10, -27, -29, 4, 7, -24))
A## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
## [1,] 3 5 3 4 0 -4 5 0 37
## [2,] 0 -5 5 -1 17 -1 3 5 -10
## [3,] 0 0 -1 -2 3 -5 -4 -3 -27
## [4,] 1 3 1 0 0 -2 -5 -2 -29
## [5,] -2 3 -3 -1 -17 -1 -2 -1 4
## [6,] -4 1 -3 2 -29 -4 -3 -5 7
## [7,] 1 4 5 -3 8 3 -1 0 -24And here is a vector b that we hope to use in solving A x = b.
## [1] 382 51 -321 -314 -86 -170 -153You can augment A with b, and call it Ab, using cbind:
## b
## [1,] 3 5 3 4 0 -4 5 0 37 382
## [2,] 0 -5 5 -1 17 -1 3 5 -10 51
## [3,] 0 0 -1 -2 3 -5 -4 -3 -27 -321
## [4,] 1 3 1 0 0 -2 -5 -2 -29 -314
## [5,] -2 3 -3 -1 -17 -1 -2 -1 4 -86
## [6,] -4 1 -3 2 -29 -4 -3 -5 7 -170
## [7,] 1 4 5 -3 8 3 -1 0 -24 -153And row reduce using rref
## b
## [1,] 1 0 0 0 5 0 0 0 -2 8
## [2,] 0 1 0 0 -2 0 0 0 2 10
## [3,] 0 0 1 0 1 0 0 0 -3 -19
## [4,] 0 0 0 1 -2 0 0 0 3 21
## [5,] 0 0 0 0 0 1 0 0 0 6
## [6,] 0 0 0 0 0 0 1 0 6 61
## [7,] 0 0 0 0 0 0 0 1 0 817.1.1 Solution to the nonhomogeneous equations Ax = b
Write out the solution to Ax=b in parametric form using the following formatting. You just need to fill in the correct values of the vectors: \[ \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \\ x_7 \\ x_8 \\ x_9 \end{bmatrix} = \begin{bmatrix} 8 \\ 10 \\ -19 \\ 21 \\ 0 \\ 6 \\ 61 \\ 8 \\ 0 \end{bmatrix} + s \begin{bmatrix} -5 \\ 2 \\ -1 \\ 2 \\ 1 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} + t \begin{bmatrix} 2 \\ -2 \\ 3 \\ -3 \\ 0 \\ 0 \\ -6 \\ 0 \\ 1 \end{bmatrix} \]
Describe this solution space (by fixing up this sentence, which is incorrect right now):
the set of solutions to A x= b is a plane in \(\mathbb{R}^9\).
17.1.2 Solution to the nonhomogeneous equations Ax = 0
Now, describe the set of solutions to the homogeneous equations A x = 0. Again, you can just edit this: \[ \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \\ x_6 \\ x_7 \\ x_8 \\ x_9 \end{bmatrix} = \begin{bmatrix} p1 \\ p2 \\ p3 \\ p4 \\ p5 \\ p6 \\ p7 \\ p8 \\ p9 \end{bmatrix} + s \begin{bmatrix} u1 \\ u2 \\ u3 \\ u4 \\ u5 \\ u6 \\ u7 \\ u8 \\ u9 \end{bmatrix} + t \begin{bmatrix} v1 \\ v2 \\ v3 \\ v4 \\ v5 \\ v6 \\ v7 \\ v8 \\ v9 \end{bmatrix} \]
And describe, in words, the geometric relationship between the solutions to Ax=b and Ax=0.
your answer here
17.1.3 Linearly dependent columns
The columns of the matrix A are linearly dependent. You can see that in rref(A).
## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
## [1,] 1 0 0 0 5 0 0 0 -2
## [2,] 0 1 0 0 -2 0 0 0 2
## [3,] 0 0 1 0 1 0 0 0 -3
## [4,] 0 0 0 1 -2 0 0 0 3
## [5,] 0 0 0 0 0 1 0 0 0
## [6,] 0 0 0 0 0 0 1 0 6
## [7,] 0 0 0 0 0 0 0 1 0Discuss in your group how you see it. Then write out a dependence relation among the columns by filling in numbers for the weights in this equation
\[ 0 = c_1 \vec{a}_1 + c_2 \vec{a}_2 + c_3 \vec{a}_3 + c_4 \vec{a}_4 + c_5 \vec{a}_5 + c_6 \vec{a}_6 + c_7 \vec{a}_7 + c_8 \vec{a}_8 + c_9 \vec{a}_9. \] Challenge: give a dependency relation that none of the other groups in the class have.
This is telling us that there is some redundancy in the matrix A. Remove columns from A to get a new matrix M whose columns are linearly independent. You can do this by removing the appropriate columns from the code below:
M = cbind( # you need to edit this matrix
c(3, 0, 0, 1, -2, -4, 1),
c(5, -5, 0, 3, 3, 1, 4),
c(3, 5, -1, 1, -3, -3, 5),
c(4, -1, -2, 0, -1, 2, -3),
c(0, 17, 3, 0, -17, -29, 8),
c(-4, -1, -5, -2, -1, -4, 3),
c(5, 3, -4, -5, -2, -3, -1),
c(0, 5, -3, -2, -1, -5, 0),
c(37, -10, -27, -29, 4, 7, -24)
)
M## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
## [1,] 3 5 3 4 0 -4 5 0 37
## [2,] 0 -5 5 -1 17 -1 3 5 -10
## [3,] 0 0 -1 -2 3 -5 -4 -3 -27
## [4,] 1 3 1 0 0 -2 -5 -2 -29
## [5,] -2 3 -3 -1 -17 -1 -2 -1 4
## [6,] -4 1 -3 2 -29 -4 -3 -5 7
## [7,] 1 4 5 -3 8 3 -1 0 -24## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
## [1,] 1 0 0 0 5 0 0 0 -2
## [2,] 0 1 0 0 -2 0 0 0 2
## [3,] 0 0 1 0 1 0 0 0 -3
## [4,] 0 0 0 1 -2 0 0 0 3
## [5,] 0 0 0 0 0 1 0 0 0
## [6,] 0 0 0 0 0 0 1 0 6
## [7,] 0 0 0 0 0 0 0 1 0Your matrix should now be square (7x7) with linearly independent columns. R has a build in solve command, solve, that works for matrices of this form (i.e., square with linearly independent columns). You can try it here. First you need to un-comment-out the solve command. I have it commented out right now, because it does not work with the matrix M (above) until you remove its redundancies.
Now, you should get a unique solution to the equation M x = b, since M has no free variables, and it should be one of the solutions to the original question A x = b. Which solution is it? That is, which of the many solutions to A x = b are you getting here (forw which values of the paramters?).
Compare this with trying to use solve on the original equation A x = b with linearly dependent columns. The solve command in the next bit of code is commented out. Delete the comment command and try executing it.
17.2 A 5 x 6 Numerical Matrix
So far, all of the matrices we’ve worked with in this class have integer values. This is only so that the calulations are nice to do by hand. All of our theory works over the real numbers. Here we will look at a real matrix with numerical values, something you might find when dealing with real-world data.
B = cbind(
c(0.717, -0.274, 0.365, 0.482, -0.362),
c(0.587, -0.545, 0.5, -0.407, -0.597),
c(-0.441, 0.886, 0.784, -0.831, -0.594),
c(0.923, -0.466, 0.222, 0.867, 0.493),
c(-0.42, -0.745, -0.02, -0.44, 0.209),
c(0.621, 0.049, -0.134, -0.844, -0.31)
)
B## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 0.717 0.587 -0.441 0.923 -0.420 0.621
## [2,] -0.274 -0.545 0.886 -0.466 -0.745 0.049
## [3,] 0.365 0.500 0.784 0.222 -0.020 -0.134
## [4,] 0.482 -0.407 -0.831 0.867 -0.440 -0.844
## [5,] -0.362 -0.597 -0.594 0.493 0.209 -0.310and here is a vector d in \(\mathbb{R}^5\).
## [1] 5.886 -4.001 3.701 -6.621 -2.199Try answering some of these questions:
- Are the columns of B linearly independent?
- Do the columns of B span \(\mathbb{R}^5\)?
- Give the parametric solution to B x = d.
- What is the geometric form of this solution (e.g., a plane in \(\mathbb{R}^4\))?
- Remove redundancies from the columns of B to get a new matrix B2 and use solve to solve the equation B2 x = d. Which of the parametric solutions to you get.
17.3 Random Matrices
The following code generates a random 5 x 5 matrix. Every time you enter it, it will give you a new matrix. Use this to try to figure out how likely it is that a random square matrix has linearly dependent columns.
## [,1] [,2] [,3] [,4] [,5]
## [1,] 0.4309142 0.9371033 0.1555187 0.1272473 0.2699421
## [2,] 0.1995602 0.1219277 0.5489718 0.6682745 0.2940909
## [3,] 0.8913652 0.5469853 0.9376556 0.7771004 0.6748736
## [4,] 0.9834069 0.6289772 0.7376989 0.5634442 0.9972644
## [5,] 0.5259985 0.1250578 0.1330764 0.5062472 0.1494804Try the same using the following code that generates a random 5 x 6 matrix.
## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 0.3706911 0.2221418 0.7540778 0.55638256 0.9618519 0.6620513
## [2,] 0.9024540 0.9275586 0.0515013 0.03947789 0.8058286 0.9327116
## [3,] 0.9808654 0.8329409 0.8067109 0.98639964 0.6370426 0.7929425
## [4,] 0.3373219 0.1686389 0.6198179 0.07127725 0.1179277 0.0595287
## [5,] 0.5535146 0.7324437 0.2227262 0.88840395 0.3965214 0.2936185Try the same using the following code that generates a random 5 x 4 matrix.
## [,1] [,2] [,3] [,4]
## [1,] 0.57234350 0.07596171 0.5694379 0.7865926
## [2,] 0.30985092 0.51236417 0.9843783 0.7016393
## [3,] 0.50260886 0.49570859 0.8307495 0.4697621
## [4,] 0.08012065 0.88655237 0.6424226 0.4507889
## [5,] 0.99767854 0.15415287 0.1375570 0.7938970## [,1] [,2] [,3] [,4]
## [1,] 1 0 0 0
## [2,] 0 1 0 0
## [3,] 0 0 1 0
## [4,] 0 0 0 1
## [5,] 0 0 0 0In each of these cases, how likely is it that the columns of the matrix spans all of \(\mathbb{R}^4\)?