Span and Linear Independence
Linear Combinations ¶
A vector space is closed under vector addition and scalar multiplication. We can generalize this property by defining the notion of a linear combination.
Definition 13: Linear Combination
Let be a vector space over a field , and let . A vector is a linear combination of iff there exist scalars such that:
The scalars are called the weights of the linear combination.
That is, a linear combination of some set of vectors is the sum of certain scalar multiples of vectors in . Since a vector space is closed under vector addition and scalar multiplication, any linear combination of vectors in must itself be a vector in .
Corollary 4
Any linear combination of a set of vectors from is itself a vector in .
Example.
The vector is a linear combination of the vectors and , since we can write:
Nonexample.
The vector is not a linear combination of the vectors and , since any linear combination of these two vectors must be of the form:
The two components of this vector must be equal, which is not the case for .
Terminology
A linear combination is trivial if all of its weights are zero, and nontrivial otherwise. A trivial linear combination, then, is always equal to .
Theorem 11: Alternative Subspace Criteria
Let be a vector space over a field , and let . Then is a subspace of if is closed under linear combinations, i.e. for all and all scalars , we have:
Proof: Alternative Subspace Criteria.
Suppose is a vector space over , and let . Also suppose that is closed under linear combinations. We will show that satisfies the three Subspace Criteria.
- Existence of additive identity. is the trivial linear combination, so .
- Closure under vector addition. Let . Then is a linear combination of and with weights , so .
- Closure under scalar multiplication. Let and . Then is a linear combination of with weight , so . ∎
Spans and Spanning Sets
Clearly, there are vectors that can be written as linear combinations of some set of vectors, but there may also exist vectors that can’t. The span of a set of vectors describes the set of all vectors that can be written as linear combinations of that set:
Definition 14: Span
Let be a vector space over a field , and let . The span of , denoted or , is the set of all linear combinations of :
Theorem 12: Spans are Subspaces
Let be a vector space, and let . Then is a subspace of .
Proof: Spans are Subspaces.
Let be a vector space over , and let . Checking the three Subspace Criteria:
Existence of additive identity. Fix all weights . Then for any linear combination of , we have:
Closure under vector addition. Let , so there exist weights and such that:
Then:
is closed under field addition, so for all . Thus, is a linear combination of , so .
Closure under scalar multiplication. Let , so there exist weights such that:
Let . Then:
is closed under field multiplication, so for all . Thus, is a linear combination of , so . ∎
Example.
Let and let . Define , so:
By Theorem 12, is a subspace of .
Terminology: Spanning Set
Let be a vector space and let . All of the following statements are equivalent:
- .
- is a spanning set for .
- spans .
- is the subspace generated by .
That is, to say that “ spans ” is to say that every vector in can be written as a linear combination of vectors in .
Example: Show that the set of vectors spans .
Nonexample.
The set of vectors does not span , since any linear combination of these two vectors must be of the form:
The two components of this vector must be equal, so there is no way to write a vector like as a linear combination of these two vectors.
Important
The empty set spans the trivial subspace .
Theorem 13: Subspaces are Spans
Let be a vector space, and let be a subspace of . Then there exists a finite subset of that spans :
That is, we can take vectors from to generate the subspace .
Proof: Subspaces are Spans.
Let be a vector space, and let be a subspace of . If , then we can take , so assume that .
TODO ∎
Theorem 14: Spanning Set Theorem
Let , and let be the subspace generated by . Then if some vector is a linear combination of the other vectors in , then the set formed from by removing still spans . Formally:
Proof: Spanning Set Theorem.
Theorem 15: Union of Spanning Sets
Let and be sets of vectors, and let and be the subspaces generated by and , respectively. Then the union of and spans the sum of and . Formally:
Proof: Theorem 15.
Let and be sets of vectors, and let and be the subspaces generated by and , respectively.
Suppose . Then there exist weights and vectors such that:
We can partition the vectors into two groups: those that are in and those that are in . Let the vectors in the first group be denoted by , and let the vectors in the second group be denoted by . Then we can rewrite the equation above as follows:
The first sum is a linear combination of vectors in , so the first sum is a vector in ; call it . Similarly, the second sum is a linear combination of vectors in , so the second sum is a vector in ; call it . Thus, we can write:
Therefore, , so .
Conversely, suppose that . Then there exist vectors and such that:
Since , there exist weights and vectors such that:
Similarly, since , there exist weights and vectors such that:
Thus, we can write:
The vectors and are all in , so is a linear combination of vectors in . Therefore, , so .
By double containment, . ∎
Linear Independence
For some set of vectors , there may be multiple configurations of weights to write a vector in . For example, if , then we can write the vector as a linear combination of in two different ways:
Some sets, called linearly independent sets, have the property that each vector in their span (except for ) can be written as a linear combination of the set with only one unique combination of weights.
Definition 15: Linear Independence
A set of vectors is linearly independent iff each vector in can be written as a linear combination of in only one way.
Formally is linearly independent iff for some sets of weights and :
We can restate Definition 15 as: to say that a set of vectors is linearly independent means that if two linear combinations of vectors in are equal, then their weights must be equal.
Definition 16: Linear Dependence
A set of vectors is linearly dependent iff it is not linearly independent.
Terminology
To say that vectors and are linearly independent means that the set is linearly independent. Also, to say that is a linearly independent set in means that is a set of vectors taken from that is linearly independent.
Theorem 16: Formalism for Linear Independence
Let be a vector space over , and let . Then is linearly independent if and only if the only solution to the equation:
is the trivial solution .
Proof: Formalism for Linear Independence.
Let be a vector space over , and let . Let be the statement that is linearly independent, and let be the statement that the only solution to the equation is the trivial solution .
. Assume is linearly independent. Let be scalars such that:
Then, by Theorem 3, realize that:
Since is linearly independent, the weights of these two linear combinations must be equal, so . Therefore, the only solution to the equation is the trivial solution.
Assume that the only solution to the equation is the trivial solution. Let and be scalars such that:
Then we can rearrange this equation as follows:
By our assumption, the only solution to this equation is the trivial solution, so for all , so , , …, . Therefore, if two linear combinations of vectors in are equal, then their weights must be equal, so is linearly independent.
Since and , we have . ∎
Following this theorem, if any one of the in the equation is the zero vector , then any weight can be used for that vector and the equation in Theorem 16 would still hold, so the set would be linearly dependent. Thus, we have the following corollary:
Corollary 5
If , then is linearly dependent.
Theorem 17: Characterization of Linearly Dependent Sets
An indexed set with is linearly dependent iff there exists some such that is a linear combination of the preceding vectors, i.e. .
This theorem can be restated to say that a set of vectors is linearly dependent iff there is a vector in that can be written as a linear combination of the other vectors in .
Proof: Theorem 17.
Let be an indexed set of vectors with . Let be the statement that is linearly dependent, and let be the statement that there exists some such that .
Assume is linearly dependent. Then there exist nontrivial weights such that:
Since , we can choose weights such that and for all for some . Then we can rearrange the equation as follows:
Since , we can multiply both sides of the equation by to get:
The right-hand side of this equation is a linear combination of the vectors , so .
Assume that there exists some such that . By the definition of span, there exist weights such that:
We can rearrange this equation as follows:
No matter what any of the are, a nontrivial weight is used for to get a linear combination that equals , so by Theorem 16, is linearly dependent.
Since and , we have . ∎
Important
Theorem 17 does not guarantee that every vector in a linearly dependent set is a linear combination of the preceding vectors; only that at least one vector is.
It follows, then, that adding vectors to a linearly dependent set results in another linearly dependent set, because if there is already a vector in the set that can be written as a linear combination of the preceding vectors, then adding more vectors won’t change that fact:
Corollary 6
If is a linearly dependent set and , then is also a linearly dependent set.
It also happens that, removing a vector from a linearly independent set results in another linearly independent set:
Proposition 5
If is a linearly independent set and , then is also a linearly independent set.
Proof: Proposition 5.
Since the empty set is a subset of every set, it follows that the empty set is linearly independent:
Corollary 7
The empty set is a linearly independent set.
Corollary 7 can also be vacuously deduced. Since any linear combination of the empty set must be the zero vector, and there is only one way to write the zero vector as a linear combination of the empty set (with no weights at all).
Also, the negation of Theorem 17 gives us a characterization of linearly independent sets:
Corollary 8: Characterization of Linearly Independent Sets
An indexed set with is linearly independent iff for every , .
Theorem 18: Existence of Linearly Independent Subsets
Let be a set of vectors, and let be the subspace generated by . Then if is a linearly dependent set, then there exists a proper subset of , , such that is linearly independent and still spans , i.e. .
Proof: Theorem 18.
Theorem 19: Linear Dependence of
Let be a field, and define as in Definition 7. The set is linearly dependent in if contains more than vectors, i.e. if .
Proof: Theorem 19.
Span and Linear Independence of Infinite Sets ¶
The previous definitions of span and linear independence build from the idea of linear combinations, which only involve finite sets of vectors.
However, we can generalize the definitions of span and linear independence to infinite sets of vectors:
Definition 17: Span of an Infinite Set
Let be an infinite subset of a vector space . is a spanning set for iff for every vector , there exists a finite subset of such that .
Definition 18: Linear Independence of an Infinite Set
Let be an infinite subset of a vector space . is linearly independent iff for every finite subset of , is linearly independent.
Example.
Exercises
Exercise 23.
Exercise 24.
Exercise 25.
Suppose and . Show that:
Exercise 26.
Exercise 27.
Exercise 28.
Exercise 29.
Exercise 30.
Exercise 31.
is the set of all infinite sequences of real numbers. Show that , a subset of defined as follows, is a linearly independent set in :
Exercise 32: Telescoping Linear Independence.
Let be a vector space, and suppose is a linearly independent set in . Define a new set of vectors by:
That is, . Show that is also a linearly independent set in .