Linear Algebra

Linear Algebra contributes a reliable share of the 13-mark Engineering Mathematics block, and it is the most mechanically scorable topic in the whole paper. The questions repeat, the methods are short, and almost nothing depends on insight.

One number decides almost everything: the rank of the matrix.

Rank fixes whether a system has no solution, one solution, or infinitely many. It fixes whether a matrix is invertible. It fixes the dimension of the column space and, through the rank-nullity theorem, the dimension of the null space. And when eigenvalues are involved, the rank of $A-\lambda I$ fixes how many independent eigenvectors that eigenvalue contributes.

The second organising fact is that eigenvalues carry two invariants you can read off without computing them. Their sum is the trace and their product is the determinant, which is enough to answer many questions in one line and to check every computation in the rest.

So the working method is: compute the rank, read off trace and determinant, and only then do heavier work if the question still demands it.

1. Rank

The rank of a matrix is the number of non-zero rows after reduction to row echelon form, equivalently the maximum number of linearly independent rows, equivalently the maximum number of linearly independent columns.

Row rank equals column rank always, which is why an $m\times n$ matrix has rank at most $\min (m,n)$.

A matrix of order $n$ is called full rank when its rank is $n$, and that is exactly the condition for invertibility.

Elementary row operations never change the rank, which is what makes row reduction the standard method. They do change the determinant in predictable ways, which is a separate matter and a standard source of errors.

The rank-nullity theorem ties rank to the null space:

$\text{rank}(A)+\text{nullity}(A)=n$

where $n$ is the number of columns. The nullity is the dimension of the solution space of $AX=0$, so a full-rank square matrix has nullity zero and only the trivial solution.

2. Determinants

The determinant is defined only for square matrices, and its properties matter far more than its computation.

Multiplying the whole $n\times n$ matrix by $k$ multiplies the determinant by $k^n$, not by $k$, because every one of the $n$ rows is scaled. This is examined directly and caught out regularly.

Two products behave simply:

$\det (AB)=\det (A)\det (B)\det (A^{-1})=\frac{1}{\det (A)}$

Note that $\det (A+B)$ has no such rule and is not $\det A+\det B$.

For a triangular matrix, the determinant is the product of the diagonal entries, which is why row reduction to triangular form is the practical computation method.

The determinant of an $n\times n$ matrix with all entries equal is 0 for $n\ge 2$, since all rows are identical.

The adjoint supplies the inverse and carries two results that are examined directly.

$A\cdot \text{adj}(A)=\det (A)I$

From this, $A^{-1}=\text{adj}(A)/\det (A)$, which exists precisely when the determinant is non-zero, tying invertibility back to full rank.

Taking determinants of both sides of the adjoint identity gives $\det (\text{adj}A)=(\det A{)}^{n-1}$ for an $n\times n$ matrix. The exponent $n-1$ rather than $n$ is what questions test, and it follows because the right-hand side is $(\det A{)}^n$ divided by $\det A$.

A matrix and its inverse share eigenvectors, with eigenvalues reciprocated, which is why questions about $A^{-1}$ rarely require computing it.

3. Systems of Linear Equations

For $AX=b$ with $A$ of order $m\times n$, everything follows from comparing two ranks: that of $A$ and that of the augmented matrix $[A\mid b]$.

When infinitely many solutions exist, the number of free parameters is $n-\text{rank}(A)$, which is the nullity. This is asked directly as a NAT question.

For a homogeneous system $AX=0$, the augmented column is zero, so the two ranks always agree and the system is always consistent — the zero vector is always a solution.

A homogeneous system has a non-trivial solution if and only if $\text{rank}(A)<n$, which for a square matrix means $\det A=0$.

A homogeneous system with more unknowns than equations always has a non-trivial solution, since the rank cannot exceed the number of equations.

4. Vector Spaces, Basis and Dimension

A basis is a linearly independent spanning set, and the dimension is the number of vectors in any basis.

A set of $n$ vectors in an $n$-dimensional space is a basis if and only if the matrix formed from them is non-singular, so the test reduces to a determinant.

Any set of more than $n$ vectors in an $n$-dimensional space is linearly dependent, and any set of fewer than $n$ cannot span it.

The column space of $A$ has dimension equal to the rank; the null space has dimension equal to the nullity. The row space and column space always have the same dimension even when they live in different spaces, which is the rank equality restated.

5. Eigenvalues and Eigenvectors

A scalar $\lambda$ is an eigenvalue of $A$ if there is a non-zero vector $X$ with $AX=\lambda X$, equivalently if

$\det (A-\lambda I)=0$

The two invariants are worth using before any computation.

$\text{trace}(A)=\sum {\lambda }_i\det (A)=\prod {\lambda }_i$

For a 2 by 2 matrix these two equations determine the eigenvalues completely, which is usually faster than expanding a characteristic polynomial.

A rich set of derived facts is examined directly and follows from the definition.

The eigenvectors are unchanged by all of these transformations, which is often the actual point of the question.

A singular matrix has 0 as an eigenvalue, since $\det (A-0\cdot I)=\det A=0$. Conversely, a matrix with 0 as an eigenvalue is singular.

For a triangular matrix, the eigenvalues are simply the diagonal entries, and no computation is needed at all.

Cayley-Hamilton states that every matrix satisfies its own characteristic equation. For a 2 by 2 matrix with characteristic polynomial ${\lambda }^2-t\lambda +d$, this gives $A^2=tA-dI$, which reduces any high power of $A$ to a linear combination of $A$ and $I$.

Special matrix classes have constrained eigenvalues, and each is a one-line question.

A real symmetric matrix is positive definite when all its eigenvalues are strictly positive, positive semi-definite when they are non-negative, and indefinite when both signs appear.

Two equivalent tests are used in practice. Every leading principal minor must be positive, which is Sylvester's criterion and is faster for small matrices. Or all eigenvalues must be positive, which is faster when they have already been computed for another part of the question.

Positive definiteness is exactly the condition under which Cholesky decomposition exists and under which the quadratic form $X^{T}AX$ is positive for every non-zero $X$.

6. Diagonalisation

A matrix is diagonalisable if $P^{-1}AP$ is diagonal for some invertible $P$, whose columns are the eigenvectors.

A matrix of order $n$ is diagonalisable if and only if it has $n$ linearly independent eigenvectors.

Two multiplicities decide this. The algebraic multiplicity of $\lambda$ is its multiplicity as a root of the characteristic polynomial. The geometric multiplicity is the dimension of the eigenspace, which by rank-nullity is $n-\text{rank}(A-\lambda I)$.

Geometric multiplicity is always at least 1 and never exceeds algebraic multiplicity, and the matrix is diagonalisable exactly when the two are equal for every eigenvalue.

The immediate consequence: a matrix with $n$ distinct eigenvalues is always diagonalisable, since each contributes an independent eigenvector. Repeated eigenvalues are the only case where the check is needed.

Every real symmetric matrix is diagonalisable, and moreover by an orthogonal $P$.

Two matrices are similar if $B=P^{-1}AP$ for some invertible $P$, and similar matrices share their characteristic polynomial. They therefore share eigenvalues, trace, determinant and rank, though not their eigenvectors.

Diagonalisation is exactly the statement that a matrix is similar to a diagonal one, so the diagonal entries of that diagonal matrix are the eigenvalues, in some order.

The practical payoff is computing powers. If $A=PD{P}^{-1}$, then $A^k=P{D}^k{P}^{-1}$, and raising a diagonal matrix to a power is elementwise. This is why diagonalisability is asked about far more often than the diagonalising matrix itself.

7. LU Decomposition

LU decomposition writes $A=LU$ with $L$ lower triangular and $U$ upper triangular, which turns solving $AX=b$ into two triangular solves.

The point is reuse. Factoring costs the same order as Gaussian elimination, but once $L$ and $U$ are available, each new right-hand side costs only the two cheap triangular solves.

Two conventions fix the otherwise underdetermined factorisation.

LU decomposition without row interchanges exists when every leading principal minor is non-zero. If a leading minor vanishes, pivoting is required and the factorisation becomes $PA=LU$.

Solving proceeds in two steps: solve $LY=b$ by forward substitution, then $UX=Y$ by back substitution. The order matters and reversing it is a standard error, since $L$ is the factor applied first.

For a symmetric positive definite matrix, Cholesky decomposition writes $A=L{L}^T$ and costs roughly half as much.

8. Worked Examples

Example 1. For what value of $k$ does the system $x+y+z=1$, $2x+3y+kz=3$, $x+ky+3z=2$ have no solution?

Row-reduce the augmented matrix.

Subtracting twice row 1 from row 2 gives $y+(k-2)z=1$. Subtracting row 1 from row 3 gives $(k-1)y+2z=1$.

Now eliminate $y$: subtract $(k-1)$ times the new row 2 from the new row 3, giving

$\left[2-(k-1)(k-2)\right]z=1-(k-1)$, that is, $(-k^2+3k)z=2-k$, or $-k(k-3)z=2-k$.

For no solution, the coefficient of $z$ must vanish while the right side does not.

The coefficient vanishes at $k=0$ and $k=3$. At $k=0$ the right side is 2, non-zero, so there is no solution. At $k=3$ the right side is $-1$, also non-zero, so again no solution.

So the system is inconsistent for $k=0$ and $k=3$.

The structural point is that inconsistency requires rank$(A)<$ rank$([A|b])$, which shows up in row-reduced form as a row of zeros on the left with a non-zero entry on the right.

Example 2. The trace of a 3 by 3 matrix is 6 and its determinant is 6. Two of its eigenvalues are 1 and 2. Find the third.

Use the invariants rather than the matrix.

The sum of eigenvalues equals the trace: $1+2+{\lambda }_3=6$, so ${\lambda }_3=3$.

Check against the determinant: the product should be $1\times 2\times 3=6$, which matches the given determinant exactly.

The third eigenvalue is 3.

This double check is worth performing every time. If the trace and determinant conditions had disagreed, it would indicate either an arithmetic slip or an inconsistent question, and catching that costs a few seconds.

Example 3. Find the rank and nullity of the matrix with rows $(1,2,3)$, $(2,4,6)$, $(1,1,1)$.

Row-reduce. Row 2 is exactly twice row 1, so subtracting twice row 1 from row 2 gives a zero row.

Subtracting row 1 from row 3 gives $(0,-1,-2)$.

The reduced rows are $(1,2,3)$, $(0,-1,-2)$ and $(0,0,0)$.

Two non-zero rows remain, so the rank is 2.

By rank-nullity with $n=3$ columns: nullity $=3-2=1$.

So $AX=0$ has a one-parameter family of solutions, and the matrix is singular, which also means 0 is one of its eigenvalues.

Example 4. Is the matrix $\left(\begin{array}{cc}3& 1\\ 0& 3\end{array}\right)$ diagonalisable?

It is triangular, so the eigenvalues are the diagonal entries: 3 and 3.

The eigenvalue 3 has algebraic multiplicity 2.

For the geometric multiplicity, compute the rank of $A-3I$, which is the matrix with rows $(0,1)$ and $(0,0)$. Its rank is 1.

Geometric multiplicity $=n-\text{rank}(A-3I)=2-1=1$.

Since the geometric multiplicity 1 is less than the algebraic multiplicity 2, the matrix has only one independent eigenvector and is not diagonalisable.

This is the canonical example: a repeated eigenvalue is necessary but not sufficient for failure, and the rank computation is what decides it.

Example 5. Use Cayley-Hamilton to compute $A^3$ for $A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$.

Trace is 4 and determinant is $4-1=3$, so the characteristic equation is ${\lambda }^2-4\lambda +3=0$.

By Cayley-Hamilton, $A^2-4A+3I=0$, so $A^2=4A-3I$.

Multiply by $A$: $A^3=4A^2-3A$.

Substitute the expression for $A^2$: $A^3=4(4A-3I)-3A=16A-12I-3A=13A-12I$.

Numerically, $13A=\left(\begin{array}{cc}26& 13\\ 13& 26\end{array}\right)$ and subtracting $12I$ gives $\left(\begin{array}{cc}14& 13\\ 13& 14\end{array}\right)$.

Verify with eigenvalues: the eigenvalues of $A$ are 1 and 3, so those of $A^3$ are 1 and 27. Their sum should be the trace of $A^3$, which is $14+14=28=1+27$. Their product should be the determinant, $196-169=27=1\times 27$. Both check.

Example 6. Find the LU decomposition of $A=\left(\begin{array}{cc}2& 3\\ 4& 7\end{array}\right)$ by the Doolittle method, and solve $AX=(5,11{)}^T$.

Doolittle sets the diagonal of $L$ to 1, so write $L$ with entries 1, 0 in the first row and $l_{21}$, 1 in the second, and $U$ with entries $u_{11},u_{12}$ and $0,u_{22}$.

Multiplying and matching entries: $u_{11}=2$ and $u_{12}=3$ from the first row.

From the second row, $l_{21}{u}_{11}=4$ gives $l_{21}=2$, and $l_{21}{u}_{12}+u_{22}=7$ gives $6+u_{22}=7$, so $u_{22}=1$.

Now solve in two stages. Forward substitution on $LY=b$: $y_1=5$, and $2y_1+y_2=11$ gives $y_2=1$.

Back substitution on $UX=Y$: $x_2=1$ from the last row, and $2x_1+3(1)=5$ gives $x_1=1$.

The solution is $x_1=1$, $x_2=1$, which satisfies both original equations.

Note that the leading principal minor $u_{11}=2$ is non-zero, which is exactly the condition permitting this factorisation without any row interchange.

Summary

Rank decides almost everything: invertibility, the number of solutions, the dimensions of the column and null spaces, and the number of independent eigenvectors for a given eigenvalue.

Row operations preserve rank but change the determinant predictably. Scaling an $n\times n$ matrix by $k$ scales the determinant by $k^n$.

A system is consistent exactly when rank$(A)$ equals rank$([A|b])$; it has one solution when that common rank is $n$, and infinitely many with $n-\text{rank}(A)$ free parameters otherwise.

A homogeneous system always has the zero solution, and has a non-trivial one exactly when the rank is less than the number of unknowns.

The trace is the sum of eigenvalues and the determinant is their product; use both before computing anything and again to check.

Powers, inverses, transposes and shifts transform the eigenvalues predictably and leave the eigenvectors alone.

Cayley-Hamilton reduces any power of a matrix to a low-degree combination.

Diagonalisability requires geometric multiplicity to equal algebraic multiplicity for every eigenvalue, so distinct eigenvalues always suffice and only repeated ones need the rank check.

LU decomposition needs non-zero leading principal minors, uses Doolittle or Crout to fix the convention, and is solved by forward substitution then back substitution in that order.