PROFESSOR: Hey, we’re back. Today we’re going to do a
singular value decomposition question. The problem is really
simple to state: find the singular value
decomposition of this matrix C equals [5, 5; -1, 7]. Hit pause, try it yourself,
I’ll be back in a minute and we can do it together. All right, we’re back,
now let’s do it together. Now, I know Professor
Strang has done a couple of these in lecture,
but as he pointed out there, it’s really easy
to make a mistake, so you can never do enough
examples of finding the SVD. So, what does the SVD look like? What do we want to end up with? Well, we want a decomposition
C equals U sigma V transpose. U and V are going to be
orthogonal matrices, that is, their columns
are orthonormal sets. Sigma is going to
be a diagonal matrix with non-negative entries. OK, good. So now, how do we find
this decomposition? Well, we need two equations, OK? One is C transpose C is equal
to V, sigma transpose, sigma, V transpose. And you get this just by
plugging in C transpose C here and noticing that U
transpose U is 1, since U is an orthogonal matrix. Okay. And the second equation is just
noticing that V transpose is V inverse, and moving it to the
other side of the equation, which is C*V equals U*sigma. OK, so these are
the two equations we need to use to find
V, sigma, and U. OK, so let’s start
with the first one. Let’s compute C transpose
C. So C transpose C is that– Well, if
you compute, we’ll get a 26, an 18, an
18, and a 74, great. Now, what you notice
diagonalization of C transpose C. So we need to find
the eigenvalues– those will be the entries
of sigma transpose sigma– and the
eigenvectors which will be the columns of a V. Okay, good. So how do we find those? Well, we look at the determinant
of C transpose C minus lambda times the identity,
which is the determinant of 26 minus lambda, 18, 18,
and 74– 74 minus lambda, thank you. Good, OK, and what
is that polynomial? Well, we get a lambda squared,
now the 26 plus 74 is 100, so minus 100*lambda. And I’ll let you do 26 times 74
minus 18 squared on your own, but you’ll see you get 1,600,
and this easily factors as lambda minus 20
times lambda minus 80. So the eigenvalues
are 20 and 80. Now what are the eigenvectors? Well, you take C transpose C
minus 20 times the identity, and you get 6, 18, 18 and 54. To find the eigenvector
with eigenvalue 20, we need to find a vector in
the null space of this matrix. Note that the second
column is three times the first column, so our
first vector, v_1, we can just take that to be, well, we
could take it to be [-3, 1], but we’d like it to be
a unit vector, right? Remember the columns of
v should be unit vectors because they’re orthonormal. So 3 squared plus
1 squared is 10, we have to divide by
the square root of 10. OK, similarly, we do C transpose
C minus 80 times the identity, we’ll get -54, 18; 18,
and -6, and similarly we can find that v_2 will
be 1 over square root of 10, 3 over the square root of 10. Great, OK, so what
information do we have now? we have our V matrix, which
is just made up of these two columns, and we actually
have our sigma matrix too, because the squares of the
diagonal entries of sigma are 20 and 80. Good, so let’s write those
down, write down what we have. So we have V– I just
add these vectors and make them the
columns of my matrix. Square root of 10, 1
over square root of 10; 1 over square root of 10,
3 over square root of 10. And sigma, this is just the
square roots of 20 and 80, which is just 2 root 5 and
4 root 5 along the diagonal. Squeezing it in here, I hope
you all can see these two. Good, so these are two of the
three parts of my singular value decomposition. The last thing I
need to find is u, and for that I need to use this
second equation right here. So you need to multiply
C times V, okay so So c is [5, 5; -1, 7],
let’s multiply it by V, 1 over root 10, 3 over
square root of 10. What do we get? Well, I’ll let you
work out the details, but it’s not hard here. You get -10 over root 10, which
is just negative square root of 10 here. Then I just get 2 square
root of 10, and then I get– 1 is 2 square root of 10 and– I think I made an error here. Give me a second to look
through my computation again. AUDIENCE: [INAUDIBLE] PROFESSOR: The (2, 1) entry
should be– oh, yes, thank you. The (2, 1) entry should
be the square root of 10. Good, yes, that’s what I was
hoping, yes, because we get– Yes, I did it in
the wrong order, right, so your recitation
instructor should know how to multiply matrices,
great, yes, thank you. You multiply this first, then
this, then this, and then this, and if you do it correctly
you will get this matrix here. Good, great. So now I’d like this
to be my U matrix, but it’s actually U times sigma,
so I need to make these entries unit length. OK, so I get -1 over root 2, 1
over root 2, 1 over root 2, 1 over root 2, times
my sigma matrix here, which is, remember,
2 square root of 5, 4 square root of 5,
and these constants are just what I needed to
divide these columns by in order to make them unit vectors. So now, here’s my U matrix,
1 over square root of 2, -1 over square root of 2;
1 over square root of 2, 1 over square root of 2, good. So now I have all three
matrices, U, V, and sigma and despite some little
errors here and there, I think this is actually right. You should go check it
yourself, because if you’re at all like me, you’ve screwed
up several times by now. But anyway, this is
a good illustration of how to find the singular
value decomposition. Recall that you’re
looking for U sigma V transpose where u and v
are orthogonal matrices, and sigma is diagonal
with non-negative entries. And you find it using
these two equations, you compute C transpose C,
that’s V sigma transpose sigma times V transpose, and you
also have C*V is U*sigma. I hope this was a

• ### Luis Lobo

To find U it is simple. Did the professor found U*Sigma from CV, didn't he?
So, to find U we do. U*Sigma = CV.
U = [a11 a12; a21 a22]*[2*sqrt(5) 0; 0 4*sqrt(5)] = [-sqrt(10) 2*sqrt(10); sqrt(10) 2*sqrt(10)].
So, we have a11=-sqrt(10)/2*sqrt(5) = -1/sqrt(2)
a12 = 2*sqrt(10)/4*sqrt(5)=1/sqrt(2)
a21 = sqrt(10)/2*sqrt(5)=1/sqrt(2)
a22 = 2*sqrt(10)/4*sqrt(5)=1/sqrt(2)

• ### Rafael Lima

Just an advice: next time try to compute without jumping steps. When you compute step-by-step calmly, your chances of success increases a lot. Anyway, thank you for the lecture anyway. It was very helpful.

• ### Pembinaan Aman Gemilang Sdn Bhd

i thought you will enlighten SVD …
BUT
you just made SVD even more confusing!!!

• ### Grigor Nazaryan

No, this was somehow not good illustration.

• ### Ulf Aslak

In which cases will you arrive at complex U?

• ### 施宇謙

In the end of ans of u, did he make the mistake of the sign between u11 and u12?

• ### Manusia Ganteng

This was not clear

Thanks buddy

• ### skilstopaybils

This is the closest to understanding the process of SVD to date! So thank you for that, but it would be great if you didn't use the convenient example for the V matrix. Anybody know of a video that uses algebra?

Also, the identity matrix is Sigma*Sigma.T right?

• ### Peter Cheung

How can i apply this to programming?

• ### Paul McGee

Nice job, dude. 🙂

• ### Johnny

Could someone explain why he divides by sqrt(10) in 5:33?

• ### Kunal Rathod

thanx a ton !

• ### Clifford Wu

Great tutorial

• ### GoJMe

How does he get U????? can some1 show me how the fug he did, this make it jsut more complicated…

• ### GoJMe

instead of showing new ways of solving the SVD can some1 explain the basic way, the idiotic proof way

• ### plavix221

But why doesn't cancel V transposed V also out in the beginning? V trans is also orthogonal, or not?

• ### Taffazzel Hossain

Easy to understand…

not clear

• ### Jacob Fields

You explained singular value decomposition better in 11 minutes than my linear algebra professor did in 50. Thanks.

• ### soundman1992

Great tutorial but in the Sigma matrix the values are the wrong way round. Max singular value should be top left. This leads to incorrect U matrix I think

• ### PaMS1995

This is the best svd tutorial I could wish for, thanks for making it easy

• ### Alan Du

You should go over the case when some of the eigen values are 0.

• ### Debby A.

Thanks for the video. Kinda confused on the signs of the final answer of U tho'. Did we need to change the signs of a11 and a21?

• ### Anjan Basumatary

formula to find v1 and v2???

• ### Jhabriel Varela

The correct values according to MATLAB are:

u = [0.7071 0.7071
0.7071 -0.7071]

s = [8.9443 0
0 4.4721]

v = [0.3162 0.9487
0.9487 -0.3162]

Regards!

• ### hedgehogs_ftw

5 min of this video taught me more than two lectures, two chapters of my book, one office hour with the TA, and 4 hours trying to figure it out on my own.

• ### hedgehogs_ftw

Does it matter if I use C'C or CC' at the beginning when calculating the determinant?

• ### SpikeyBryan

fricken NIRD!!!!

• ### Anuraag

This is a bit confusing. If anyone wants to know how to find U, have a look at my workings. (S=sigma)

Firstly, when calculating the eigenvector for the eigenvalue 20, swap the signs around so that the vector is (3/root10, -1/root10).
Note that this also a correct eigenvector for the given matrix and its eigenvalue 20. You can find out why by reading up on eigendecomposition.

Secondly, swap the order of the columns around in S, so that the values go from high to low when looking from left to right. This is the conventional format of the sigma matrix.

Now when finding U, I'm not sure why he's done the unit length thing, and I can't even see how he's got his final answer from it.

Anyway, we know that CV = US, which means CVS^-1 = USS^-1.
Since S is diagonal, SS^-1 = I, the identity matrix i.e. it can be ignored.
So now we have CVS^-1 = U.

To find S^-1: Since we have a diagonal matrix, just invert the values along the diagonal i.e. any value on the diagonal, x, becomes 1/x.

Now multiply your CV by your S^-1 and you should get the same result for U as in the video, but with the columns swapped around i.e. in the correct format.

• ### Vishnu S

why he swapped the negative sign in the last section of finding u(in column 1)?
-(1/2) change to +(1/2) and +(1/2) change to -(1/2) in column 1?
got confused in the last section?
plz help
thankyou

9:49

• ### Ardian Umam

I'm always amazed by MIT OCW videos. The way they teach is just ideal. Clear/big enough writen on the board, systematic explanation and comfortable to understand.

• ### 김서현

I'm sincerely thanks for MIT to give me this wonderful lecture for free .
It's really helpful for me to learn SVD.

• ### samuel salomon

thank you bro

• ### Qlzldxx Rkddsc

EXCELENT!!

How to make matrix entries unit length? 9:58

• ### ferox7878

funny that you say eigen(vector) and eigen(value) in english, just like in german

• ### Wali Kamal

are the values of the Sigma matrix determined by the eigenvalues of C*C-transpose or C-transpose * C??

• ### Jeetendra Ahuja

Not for someone who has no idea about it, it's like you are practicing what you know already.. Not worth watching

• ### Sanjukta Dawn

Hi , This video was very helpful. However I think the eigenvectors for the corresponding eigenvalues have been interchanged somehow. That is the eigen vector for lambda = 20 is the one which had been shown against lambda = 80

• ### 곽성실

u look so young!! wow

• ### royxss

sigma matrix values should have been switched such that a11 >= a22. That's what svd says. Correct me if I am wrong

• ### Petar Ivanov

When finding the eigenvectors for a given eigenvalue, why is it necessary that we find a single element of the nullspace, instead of the whole nullspace?

• ### Марчо Камара

What is the importance of SVD?

• ### The Desi Engineer

MIT OCW is the big reason due to which I pass my courses like Linear Algebra in my university …
THANKS MIT_OCW

• ### Kubilay Yazoğlu

U matrix signs are wrong. Everyone compute yourself.

• ### H Hafizhan

why we can just put square root of 20 and 80 for the sigma matrix? I mean, shouldn't it be just 20 and 80?

• ### Faraz Mazhar

Are you a wizard?

I think he mistaken when writing the final V. It should be write down as the eigen vector of the largest eigen value is column one, the eigen vector of the next largest eigen value is column two, and so forth and so on until we have the eigen vector of the smallest eigen value as the last column of our matrix. Right?

• ### Benny B.

i don't get where the matrix he writes down at 10:00 comes from, can somebody help?
The matrix with (1/sqrt(2))

• ### 1103 Musik Berlin

this style is nice, i know you enjoy my profile https://www.youtube.com/watch?v=mGxIkzDhU2Y

• ### Uthman Zubair

What if C iss not a square matric, you need to show that you have to find c^Tc and cc^T, then find there coresponding eigen vectors

• ### Jackson McKenzie

I thought that for the sigma matrix, the eigenvalues were listed in descending order, so it should be sqrt(80) then sqrt(20). Is this true or does it matter?

• ### Yan Gong

Gotta save my life for the rest of the quarter!!! So lucky to find this tutorial right before the midterm tomorrow LOL

Last minute Mistake: He put a wrong sign in u11 and u21 position.
Correction: u11 = -1/root(2), u21= 1/root(2)

Correct me If I am going somewhere wrong.

• ### Jonathan Sum

Look at we, we are the failures. Look at this kid, he is the successor. They have the money. They have the better education! Our professors in the community college are just a joke.

• ### Ramon Massoni

You're the real MVP man

• ### Melih Han

What if one of the lambda value is equal to 0. How can i solve the question?

• ### Sanjay Krish

you are a born teacher. justified because you are a student of Mr.Gilbert

• ### Miten Mehta

how did you decide you need to do determinent in there ?

• ### Tomas Kovarik

its funny how this guy explains simple stuff but then doesn't bother to explain at all, how he got the sigma matrix. Because the result was for sigma transposed time sigma. From here we get that the squares of the "singular values" equal the eigenvalues. That's how you get them. I am suspicious if this little guy knows what he is doing….then after all that he obviously doesn't know how to multiply two matrices. What a shame

• ### Eliot McLellan

Thanks, noob!lol

• ### Joujau9

Can anyone explain me precisely, what is happening @10:00, how does he get this first matrix? What is he diving it with? I'm a bit confused.

• ### Aisha Anwar Malik

How are the values of E filled in? 7:00

• ### Marwan Sabri

Thank you very much

mini gilbert

• ### Abdul Fatah Rajri

Excellent effort of this young man ! But I have a little confusion that if the evectors V1 and V2 do not happen to be orthonormals to each other then what should be there because we always require our V to be orthogonal
?

• ### Trang Nguyen

Perhaps there's a faster way. let's note C' the transpose of C, S for sigma matrix
By using two equations : CC' = V S'S V'
C'C = US'SU'
You can compute S'S by finding eigenvalues of CC' but it happens to be the same eigen values of C'C. So after finding V, instead of using your second equation. You can just find the eigen vectors of C'C by doing :
C'C – 20I
C'C – 80I
You'll find the vectors of U faster and without inversing S.
Also by factorizing 1/sqrt(10), it's easier to compute

• ### Locke Jet

Thank you for your another method to get U.

• ### Free

7:24 No need to look for me man, I'm right here.

• ### Shubhi Sinha

Chutiya hai tu saala budbak

Awesome 🙂

• ### Mehmet Kazakli

Singular values need to be ordered decreasingly. When you write sigma, should not 1st and 4th values switched ?

• ### Smruti Charkha

It was a really good illustration.

• ### Andrew Hutchings

Explained a complicated problem in a simple way. Amazing work.

• ### gloriousholy

Eigenvector signs are wrong, that's what's caused the confusion. The eigenvectors should be this: [[-3/sqrt(10), -1/sqrt(10)],[1/sqrt(10), -3/sqrt(10)]].

• ### 冯毅强

nice video, thanks

• ### Roberto Augusto Gómez Loenzo

Excellent video. Quite clear.

• ### Lemaure44

SVD? its a russian sniper idiots!

Thank you, It was very helpful.

• ### Tttt Y

for rectangular matrix, is sigma still symmetric?

• ### Al En

Do you habe more explaination videos ?

• ### ZeroDay Fracture

But, what do the numbers mean mason???

• ### Leandro Souza

God, I love MIT.

• ### Andrei Platonov

I would be scared af with those numbers

I think he skipped a step where product CV is multiplied by inverse of sigma matrix

• ### ved prakash dubey

In final step there is a mistake in first column of u reverses sign of element.

• ### PRIYANSHU VERMA

At 5:08, can you explain how did you calculated the value of V1?

• ### JasonT760

Thank you, most examples I found for this were simple examples, this helped me figure out the more complex problems.

• ### Rivaldi Julviar

Great way of teching!! you've teach me SVD in 11 minutes. a few error on the end but that's understandable

• ### star hopper

HOMIE U A REAL G

Great video!

• ### Ajay Sharma

Nice both video and the way you up the eyebrow in last second