Advanced Math/Given a second order nonhomogeneous difference equation, how did the choice of particular solution was arrived at?
QUESTION: Hi Clyde,
I've attached an excerpt from my lecture notes.
It shows the following:
Given a 2nd order non-homogeneous difference equation, it tells us what form the particular solution will take (second column) and this is solely based on the form of the equations right hand side (1st column).
How do we deduce the forms of the second column? I am not able to discern any pattern to it. I'd like to know the method and theory behind it just to enhance my understanding on this topic.
1st year Mathematics and Economics student. I've covered multivariable calculus and linear algebra. So, if the answer involves really high end maths, then I would appreciate it if you could tell me the general idea behind it, some examples and some material to read.
ANSWER: The key to solving these problems is to recognize how linear differential equations really represent "functionals" or "operators." The idea is that the left hand side of the equation is an operation with input y.
For example, say you are trying to solve:
2y'' + 3y' + y = 0
Well, you are trying to plug in some input (y = some function of x) to the left side, and the output you want is zero.
Well, in this case, y'' and y' and y all have to cancel out. The coefficients are 2, 3, and 1, which are all constants, so somehow you need y'' and y' and y to all be very similar. You know y could not be a polynomial because if so, it is impossible for y'' and y' and y to cancel out. For example, if y=x^5, then:
2y'' + 3y' + y = 2 (20x^3) + 3 (5x^4) + x^5 = x^5 + 15x^4 + 40x^3
That's not a solution because it does not cancel out to zero.
In this case, you know to try "y=e^(kx)" because there is a chance to cancel out. You use this idea to determine that:
2 (r^2 e^(rx)) + 3 (r e^(rx)) + e^(rx) = (2r^2 + 3r + 1)e^(rx)
This is a solution for certain values of r (in this case, r=-1 and r=-1/2).
This will give you the solution y = c1 e^(-t) + c2 e^(-t/2).
Now, all that gives you an idea of how to find the solutions to a homogeneous equation. But once you've solved that, you need to learn how to solve something like:
2y'' + 3y' + y = e^(3x)
2y'' + 3y' + y = cos(3x)
2y'' + 3y' + y = x^2 + 1
Well, now the idea is you want to find something you can plug in to the left side of these differential equations, and the output will be something like the right hand side. This is called the particular solution. And because the coefficients of the DE are constant, it is easy to guess which type of functions y you should plug in.
You can identify the form of the particular solution by looking at whatever is on the right-hand side. So if the right side is e^(3x) then the particular solution must be y=Ae^(3x), where A is some unknown constant multiplier.
If it is cos(3x), then y = Acos(3x) + Bsin(3x), where A and B are constants.
If it is x^2+1, then y = Ax^2 + Bx + C.
Whatever "form" you have, if it is exponential, trigonometric, polynomial on the right side, this tells you exactly the same "form" for the particular solution.
This method is called "undetermined coefficients" because all this tells you is the "form" of the solution. For example, if y=Ax^2+Bx+C, you still have to figure out what A, B, and C are.
To do that, you plug them in:
y = Ax^2 + Bx + C
y' = 2Ax + B
y'' = 2A
2y'' + 3y' + y = 4A + 6Ax + 3B + Ax^2 + Bx + C
To simplify this, remember that this is all a function of x, so group those terms:
3y'' + 2y' + y = Ax^2 + (6A + B)x + (4A + 3B + C)
This is supposed to be equal to x^2+1, which means:
A = 1
6A + B = 0
4A + 3B + C = 1
You can solve this algebraically, and it tells you that your particular solution is:
A = 1
B = -6
C = 15
which means y = x^2-4x+2. You can check it:
y = x^2 - 6x + 15
y' = 2x - 6
y'' = 2
2y'' + 3y' + y
= 2(2) + 3(2x-6) + (x^2-6x+15)
= 4 + 6x - 18 + x^2 - 6x + 15
= x^2 + 1
It does indeed work!
Any other type of function on the right side of the equation means you plug in a different "form" with coefficients like "A" "B" and "C" -- and then you solve for the values of those constants algebraically by matching them with the coefficients you want them to be on the right side of the differential equation.
---------- FOLLOW-UP ----------
QUESTION: Hi Clyde,
Thank you very much for your reply. Stating that linear differential equations are like functionals was very helpful.
I hope you can clarify the following points:
When you started to discuss solving non-homogeneous equations, you mentioned:
"And because the coefficients of the DE are constant, it is easy to guess which type of functions y you should plug in.
You can identify the form of the particular solution by looking at whatever is on the right-hand side. So if the right side is e^(3x) then the particular solution must be y=Ae^(3x), where A is some unknown constant multiplier."
This is not easy for me at all. I don't see the connection between having constant coefficients and finding a solution. Furthermore, why must there a constant multiplier in the solution? I'm sorry if this is a basic question but I've spent days thinking about this and still could not figure this out.
When you discussed solving the homogeneous differential equation, you mentioned:
"In this case, you know to try "y=e^(kx)" because there is a chance to cancel out"
It wasn't immediately clear to me why they have a chance to cancel out. Could you please elaborate further on how you managed to deduce that it has a chance to cancel out?
My question was about difference equation, but you answered in terms of differential equation.
So, in the case of the differential equation, if you know y, then you can know y' and y'' by differentiating y. What about the case of the difference equation?
ANSWER: To follow up each part separately:
1. If you plug in a function like "e^(3x)" then what comes out will also be some multiple of e^(3x), because the derivatives of e^(3x) are all the same. Likewise for a combination of sine and cosine, or for other such things.
This means that if the goal is to get a multiple of e^(3x) to be the output of the functional, the input must also be a multiple of e^(3x). The only difficulty will be determining what the multiple is.
2. Here, the goal is to get zero as the output of the functional. So everything must cancel out.
3. It's the same -- I apologize of course for speaking in terms of the wrong context, but the idea is the same for both. For example, take the following DE:
y'' - 3y' + 2y = e^(-x)
Here, you can solve to get:
y = c1 e^x + c2 e^(2x) + (1/6) e^(-x)
The e^(-x) and e^(-2x) terms represent functions that, when plugged into the functional L[y]=y''+3y'+y will give you L[y]=0. You can check each of those gives output zero. They are determined by plugging in e^(rt) and determining what r must be:
r^2 e^(rt) - 3 r e^(rt) + 2 e^(rt) = 0
e^(rt) ( r^2 - 3r + 2 ) = 0
r^2 - 3r + 2 = 0
r = 1, 2
The other term, (1/6)e^(-x), will give the output e^(-x), which matches the right side of the DE. That is determined by plugging in Ae^x and figuring out what A should be as follows:
A e^(-x) - 3 (-A e^(-x)) + 2 A e^(-x) = e^(-x)
6 A e^(-x) = e^(-x)
6A = 1
A = 1/6
If you were looking at a difference equation instead, the methodology is not different, but the numbers will become different. To solve first for the homogeneous part:
y[x+2] - 3 y[x+1] + 2 y[x] = 0
You make the assumption that the answer is e^(rx) for some value of r. Then:
e^(rx+2r) - 3 e^(rx+r) + 2 e^(rx) = 0
e^(rx) + ( e^(2r) - 3 e^r + 2 ) = 0
(e^r)^2 - 3 e^r + 2 = 0
This is a similar quadratic as above, except that there is e^r instead of r, so:
e^r = 1, 2
r = 0, ln(2).
This gives two solutions, y = e^(0x) = 1 and y = e^(ln(2)x) = 2^x.
The particular solution can be similarly obtained, by letting y = A e^(-x)
y[x+2] - 3 y[x+1] + 2 y[x] = e^(-x)
A e^(-x-2) - 3 A e^(-x-1) + 2 A e^(-x) = e^(-x)
A e^(-x) ( e^(-2) - 3 e^(-1) + 2 ) = e^(-x)
A = 1 / ( e^(-2) - 3 e^(-1) + 2 ) ≈ 0.969...
Although that constant is not as pretty as 1/6, it is correct and comes from the same methodology, just applied to a difference eq. instead of a differential eq. It is the same idea. The final answer here is:
y = c1 + c2 2^x + ( 1 / ( e^(-2) - 3 e^(-1) + 2 ) ) e^(-x)
which you might write more neatly as:
y = c1 + c2 2^x + A e^(-x)
where A = 1 / ( e^(-2) - 3 e^(-1) + 2 ).
I hope this shows you that the methods are the same for differential and difference equations, and clears up the other questions you have presented.
---------- FOLLOW-UP ----------
QUESTION: Hi Clyde,
Thank you so much for your answer! I now have a better understanding of what difference equations are and how to solve them.
I hope you don't mind answering another follow up question.
In point 3 of your answer, I am surprised that the form of the solution to a difference equation is the same as the differential equation version i.e the solution to:
y'' - 3y' + 2y = e^(-x) has the form y=Ae^(-x)
and the solution to:
y(x+2)-3y(x+1)+2y(x)=e^(-x) has the form y=Ae^(-x) too.
I find this surprising because in the case of the differential equation, we get the next value of y(x) by differentiating y(x) and in the case of the difference equation, we get the next value of y(x) by incrementing x, which in this case, is the exponent of y(x). To me, both are different operations yet both lead to the same result. Is this just a coincidence or is there actually a link between differential equations and difference equations?
I am wondering if the method you discussed here is applicable to all kinds of second order difference equations?
For example, I am trying to solve y(t+2)-y(t+1)-12y(t)=0. My answer is y(t)=4^t. I've attached my workings in case you want to refer to it.
But according to the suggested solution I should also get y(t)=(-3)^t as another solution.
I would appreciate it if you could tell me if I had made a mistake or I should have used another method to solve this problem.
To answer the first follow up, simply put yes indeed, differential equations and difference equations are often VERY similar in nature. Although they are not the same (of course), there are remarkable similarities. Both act as functionals, if that's one way you like to think of them, and among the ones that are easiest (linear, constant coefficient, low-order), we can find many similarities and many methods to solve one type will solve the other.
For your second question, you should be on the look out for solving:
y(t+2) - y(t+1) + 12 y(t) = 0
So you assume the answer is e^(rt) and you get:
e^(2r) e^(rt) - e^r e^(rt) + 12 e^(rt) = 0
and like before you can solve this as a quadratic:
(e^r)^2 - e^r + 12 = 0
That gives you two solutions, e^r = 4 and e^r = -3.
Now, this gives you r = ln(4) but also r=ln(-3). Although ln(-3) is not defined, it DOES still show up in the answer. First, there is a reason why -- the reason being that the natural log function is a bit strange, you may learn in other mathematics courses that it CAN be defined for negative numbers, it has multiple values, etc.
For the time being, if you know y = e^(rt), then y = (e^r)^t. You have two values of e^r, the values 4 and -3. So either one will work. You could have started by assuming y = k^t and then you would have just gotten k=4 and -3.
Note that (-3)^t is a weirdly defined function (not surprisingly and not coincidentally the reason it is weird stems from the same oddities as the natural log function). It is mathematically problematic to define a negative number to any power. That is why the answer is "weird." However, it is valid -- or at least, good enough for solving this type of problem. If you assume t is positive, rational, integer, etc. you are totally fine.