There appear to be some mistakes in your attachment. Assuming your question is:
minimize [tex]J = (0.18-b)^2+(0.33-b(1-a))^2+(0.45-b(1-a+a^2))^2[/tex]
I get the answer
[tex]a=-0.819145705784351526557732...[/tex]
[tex]b= 0.180862291009977588342398...[/tex]
By differentiating [tex]J[/tex] with respect to [tex]a[/tex] you get
[tex]2(0.33-b(1-a))b+2(0.45-b(1-a+a^2))b(1-2a)[/tex]
which should be [tex]0[/tex] at the minimum.
Rearranging gives:
[tex]2b(0.78-0.9a-b(2-4a+3a^2-2a^3)) = 0[/tex]
So either [tex]b=0[/tex] or
[tex]b = (0.78-0.9a)/(2-4a+3a^2-2a^3)[/tex]
(It is easy to check at the end that [tex]b=0[/tex] doesn't give the global minimum.)
Similarly by considering [tex]dJ/db=0[/tex] we get
[tex]b=(0.96-0.78a+0.45a^2)/(3-4a+4a^2-2a^3+a^4)[/tex].
Which means
[tex](0.78-0.9a)/(2-4a+3a^2-2a^3) = (0.96-0.78a+0.45a^2)/(3-4a+4a^2-2a^3+a^4)[/tex]
Which rearranges and simplifies to
[tex]11a^4-30a^3+6a^2+14a-14 = 0[/tex]
Quartic equations can be solved exactly
https://en.wikipedia.org/wiki/Quartic_f ... _for_rootshowever the exact solution is very complicated. The numerical answer you get is
[tex]a=-0.819145705784351526557732...[/tex] which means [tex]b= 0.180862291009977588342398...[/tex]
(or [tex]a=2.3654...[/tex] which you can easily check does not give the global minimum).
Hope this helped,
R. Baber.