29-Jul-2022 15:03:59 gradient_descent_test(): MATLAB/Octave version 9.8.0.1380330 (R2020a) Update 2 gradient_descent() uses derivative information to iteratively estimate the minimizer of a function. gradient_descent_linear_test(): gradient_descent_linear() approximates the solution of a least squares problem, to find a solution x to A*x=b, by minimizing ||Ax-b|| using gradient descent. Learning rate = 0.02 Stepsize tolerance = 1e-06 Maximum iterations = 10000 Number of iterations = 10000 Estimated solution: 61.2703 -39.0588 Exact solution: 61.2720 -39.0620 Error = 0.0035884 Residual for estimated solution = 2.73689 Residual for exact solution = 2.73689 gradient_descent_nonlinear_test(): Seek local minimizer of a scalar function quartic(x) Minimizer is probably in the interval [-2,2] Use a very simple version of the gradient descent method. Graphics saved in "quartic.png" it x f(x) f'(x) 0 -1.5 19.625 -14 1 -0.12625 19.810502 1.9939015 2 -0.32375095 19.278963 3.3185368 3 -0.64364068 18.042511 4.0159801 4 -1.0059325 16.994351 0.90423354 5 -1.0827668 16.976666 -0.49321323 6 -1.0409012 16.973035 0.30488295 7 -1.0667752 16.971318 -0.17779907 8 -1.0516878 16.970804 0.10777215 9 -1.0608327 16.970601 -0.063938188 10 -1.0554073 16.970532 0.038442935 11 -1.0586693 16.970507 -0.022934387 12 -1.0567233 16.970498 0.013747163 13 -1.0578897 16.970495 -0.0082171272 14 -1.0571925 16.970493 0.0049199442 15 -1.05761 16.970493 -0.0029428151 16 -1.0573603 16.970493 0.0017612783 17 -1.0575097 16.970493 -0.0010537468 18 -1.0574203 16.970493 0.00063057739 19 -1.0574738 16.970493 -0.0003772979 20 -1.0574418 16.970493 0.00022576883 21 -1.0574609 16.970493 -0.00013509008 22 -1.0574495 16.970493 8.0834172e-05 23 -1.0574563 16.970493 -4.8368132e-05 24 -1.0574522 16.970493 2.894196e-05 25 -1.0574547 16.970493 -1.7317851e-05 26 -1.0574532 16.970493 1.036243e-05 27 -1.0574541 16.970493 -6.2005221e-06 28 -1.0574536 16.970493 3.7101843e-06 29 -1.0574539 16.970493 -2.2200481e-06 30 -1.0574537 16.970493 1.3284018e-06 31 -1.0574538 16.970493 -7.9487062e-07 32 -1.0574537 16.970493 4.756237e-07 33 -1.0574538 16.970493 -2.8459711e-07 34 -1.0574538 16.970493 1.7029329e-07 35 -1.0574538 16.970493 -1.0189773e-07 36 -1.0574538 16.970493 6.0972157e-08 37 -1.0574538 16.970493 -3.6483675e-08 38 -1.0574538 16.970493 2.1830596e-08 39 -1.0574538 16.970493 -1.3062689e-08 40 -1.0574538 16.970493 7.8162721e-09 Initial x = -1.5, f(x) = 19.625, f'(x) = -14 Final x = -1.05745, f(x) = 16.9705, f'(x) = 7.81627e-09 Graphics saved in "quartic_minimizer.png" gradient_descent_vector_x_test(): Seek minimizer of a function z(x,y). Initial x,y = (1,1.5), f(x,y) = 4.86667, f'(x,y) = (2.3,4) Final x,y = (0.00634006,-0.0148111), f(x,y) = 0.000205856, f'(x,y) = (0.0105481,-0.0232821) gradient_descent_vector_f_test(): Seek minimizer of vector function f(x). it ||x|| ||f(x)|| ||J(x)|| 0 0 58.456136 20.322401 1 0.2095833 23.306394 20.230019 2 0.33545194 10.616628 20.224202 3 0.41113662 6.013408 20.227599 4 0.45674602 4.3216981 20.230302 5 0.48434855 3.6785445 20.234899 6 0.50118453 3.4131958 20.240197 7 0.51159719 3.2841799 20.246503 8 0.51819108 3.2045617 20.253654 9 0.52252743 3.1430494 20.261637 10 0.52554149 3.0883824 20.270402 11 0.52779298 3.0365108 20.279918 12 0.52961638 2.9859799 20.290152 13 0.53121112 2.936268 20.301077 14 0.53269566 2.8871887 20.312669 15 0.53413996 2.8386757 20.324907 16 0.53558491 2.7907056 20.337776 17 0.53705399 2.74327 20.35126 18 0.53856032 2.696366 20.365347 19 0.54011078 2.6499923 20.380029 20 0.54170854 2.6041484 20.3953 21 0.54335461 2.5588339 20.411154 22 0.5450487 2.5140484 20.427591 23 0.54678974 2.4697918 20.44461 24 0.54857627 2.4260636 20.462213 25 0.55040657 2.3828637 20.480406 26 0.55227882 2.3401916 20.499193 27 0.55419113 2.2980471 20.518585 28 0.55614163 2.2564298 20.53859 29 0.55812845 2.2153394 20.559223 30 0.56014974 2.1747755 20.580496 31 0.56220372 2.1347376 20.602428 32 0.56428864 2.0952253 20.625036 33 0.56640278 2.056238 20.648341 34 0.5685445 2.0177751 20.672368 35 0.57071219 1.979836 20.697141 36 0.5729043 1.9424199 20.722688 37 0.57511931 1.9055259 20.74904 38 0.57735578 1.8691533 20.77623 39 0.57961227 1.833301 20.804295 40 0.58188743 1.797968 20.833272 41 0.58417995 1.7631532 20.863205 42 0.58648853 1.7288553 20.894139 43 0.58881195 1.6950729 20.926122 44 0.59114902 1.6618048 20.959207 45 0.59349859 1.6290494 20.993451 46 0.59585954 1.596805 21.028914 47 0.59823082 1.5650701 21.065661 48 0.60061139 1.5338428 21.103763 49 0.60300026 1.5031213 21.143293 50 0.60539646 1.4729036 21.184334 51 0.60779908 1.4431876 21.226969 52 0.61020724 1.4139712 21.271292 53 0.61262007 1.3852522 21.317401 54 0.61503675 1.3570281 21.365402 55 0.6174565 1.3292966 21.415406 56 0.61987855 1.3020551 21.467535 57 0.62230218 1.2753011 21.521918 58 0.62472668 1.2490317 21.578691 59 0.62715139 1.2232443 21.638002 60 0.62957564 1.1979358 21.700007 61 0.63199884 1.1731034 21.764872 62 0.63442037 1.1487438 21.832774 63 0.63683967 1.124854 21.903902 64 0.63925619 1.1014305 21.978456 65 0.64166941 1.07847 22.056646 66 0.64407884 1.0559691 22.138696 67 0.64648398 1.033924 22.224842 68 0.64888438 1.012331 22.315331 69 0.65127962 0.99118621 22.410425 70 0.65366925 0.9704857 22.510394 71 0.6560529 0.95022529 22.615522 72 0.65843017 0.93040069 22.726102 73 0.66080071 0.91100744 22.842437 74 0.66316417 0.89204094 22.964836 75 0.66552022 0.87349638 23.093615 76 0.66786855 0.85536877 23.229094 77 0.67020886 0.8376529 23.371591 78 0.67254087 0.82034337 23.521422 79 0.67486431 0.80343451 23.678894 80 0.67717893 0.78692045 23.844305 81 0.67948449 0.77079502 24.017931 82 0.68178076 0.75505182 24.20003 83 0.68406753 0.73968416 24.390828 Initial x = (0,0,0), ||f(x)|| = 58.4561, ||J(x)|| = (20.3224) Final x = (0.439901,0.0121326,-0.523726), ||f(x)|| = 0.739684, ||J(x)|| = (24.3908) gradient_descent_test(): Normal end of execution. 29-Jul-2022 15:04:03