1### ensmallen 2.17.0: "Pachis Din Me Pesa Double"
2###### 2021-07-06
3 * CheckArbitraryFunctionTypeAPI extended for MOO support
4   ([#283](https://github.com/mlpack/ensmallen/pull/283)).
5
6 * Refactor NSGA2
7   ([#263](https://github.com/mlpack/ensmallen/pull/263),
8   [#304](https://github.com/mlpack/ensmallen/pull/304)).
9
10 * Add Indicators for Multiobjective optimizers
11   ([#285](https://github.com/mlpack/ensmallen/pull/285)).
12
13 * Make Callback flexible for MultiObjective Optimizers
14   ([#289](https://github.com/mlpack/ensmallen/pull/289)).
15
16 * Add ZDT Test Suite
17   ([#273](https://github.com/mlpack/ensmallen/pull/273)).
18
19 * Add MOEA-D/DE Optimizer
20   ([#269](https://github.com/mlpack/ensmallen/pull/269)).
21
22 * Introduce Policy Methods for MOEA/D-DE
23   ([#293](https://github.com/mlpack/ensmallen/pull/293)).
24
25 * Add Das-Dennis weight initialization method
26   ([#295](https://github.com/mlpack/ensmallen/pull/295)).
27
28 * Add Dirichlet Weight Initialization
29   ([#296](https://github.com/mlpack/ensmallen/pull/296)).
30
31 * Improved installation and compilation instructions
32   ([#300](https://github.com/mlpack/ensmallen/pull/300)).
33
34 * Disable building the tests by default for faster installation
35   ([#303](https://github.com/mlpack/ensmallen/pull/303)).
36
37 * Modify matrix initialisation to take into account
38   default element zeroing in Armadillo 10.5
39   ([#305](https://github.com/mlpack/ensmallen/pull/305)).
40
41### ensmallen 2.16.2: "Severely Dented Can Of Polyurethane"
42###### 2021-03-24
43 * Fix CNE test trials
44   ([#267](https://github.com/mlpack/ensmallen/pull/267)).
45
46 * Update Catch2 to 2.13.4
47   ([#268](https://github.com/mlpack/ensmallen/pull/268)).
48
49 * Fix typos in documentation
50   ([#270](https://github.com/mlpack/ensmallen/pull/270),
51    [#271](https://github.com/mlpack/ensmallen/pull/271)).
52
53 * Add clarifying comments in problems/ implementations
54   ([#276](https://github.com/mlpack/ensmallen/pull/276)).
55
56### ensmallen 2.16.1: "Severely Dented Can Of Polyurethane"
57###### 2021-03-02
58 * Fix test compilation issue when `ENS_USE_OPENMP` is set
59   ([#255](https://github.com/mlpack/ensmallen/pull/255)).
60
61 * Fix CNE initial population generation to use normal distribution
62   ([#258](https://github.com/mlpack/ensmallen/pull/258)).
63
64 * Fix compilation warnings
65   ([#259](https://github.com/mlpack/ensmallen/pull/259)).
66
67 * Remove `AdamSchafferFunctionN2Test` test from Adam test suite to prevent
68   spurious issue on some aarch64 ([#265](https://github.com/mlpack/ensmallen/pull/259)).
69
70### ensmallen 2.16.0: "Severely Dented Can Of Polyurethane"
71###### 2021-02-11
72 * Expand README with example installation and add simple example program
73   showing usage of the L-BFGS optimizer
74   ([#248](https://github.com/mlpack/ensmallen/pull/248)).
75
76 * Refactor tests to increase stability and reduce random errors
77   ([#249](https://github.com/mlpack/ensmallen/pull/249)).
78
79### ensmallen 2.15.1: "Why Can't I Manage To Grow Any Plants?"
80###### 2020-11-05
81 * Fix include order to ensure traits is loaded before reports
82   ([#239](https://github.com/mlpack/ensmallen/pull/239)).
83
84### ensmallen 2.15.0: "Why Can't I Manage To Grow Any Plants?"
85###### 2020-11-01
86 * Make a few tests more robust
87   ([#228](https://github.com/mlpack/ensmallen/pull/228)).
88
89 * Add release date to version information. ([#226](https://github.com/mlpack/ensmallen/pull/226))
90
91 * Fix typo in release script
92   ([#236](https://github.com/mlpack/ensmallen/pull/236)).
93
94### ensmallen 2.14.2: "No Direction Home"
95###### 2020-08-31
96 * Fix implementation of fonesca fleming problem function f1 and f2
97   type usage and negative signs. ([#223](https://github.com/mlpack/ensmallen/pull/223))
98
99### ensmallen 2.14.1: "No Direction Home"
100###### 2020-08-19
101 * Fix release script (remove hardcoded information, trim leading whitespaces
102   introduced by `wc -l` in MacOS)
103    ([#216](https://github.com/mlpack/ensmallen/pull/216),
104     [#220](https://github.com/mlpack/ensmallen/pull/220)).
105
106 * Adjust tolerance for AugLagrangian convergence based on element type
107   ([#217](https://github.com/mlpack/ensmallen/pull/217)).
108
109### ensmallen 2.14.0: "No Direction Home"
110###### 2020-08-10
111 * Add NSGA2 optimizer for multi-objective functions
112    ([#149](https://github.com/mlpack/ensmallen/pull/149)).
113
114 * Update automatic website update release script
115   ([#207](https://github.com/mlpack/ensmallen/pull/207)).
116
117 * Clarify and fix documentation for constrained optimizers
118   ([#201](https://github.com/mlpack/ensmallen/pull/201)).
119
120 * Fix L-BFGS convergence when starting from a minimum
121   ([#201](https://github.com/mlpack/ensmallen/pull/201)).
122
123* Add optimizer summary report callback
124   ([#213](https://github.com/mlpack/ensmallen/pull/213)).
125
126### ensmallen 2.13.0: "Automatically Automated Automation"
127###### 2020-07-15
128 * Fix CMake package export
129    ([#198](https://github.com/mlpack/ensmallen/pull/198)).
130
131 * Allow early stop callback to accept a lambda function
132   ([#165](https://github.com/mlpack/ensmallen/pull/165)).
133
134### ensmallen 2.12.1: "Stir Crazy"
135###### 2020-04-20
136 * Fix total number of epochs and time estimation for ProgressBar callback
137    ([#181](https://github.com/mlpack/ensmallen/pull/181)).
138
139 * Handle SpSubview_col and SpSubview_row in Armadillo 9.870
140    ([#194](https://github.com/mlpack/ensmallen/pull/194)).
141
142 * Minor documentation fixes
143    ([#197](https://github.com/mlpack/ensmallen/pull/197)).
144
145### ensmallen 2.12.0: "Stir Crazy"
146###### 2020-03-28
147 * Correction in the formulation of sigma in CMA-ES
148    ([#183](https://github.com/mlpack/ensmallen/pull/183)).
149
150 * Remove deprecated methods from PrimalDualSolver implementation
151    ([#185](https://github.com/mlpack/ensmallen/pull/185).
152
153 * Update logo ([#186](https://github.com/mlpack/ensmallen/pull/186)).
154
155### ensmallen 2.11.5: "The Poster Session Is Full"
156###### 2020-03-11
157  * Change "mathematical optimization" term to "numerical optimization" in the
158    documentation ([#177](https://github.com/mlpack/ensmallen/pull/177)).
159
160### ensmallen 2.11.4: "The Poster Session Is Full"
161###### 2020-03-03
162  * Require new HISTORY.md entry for each PR.
163    ([#171](https://github.com/mlpack/ensmallen/pull/171),
164     [#172](https://github.com/mlpack/ensmallen/pull/172),
165     [#175](https://github.com/mlpack/ensmallen/pull/175)).
166
167  * Update/fix example documentation
168    ([#174](https://github.com/mlpack/ensmallen/pull/174)).
169
170### ensmallen 2.11.3: "The Poster Session Is Full"
171###### 2020-02-19
172  * Prevent spurious compiler warnings
173    ([#161](https://github.com/mlpack/ensmallen/pull/161)).
174
175  * Fix minor memory leaks
176    ([#167](https://github.com/mlpack/ensmallen/pull/167)).
177
178  * Revamp CMake configuration
179    ([#152](https://github.com/mlpack/ensmallen/pull/152)).
180
181### ensmallen 2.11.2: "The Poster Session Is Full"
182###### 2020-01-16
183  * Allow callback instantiation for SGD based optimizer
184    ([#138](https://github.com/mlpack/ensmallen/pull/155)).
185
186  * Minor test stability fixes on i386
187    ([#156](https://github.com/mlpack/ensmallen/pull/156)).
188
189  * Fix Lookahead MaxIterations() check.
190    ([#159](https://github.com/mlpack/ensmallen/pull/159)).
191
192### ensmallen 2.11.1: "The Poster Session Is Full"
193###### 2019-12-28
194  * Fix Lookahead Synchronization period type
195    ([#153](https://github.com/mlpack/ensmallen/pull/153)).
196
197### ensmallen 2.11.0: "The Poster Session Is Full"
198###### 2019-12-24
199  * Add Lookahead
200    ([#138](https://github.com/mlpack/ensmallen/pull/138)).
201
202  * Add AdaBound and AMSBound
203    ([#137](https://github.com/mlpack/ensmallen/pull/137)).
204
205### ensmallen 2.10.5: "Fried Chicken"
206###### 2019-12-13
207  * SGD callback test 32-bit safety (big number)
208    ([#143](https://github.com/mlpack/ensmallen/pull/143)).
209
210  * Use "arbitrary" and "separable" terms in static function type checks
211    ([#145](https://github.com/mlpack/ensmallen/pull/145)).
212
213  * Remove 'using namespace std' from `problems/` files
214    ([#147](https://github.com/mlpack/ensmallen/pull/147)).
215
216### ensmallen 2.10.4: "Fried Chicken"
217###### 2019-11-18
218  * Add optional tests building.
219    ([#141](https://github.com/mlpack/ensmallen/pull/141)).
220
221  * Make code samples collapsible in the documentation.
222    ([#140](https://github.com/mlpack/ensmallen/pull/140)).
223
224### ensmallen 2.10.3: "Fried Chicken"
225###### 2019-09-26
226  * Fix ParallelSGD runtime bug.
227    ([#135](https://github.com/mlpack/ensmallen/pull/135)).
228
229  * Add additional L-BFGS convergence check
230    ([#136](https://github.com/mlpack/ensmallen/pull/136)).
231
232### ensmallen 2.10.2: "Fried Chicken"
233###### 2019-09-11
234  * Add release script to rel/ for maintainers
235    ([#128](https://github.com/mlpack/ensmallen/pull/128)).
236
237  * Fix Armadillo version check
238    ([#133](https://github.com/mlpack/ensmallen/pull/133)).
239
240### ensmallen 2.10.1: "Fried Chicken"
241###### 2019-09-10
242  * Documentation fix for callbacks
243    ([#129](https://github.com/mlpack/ensmallen/pull/129).
244
245  * Compatibility fixes for ensmallen 1.x
246    ([#131](https://github.com/mlpack/ensmallen/pull/131)).
247
248### ensmallen 2.10.0: "Fried Chicken"
249###### 2019-09-07
250  * All `Optimize()` functions now take any matrix type; so, e.g., `arma::fmat`
251    or `arma::sp_mat` can be used for optimization.  See the documentation for
252    more details ([#113](https://github.com/mlpack/ensmallen/pull/113),
253    [#119](https://github.com/mlpack/ensmallen/pull/119)).
254
255  * Introduce callback support.  Callbacks can be appended as the last arguments
256    of an `Optimize()` call, and can perform custom behavior at different points
257    during the optimization.  See the documentation for more details
258    ([#119](https://github.com/mlpack/ensmallen/pull/119)).
259
260  * Slight speedups for `FrankWolfe` optimizer
261    ([#127](https://github.com/mlpack/ensmallen/pull/127)).
262
263### ensmallen 1.16.2: "Loud Alarm Clock"
264###### 2019-08-12
265  * Fix PSO return type bug
266    ([#126](https://github.com/mlpack/ensmallen/pull/126)).
267
268### ensmallen 1.16.1: "Loud Alarm Clock"
269###### 2019-08-11
270  * Update HISTORY.md to use Markdown links to the PR and add release names.
271
272  * Fix PSO return type bug
273    ([#124](https://github.com/mlpack/ensmallen/pull/124)).
274
275### ensmallen 1.16.0: "Loud Alarm Clock"
276###### 2019-08-09
277  * Add option to avoid computing exact objective at the end of the optimization
278    ([#109](https://github.com/mlpack/ensmallen/pull/109)).
279
280  * Fix handling of curvature for BigBatchSGD
281    ([#118](https://github.com/mlpack/ensmallen/pull/118)).
282
283  * Reduce runtime of tests
284    ([#118](https://github.com/mlpack/ensmallen/pull/118)).
285
286  * Introduce local-best particle swarm optimization, `LBestPSO`, for
287    unconstrained optimization problems
288    ([#86](https://github.com/mlpack/ensmallen/pull/86)).
289
290### ensmallen 1.15.1: "Wrong Side Of The Road"
291###### 2019-05-22
292  * Fix `-Wreorder` in `qhadam` warning
293    ([#115](https://github.com/mlpack/ensmallen/pull/115)).
294
295  * Fix `-Wunused-private-field` warning in `spsa`
296    ([#115](https://github.com/mlpack/ensmallen/pull/115)).
297
298  * Add more warning output for gcc/clang
299    ([#116](https://github.com/mlpack/ensmallen/pull/116)).
300
301### ensmallen 1.15.0: "Wrong Side Of The Road"
302###### 2019-05-14
303  * Added QHAdam and QHSGD optimizers
304    ([#81](https://github.com/mlpack/ensmallen/pull/81)).
305
306### ensmallen 1.14.4: "Difficult Crimp"
307###### 2019-05-12
308  * Fixes for BigBatchSGD ([#91](https://github.com/mlpack/ensmallen/pull/91)).
309
310### ensmallen 1.14.3: "Difficult Crimp"
311###### 2019-05-06
312  * Handle `eig_sym()` failures correctly
313    ([#100](https://github.com/mlpack/ensmallen/pull/100)).
314
315### ensmallen 1.14.2: "Difficult Crimp"
316###### 2019-03-14
317  * SPSA test tolerance fix
318    ([#97](https://github.com/mlpack/ensmallen/pull/97)).
319
320  * Minor documentation fixes (#95, #98).
321
322  * Fix newlines at end of file
323    ([#92](https://github.com/mlpack/ensmallen/pull/92)).
324
325### ensmallen 1.14.1: "Difficult Crimp"
326###### 2019-03-09
327  * Fixes for SPSA ([#87](https://github.com/mlpack/ensmallen/pull/87)).
328
329  * Optimized CNE and DE ([#90](https://github.com/mlpack/ensmallen/pull/90)).
330    Changed initial population generation in CNE to be a normal distribution
331    about the given starting point, which should accelerate convergence.
332
333### ensmallen 1.14.0: "Difficult Crimp"
334###### 2019-02-20
335  * Add DE optimizer ([#77](https://github.com/mlpack/ensmallen/pull/77)).
336
337  * Fix for Cholesky decomposition in CMAES
338    ([#83](https://github.com/mlpack/ensmallen/pull/83)).
339
340### ensmallen 1.13.2: "Coronavirus Invasion"
341###### 2019-02-18
342 * Minor documentation fixes ([#82](https://github.com/mlpack/ensmallen/pull/82)).
343
344### ensmallen 1.13.1: "Coronavirus Invasion"
345###### 2019-01-24
346 * Fix -Wreorder warning ([#75](https://github.com/mlpack/ensmallen/pull/75)).
347
348### ensmallen 1.13.0: "Coronavirus Invasion"
349###### 2019-01-14
350 * Enhance options for AugLagrangian optimizer
351   ([#66](https://github.com/mlpack/ensmallen/pull/66)).
352
353 * Add SPSA optimizer ([#69](https://github.com/mlpack/ensmallen/pull/69)).
354
355### ensmallen 1.12.2: "New Year's Party"
356###### 2019-01-05
357 * Fix list of contributors.
358
359### ensmallen 1.12.1: "New Year's Party"
360###### 2019-01-03
361 * Make sure all files end with newlines.
362
363### ensmallen 1.12.0: "New Year's Party"
364###### 2018-12-30
365 * Add link to ensmallen PDF to README.md.
366
367 * Minor documentation fixes.  Remove too-verbose documentation from source for
368   each optimizer ([#61](https://github.com/mlpack/ensmallen/pull/61)).
369
370 * Add FTML optimizer ([#48](https://github.com/mlpack/ensmallen/pull/48)).
371
372 * Add SWATS optimizer ([#42](https://github.com/mlpack/ensmallen/pull/42)).
373
374 * Add Padam optimizer ([#46](https://github.com/mlpack/ensmallen/pull/46)).
375
376 * Add Eve optimizer ([#45](https://github.com/mlpack/ensmallen/pull/45)).
377
378 * Add ResetPolicy() to SGD-like optimizers
379   ([#60](https://github.com/mlpack/ensmallen/pull/60)).
380
381### ensmallen 1.11.1: "Jet Lag"
382###### 2018-11-29
383 * Minor documentation fixes.
384
385### ensmallen 1.11.0: "Jet Lag"
386###### 2018-11-28
387 * Add WNGrad optimizer.
388
389 * Fix header name in documentation samples.
390
391### ensmallen 1.10.1: "Corporate Catabolism"
392###### 2018-11-16
393 * Fixes for GridSearch optimizer.
394
395 * Include documentation with release.
396
397### ensmallen 1.10.0: "Corporate Catabolism"
398###### 2018-10-20
399 * Initial release.
400
401 * Includes the ported optimization framework from mlpack
402   (http://www.mlpack.org/).
403