1# Training
2Most estimators have the ability to be trained with data. Estimators that require training are called [Learners](learner.md) and implement the `train()` method among others. Training is the process of feeding data to the learner so that it can build an internal representation (or *model*). Supervised learners require a [Labeled](datasets/labeled.md) training set. Unsupervised learners can be trained with either a Labeled or [Unlabeled](datasets/unlabeled.md) dataset but only the information contained within the features are used to build the model. Depending on the size of your dataset and choice of learning algorithm, training can a long time or just a few seconds. We recommend assessing your time (compute) and memory requirements before training large models.
3
4To begin training a learner, pass a training Dataset object to the `train()` method on the learner instance like in the example below.
5
6```php
7$estimator->train($dataset);
8```
9
10We can verify that a learner has been trained by calling the `trained()` method which returns true if the estimator is ready to make predictions.
11
12```php
13var_dump($estimator->trained());
14```
15
16```sh
17bool(true)
18```
19
20## Batch vs Online Learning
21Batch learning is when a learner is trained in full using only one dataset in a single session. Calling the `train()` method on a learner instance is an example of batch learning. In contrast, *online* learning occurs when a learner is trained over multiple sessions with multiple datasets as small as a single sample each. Learners that are capable of being partially trained like this implement the [Online](online.md) interface which includes the `partial()` method for training with a dataset in an online scheme. Subsequent calls to the `partial()` method will continue training where the learner left off. Online learning is especially useful for when you have a dataset that is too large to fit into memory all at once or when your dataset is in the form of a stream.
22
23```php
24$estimator->train($dataset1);
25
26$estimator->partial($dataset2);
27
28$estimator->partial($dataset3);
29```
30
31!!! note
32    After the initial training, the learner will expect subsequent training sets to contain the same number and order of features.
33
34## Monitoring Progress
35Since training is often an iterative process, it is useful to obtain feedback as to how the learner is progressing in real-time. For example, you may want to monitor the training loss to make sure that it isn't increasing instead of decreasing with training. Such early feedback saves you time by allowing you to abort training early if things aren't going well. Learners that implement the [Verbose](verbose.md) interface accept a [PSR-3](https://www.php-fig.org/psr/psr-3/) logger instance that can be used to output training information at each time step (or *epoch*). The library comes built-in with the [Screen](other/loggers/screen.md) logger that does the job for most cases.
36
37```php
38use Rubix\ML\Classifiers\LogisticRegression;
39use Rubix\ML\NeuralNet\Optimizers\Adam;
40use Rubix\ML\Other\Loggers\Screen;
41
42$estimator = new LogisticRegression(128, new Adam(0.01));
43
44$estimator->setLogger(new Screen());
45
46$estimator->train($dataset);
47```
48
49```sh
50[2020-09-04 08:39:04] INFO: Logistic Regression (batch_size: 128, optimizer: Adam (rate: 0.01, momentum_decay: 0.1, norm_decay: 0.001), alpha: 0.0001, epochs: 1000, min_change: 0.0001, window: 5, cost_fn: Cross Entropy) initialized
51[2020-09-04 08:39:04] INFO: Epoch 1 - Cross Entropy: 0.16895133388673
52[2020-09-04 08:39:04] INFO: Epoch 2 - Cross Entropy: 0.16559247705179
53[2020-09-04 08:39:04] INFO: Epoch 3 - Cross Entropy: 0.16294448401323
54[2020-09-04 08:39:04] INFO: Epoch 4 - Cross Entropy: 0.16040135038265
55[2020-09-04 08:39:04] INFO: Epoch 5 - Cross Entropy: 0.15786801071483
56[2020-09-04 08:39:04] INFO: Epoch 6 - Cross Entropy: 0.1553151426337
57[2020-09-04 08:39:04] INFO: Epoch 7 - Cross Entropy: 0.15273253982757
58[2020-09-04 08:39:04] INFO: Epoch 8 - Cross Entropy: 0.15011771931339
59[2020-09-04 08:39:04] INFO: Epoch 9 - Cross Entropy: 0.14747194148672
60[2020-09-04 08:39:04] INFO: Epoch 10 - Cross Entropy: 0.14479847759871
61...
62[2020-09-04 08:39:04] INFO: Epoch 77 - Cross Entropy: 0.0082096137827592
63[2020-09-04 08:39:04] INFO: Epoch 78 - Cross Entropy: 0.0081004235278088
64[2020-09-04 08:39:04] INFO: Epoch 79 - Cross Entropy: 0.0079956096838174
65[2020-09-04 08:39:04] INFO: Epoch 80 - Cross Entropy: 0.0078948616067878
66[2020-09-04 08:39:04] INFO: Epoch 81 - Cross Entropy: 0.0077978960869396
67[2020-09-04 08:39:04] INFO: Training complete
68
69```
70
71## Parallel Training
72Learners that implement the [Parallel](parallel.md) interface can utilize a parallel processing (multiprocessing) backend for training. Parallel computing can greatly reduce training time on multicore systems at the cost of some overhead to synchronize the data. For small datasets, the overhead may actually cause the runtime to increase. Most parallel learners do not use parallel processing by default, so to enable it you must set a parallel backend using the `setBackend()` method. In the example below, we'll train a [Random Forest](classifiers/random-forest.md) classifier with 500 trees in parallel using the [Amp](backends/amp.md) backend under the hood. By settings the `$workers` argument to 4 we tell the backend to use up to 4 cores at a time to execute the computation.
73
74```php
75use Rubix\ML\Classifiers\RandomForest;
76use Rubix\ML\Classifiers\ClassificationTree;
77use Rubix\ML\Backends\Amp;
78
79$estimator = new RandomForest(new ClassificationTree(20), 500);
80
81$estimator->setBackend(new Amp(4));
82
83$estimator->train($dataset);
84```
85
86## Feature Importances
87Learners that implement the [Ranks Features](ranks-features.md) interface can evaluate the importance of each feature in the training set. Feature importances are defined as the degree to which an individual feature influences the model. Feature importances are useful for feature selection and for helping to explain predictions derived from a model. To output the normalized importance scores, call the `featureImportances()` method of a trained learner that implements the Ranks Features interface. In the example below, we'll sort the importances array so that we can easily see which features are most influential.
88
89```php
90use Rubix\ML\Regressors\Ridge;
91
92$estimator = new Ridge();
93
94$estimator->train($dataset);
95
96$importances = $estimator->featureImportances();
97
98arsort($importances);
99
100print_r($importances);
101```
102
103```sh
104Array
105(
106    [2] => 0.53170249909942
107    [1] => 0.3794817175945
108    [0] => 0.047576266783176
109    [3] => 0.041239516522901
110)
111```
112