README
1Libsvm is a simple, easy-to-use, and efficient software for SVM
2classification and regression. It solves C-SVM classification, nu-SVM
3classification, one-class-SVM, epsilon-SVM regression, and nu-SVM
4regression. It also provides an automatic model selection tool for
5C-SVM classification. This document explains the use of libsvm.
6
7Libsvm is available at
8http://www.csie.ntu.edu.tw/~cjlin/libsvm
9Please read the COPYRIGHT file before using libsvm.
10
11Table of Contents
12=================
13
14- Quick Start
15- Installation and Data Format
16- `svm-train' Usage
17- `svm-predict' Usage
18- `svm-scale' Usage
19- Tips on Practical Use
20- Examples
21- Precomputed Kernels
22- Library Usage
23- Java Version
24- Building Windows Binaries
25- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.
26- MATLAB/OCTAVE Interface
27- Python Interface
28- Additional Information
29
30Quick Start
31===========
32
33If you are new to SVM and if the data is not large, please go to
34`tools' directory and use easy.py after installation. It does
35everything automatic -- from data scaling to parameter selection.
36
37Usage: easy.py training_file [testing_file]
38
39More information about parameter selection can be found in
40`tools/README.'
41
42Installation and Data Format
43============================
44
45On Unix systems, type `make' to build the `svm-train' and `svm-predict'
46programs. Run them without arguments to show the usages of them.
47
48On other systems, consult `Makefile' to build them (e.g., see
49'Building Windows binaries' in this file) or use the pre-built
50binaries (Windows binaries are in the directory `windows').
51
52The format of training and testing data files is:
53
54<label> <index1>:<value1> <index2>:<value2> ...
55.
56.
57.
58
59Each line contains an instance and is ended by a '\n' character. For
60<labal> in the training set, we have the following cases.
61
62* classification: <label> is an integer indicating the class label
63 (multi-class is supported).
64
65* For regression, <label> is the target value which can be any real
66 number.
67
68* For one-class SVM, <label> is not used and can be any number.
69
70In the test set, <label> is used only to calculate accuracy or
71errors. If it's unknown, any number is fine. For one-class SVM, if
72non-outliers/outliers are known, their labels in the test file must be
73+1/-1 for evaluation.
74
75The pair <index>:<value> gives a feature (attribute) value: <index> is
76an integer starting from 1 and <value> is a real number. The only
77exception is the precomputed kernel, where <index> starts from 0; see
78the section of precomputed kernels. Indices must be in ASCENDING
79order.
80
81A sample classification data included in this package is
82`heart_scale'. To check if your data is in a correct form, use
83`tools/checkdata.py' (details in `tools/README').
84
85Type `svm-train heart_scale', and the program will read the training
86data and output the model file `heart_scale.model'. If you have a test
87set called heart_scale.t, then type `svm-predict heart_scale.t
88heart_scale.model output' to see the prediction accuracy. The `output'
89file contains the predicted class labels.
90
91For classification, if training data are in only one class (i.e., all
92labels are the same), then `svm-train' issues a warning message:
93`Warning: training data in only one class. See README for details,'
94which means the training data is very unbalanced. The label in the
95training data is directly returned when testing.
96
97There are some other useful programs in this package.
98
99svm-scale:
100
101 This is a tool for scaling input data file.
102
103svm-toy:
104
105 This is a simple graphical interface which shows how SVM
106 separate data in a plane. You can click in the window to
107 draw data points. Use "change" button to choose class
108 1, 2 or 3 (i.e., up to three classes are supported), "load"
109 button to load data from a file, "save" button to save data to
110 a file, "run" button to obtain an SVM model, and "clear"
111 button to clear the window.
112
113 You can enter options in the bottom of the window, the syntax of
114 options is the same as `svm-train'.
115
116 Note that "load" and "save" consider dense data format both in
117 classification and the regression cases. For classification,
118 each data point has one label (the color) that must be 1, 2,
119 or 3 and two attributes (x-axis and y-axis values) in
120 [0,1). For regression, each data point has one target value
121 (y-axis) and one attribute (x-axis values) in [0, 1).
122
123 Type `make' in respective directories to build them.
124
125 You need Qt library to build the Qt version.
126 (available from http://www.trolltech.com)
127
128 You need GTK+ library to build the GTK version.
129 (available from http://www.gtk.org)
130
131 The pre-built Windows binaries are in the `windows'
132 directory. We use Visual C++ on a 64-bit machine.
133
134`svm-train' Usage
135=================
136
137Usage: svm-train [options] training_set_file [model_file]
138options:
139-s svm_type : set type of SVM (default 0)
140 0 -- C-SVC (multi-class classification)
141 1 -- nu-SVC (multi-class classification)
142 2 -- one-class SVM
143 3 -- epsilon-SVR (regression)
144 4 -- nu-SVR (regression)
145-t kernel_type : set type of kernel function (default 2)
146 0 -- linear: u'*v
147 1 -- polynomial: (gamma*u'*v + coef0)^degree
148 2 -- radial basis function: exp(-gamma*|u-v|^2)
149 3 -- sigmoid: tanh(gamma*u'*v + coef0)
150 4 -- precomputed kernel (kernel values in training_set_file)
151-d degree : set degree in kernel function (default 3)
152-g gamma : set gamma in kernel function (default 1/num_features)
153-r coef0 : set coef0 in kernel function (default 0)
154-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
155-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
156-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
157-m cachesize : set cache memory size in MB (default 100)
158-e epsilon : set tolerance of termination criterion (default 0.001)
159-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)
160-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
161-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)
162-v n: n-fold cross validation mode
163-q : quiet mode (no outputs)
164
165
166The k in the -g option means the number of attributes in the input data.
167
168option -v randomly splits the data into n parts and calculates cross
169validation accuracy/mean squared error on them.
170
171See libsvm FAQ for the meaning of outputs.
172
173`svm-predict' Usage
174===================
175
176Usage: svm-predict [options] test_file model_file output_file
177options:
178-b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0); for one-class SVM only 0 is supported
179
180model_file is the model file generated by svm-train.
181test_file is the test data you want to predict.
182svm-predict will produce output in the output_file.
183
184`svm-scale' Usage
185=================
186
187Usage: svm-scale [options] data_filename
188options:
189-l lower : x scaling lower limit (default -1)
190-u upper : x scaling upper limit (default +1)
191-y y_lower y_upper : y scaling limits (default: no y scaling)
192-s save_filename : save scaling parameters to save_filename
193-r restore_filename : restore scaling parameters from restore_filename
194
195See 'Examples' in this file for examples.
196
197Tips on Practical Use
198=====================
199
200* Scale your data. For example, scale each attribute to [0,1] or [-1,+1].
201* For C-SVC, consider using the model selection tool in the tools directory.
202* nu in nu-SVC/one-class-SVM/nu-SVR approximates the fraction of training
203 errors and support vectors.
204* If data for classification are unbalanced (e.g. many positive and
205 few negative), try different penalty parameters C by -wi (see
206 examples below).
207* Specify larger cache size (i.e., larger -m) for huge problems.
208
209Examples
210========
211
212> svm-scale -l -1 -u 1 -s range train > train.scale
213> svm-scale -r range test > test.scale
214
215Scale each feature of the training data to be in [-1,1]. Scaling
216factors are stored in the file range and then used for scaling the
217test data.
218
219> svm-train -s 0 -c 5 -t 2 -g 0.5 -e 0.1 data_file
220
221Train a classifier with RBF kernel exp(-0.5|u-v|^2), C=10, and
222stopping tolerance 0.1.
223
224> svm-train -s 3 -p 0.1 -t 0 data_file
225
226Solve SVM regression with linear kernel u'v and epsilon=0.1
227in the loss function.
228
229> svm-train -c 10 -w1 1 -w-2 5 -w4 2 data_file
230
231Train a classifier with penalty 10 = 1 * 10 for class 1, penalty 50 =
2325 * 10 for class -2, and penalty 20 = 2 * 10 for class 4.
233
234> svm-train -s 0 -c 100 -g 0.1 -v 5 data_file
235
236Do five-fold cross validation for the classifier using
237the parameters C = 100 and gamma = 0.1
238
239> svm-train -s 0 -b 1 data_file
240> svm-predict -b 1 test_file data_file.model output_file
241
242Obtain a model with probability information and predict test data with
243probability estimates
244
245Precomputed Kernels
246===================
247
248Users may precompute kernel values and input them as training and
249testing files. Then libsvm does not need the original
250training/testing sets.
251
252Assume there are L training instances x1, ..., xL and.
253Let K(x, y) be the kernel
254value of two instances x and y. The input formats
255are:
256
257New training instance for xi:
258
259<label> 0:i 1:K(xi,x1) ... L:K(xi,xL)
260
261New testing instance for any x:
262
263<label> 0:? 1:K(x,x1) ... L:K(x,xL)
264
265That is, in the training file the first column must be the "ID" of
266xi. In testing, ? can be any value.
267
268All kernel values including ZEROs must be explicitly provided. Any
269permutation or random subsets of the training/testing files are also
270valid (see examples below).
271
272Note: the format is slightly different from the precomputed kernel
273package released in libsvmtools earlier.
274
275Examples:
276
277 Assume the original training data has three four-feature
278 instances and testing data has one instance:
279
280 15 1:1 2:1 3:1 4:1
281 45 2:3 4:3
282 25 3:1
283
284 15 1:1 3:1
285
286 If the linear kernel is used, we have the following new
287 training/testing sets:
288
289 15 0:1 1:4 2:6 3:1
290 45 0:2 1:6 2:18 3:0
291 25 0:3 1:1 2:0 3:1
292
293 15 0:? 1:2 2:0 3:1
294
295 ? can be any value.
296
297 Any subset of the above training file is also valid. For example,
298
299 25 0:3 1:1 2:0 3:1
300 45 0:2 1:6 2:18 3:0
301
302 implies that the kernel matrix is
303
304 [K(2,2) K(2,3)] = [18 0]
305 [K(3,2) K(3,3)] = [0 1]
306
307Library Usage
308=============
309
310These functions and structures are declared in the header file
311`svm.h'. You need to #include "svm.h" in your C/C++ source files and
312link your program with `svm.cpp'. You can see `svm-train.c' and
313`svm-predict.c' for examples showing how to use them. We define
314LIBSVM_VERSION and declare `extern int libsvm_version;' in svm.h, so
315you can check the version number.
316
317Before you classify test data, you need to construct an SVM model
318(`svm_model') using training data. A model can also be saved in
319a file for later use. Once an SVM model is available, you can use it
320to classify new data.
321
322- Function: struct svm_model *svm_train(const struct svm_problem *prob,
323 const struct svm_parameter *param);
324
325 This function constructs and returns an SVM model according to
326 the given training data and parameters.
327
328 struct svm_problem describes the problem:
329
330 struct svm_problem
331 {
332 int l;
333 double *y;
334 struct svm_node **x;
335 };
336
337 where `l' is the number of training data, and `y' is an array containing
338 their target values. (integers in classification, real numbers in
339 regression) `x' is an array of pointers, each of which points to a sparse
340 representation (array of svm_node) of one training vector.
341
342 For example, if we have the following training data:
343
344 LABEL ATTR1 ATTR2 ATTR3 ATTR4 ATTR5
345 ----- ----- ----- ----- ----- -----
346 1 0 0.1 0.2 0 0
347 2 0 0.1 0.3 -1.2 0
348 1 0.4 0 0 0 0
349 2 0 0.1 0 1.4 0.5
350 3 -0.1 -0.2 0.1 1.1 0.1
351
352 then the components of svm_problem are:
353
354 l = 5
355
356 y -> 1 2 1 2 3
357
358 x -> [ ] -> (2,0.1) (3,0.2) (-1,?)
359 [ ] -> (2,0.1) (3,0.3) (4,-1.2) (-1,?)
360 [ ] -> (1,0.4) (-1,?)
361 [ ] -> (2,0.1) (4,1.4) (5,0.5) (-1,?)
362 [ ] -> (1,-0.1) (2,-0.2) (3,0.1) (4,1.1) (5,0.1) (-1,?)
363
364 where (index,value) is stored in the structure `svm_node':
365
366 struct svm_node
367 {
368 int index;
369 double value;
370 };
371
372 index = -1 indicates the end of one vector. Note that indices must
373 be in ASCENDING order.
374
375 struct svm_parameter describes the parameters of an SVM model:
376
377 struct svm_parameter
378 {
379 int svm_type;
380 int kernel_type;
381 int degree; /* for poly */
382 double gamma; /* for poly/rbf/sigmoid */
383 double coef0; /* for poly/sigmoid */
384
385 /* these are for training only */
386 double cache_size; /* in MB */
387 double eps; /* stopping criteria */
388 double C; /* for C_SVC, EPSILON_SVR, and NU_SVR */
389 int nr_weight; /* for C_SVC */
390 int *weight_label; /* for C_SVC */
391 double* weight; /* for C_SVC */
392 double nu; /* for NU_SVC, ONE_CLASS, and NU_SVR */
393 double p; /* for EPSILON_SVR */
394 int shrinking; /* use the shrinking heuristics */
395 int probability; /* do probability estimates */
396 };
397
398 svm_type can be one of C_SVC, NU_SVC, ONE_CLASS, EPSILON_SVR, NU_SVR.
399
400 C_SVC: C-SVM classification
401 NU_SVC: nu-SVM classification
402 ONE_CLASS: one-class-SVM
403 EPSILON_SVR: epsilon-SVM regression
404 NU_SVR: nu-SVM regression
405
406 kernel_type can be one of LINEAR, POLY, RBF, SIGMOID.
407
408 LINEAR: u'*v
409 POLY: (gamma*u'*v + coef0)^degree
410 RBF: exp(-gamma*|u-v|^2)
411 SIGMOID: tanh(gamma*u'*v + coef0)
412 PRECOMPUTED: kernel values in training_set_file
413
414 cache_size is the size of the kernel cache, specified in megabytes.
415 C is the cost of constraints violation.
416 eps is the stopping criterion. (we usually use 0.00001 in nu-SVC,
417 0.001 in others). nu is the parameter in nu-SVM, nu-SVR, and
418 one-class-SVM. p is the epsilon in epsilon-insensitive loss function
419 of epsilon-SVM regression. shrinking = 1 means shrinking is conducted;
420 = 0 otherwise. probability = 1 means model with probability
421 information is obtained; = 0 otherwise.
422
423 nr_weight, weight_label, and weight are used to change the penalty
424 for some classes (If the weight for a class is not changed, it is
425 set to 1). This is useful for training classifier using unbalanced
426 input data or with asymmetric misclassification cost.
427
428 nr_weight is the number of elements in the array weight_label and
429 weight. Each weight[i] corresponds to weight_label[i], meaning that
430 the penalty of class weight_label[i] is scaled by a factor of weight[i].
431
432 If you do not want to change penalty for any of the classes,
433 just set nr_weight to 0.
434
435 *NOTE* Because svm_model contains pointers to svm_problem, you can
436 not free the memory used by svm_problem if you are still using the
437 svm_model produced by svm_train().
438
439 *NOTE* To avoid wrong parameters, svm_check_parameter() should be
440 called before svm_train().
441
442 struct svm_model stores the model obtained from the training procedure.
443 It is not recommended to directly access entries in this structure.
444 Programmers should use the interface functions to get the values.
445
446 struct svm_model
447 {
448 struct svm_parameter param; /* parameter */
449 int nr_class; /* number of classes, = 2 in regression/one class svm */
450 int l; /* total #SV */
451 struct svm_node **SV; /* SVs (SV[l]) */
452 double **sv_coef; /* coefficients for SVs in decision functions (sv_coef[k-1][l]) */
453 double *rho; /* constants in decision functions (rho[k*(k-1)/2]) */
454 double *probA; /* pairwise probability information */
455 double *probB;
456 int *sv_indices; /* sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to indicate SVs in the training set */
457
458 /* for classification only */
459
460 int *label; /* label of each class (label[k]) */
461 int *nSV; /* number of SVs for each class (nSV[k]) */
462 /* nSV[0] + nSV[1] + ... + nSV[k-1] = l */
463 /* XXX */
464 int free_sv; /* 1 if svm_model is created by svm_load_model*/
465 /* 0 if svm_model is created by svm_train */
466 };
467
468 param describes the parameters used to obtain the model.
469
470 nr_class is the number of classes. It is 2 for regression and one-class SVM.
471
472 l is the number of support vectors. SV and sv_coef are support
473 vectors and the corresponding coefficients, respectively. Assume there are
474 k classes. For data in class j, the corresponding sv_coef includes (k-1) y*alpha vectors,
475 where alpha's are solutions of the following two class problems:
476 1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k
477 and y=1 for the first j-1 vectors, while y=-1 for the remaining k-j
478 vectors. For example, if there are 4 classes, sv_coef and SV are like:
479
480 +-+-+-+--------------------+
481 |1|1|1| |
482 |v|v|v| SVs from class 1 |
483 |2|3|4| |
484 +-+-+-+--------------------+
485 |1|2|2| |
486 |v|v|v| SVs from class 2 |
487 |2|3|4| |
488 +-+-+-+--------------------+
489 |1|2|3| |
490 |v|v|v| SVs from class 3 |
491 |3|3|4| |
492 +-+-+-+--------------------+
493 |1|2|3| |
494 |v|v|v| SVs from class 4 |
495 |4|4|4| |
496 +-+-+-+--------------------+
497
498 See svm_train() for an example of assigning values to sv_coef.
499
500 rho is the bias term (-b). probA and probB are parameters used in
501 probability outputs. If there are k classes, there are k*(k-1)/2
502 binary problems as well as rho, probA, and probB values. They are
503 aligned in the order of binary problems:
504 1 vs 2, 1 vs 3, ..., 1 vs k, 2 vs 3, ..., 2 vs k, ..., k-1 vs k.
505
506 sv_indices[0,...,nSV-1] are values in [1,...,num_traning_data] to
507 indicate support vectors in the training set.
508
509 label contains labels in the training data.
510
511 nSV is the number of support vectors in each class.
512
513 free_sv is a flag used to determine whether the space of SV should
514 be released in free_model_content(struct svm_model*) and
515 free_and_destroy_model(struct svm_model**). If the model is
516 generated by svm_train(), then SV points to data in svm_problem
517 and should not be removed. For example, free_sv is 0 if svm_model
518 is created by svm_train, but is 1 if created by svm_load_model.
519
520- Function: double svm_predict(const struct svm_model *model,
521 const struct svm_node *x);
522
523 This function does classification or regression on a test vector x
524 given a model.
525
526 For a classification model, the predicted class for x is returned.
527 For a regression model, the function value of x calculated using
528 the model is returned. For an one-class model, +1 or -1 is
529 returned.
530
531- Function: void svm_cross_validation(const struct svm_problem *prob,
532 const struct svm_parameter *param, int nr_fold, double *target);
533
534 This function conducts cross validation. Data are separated to
535 nr_fold folds. Under given parameters, sequentially each fold is
536 validated using the model from training the remaining. Predicted
537 labels (of all prob's instances) in the validation process are
538 stored in the array called target.
539
540 The format of svm_prob is same as that for svm_train().
541
542- Function: int svm_get_svm_type(const struct svm_model *model);
543
544 This function gives svm_type of the model. Possible values of
545 svm_type are defined in svm.h.
546
547- Function: int svm_get_nr_class(const svm_model *model);
548
549 For a classification model, this function gives the number of
550 classes. For a regression or an one-class model, 2 is returned.
551
552- Function: void svm_get_labels(const svm_model *model, int* label)
553
554 For a classification model, this function outputs the name of
555 labels into an array called label. For regression and one-class
556 models, label is unchanged.
557
558- Function: void svm_get_sv_indices(const struct svm_model *model, int *sv_indices)
559
560 This function outputs indices of support vectors into an array called sv_indices.
561 The size of sv_indices is the number of support vectors and can be obtained by calling svm_get_nr_sv.
562 Each sv_indices[i] is in the range of [1, ..., num_traning_data].
563
564- Function: int svm_get_nr_sv(const struct svm_model *model)
565
566 This function gives the number of total support vector.
567
568- Function: double svm_get_svr_probability(const struct svm_model *model);
569
570 For a regression model with probability information, this function
571 outputs a value sigma > 0. For test data, we consider the
572 probability model: target value = predicted value + z, z: Laplace
573 distribution e^(-|z|/sigma)/(2sigma)
574
575 If the model is not for svr or does not contain required
576 information, 0 is returned.
577
578- Function: double svm_predict_values(const svm_model *model,
579 const svm_node *x, double* dec_values)
580
581 This function gives decision values on a test vector x given a
582 model, and return the predicted label (classification) or
583 the function value (regression).
584
585 For a classification model with nr_class classes, this function
586 gives nr_class*(nr_class-1)/2 decision values in the array
587 dec_values, where nr_class can be obtained from the function
588 svm_get_nr_class. The order is label[0] vs. label[1], ...,
589 label[0] vs. label[nr_class-1], label[1] vs. label[2], ...,
590 label[nr_class-2] vs. label[nr_class-1], where label can be
591 obtained from the function svm_get_labels. The returned value is
592 the predicted class for x. Note that when nr_class = 1, this
593 function does not give any decision value.
594
595 For a regression model, dec_values[0] and the returned value are
596 both the function value of x calculated using the model. For a
597 one-class model, dec_values[0] is the decision value of x, while
598 the returned value is +1/-1.
599
600- Function: double svm_predict_probability(const struct svm_model *model,
601 const struct svm_node *x, double* prob_estimates);
602
603 This function does classification or regression on a test vector x
604 given a model with probability information.
605
606 For a classification model with probability information, this
607 function gives nr_class probability estimates in the array
608 prob_estimates. nr_class can be obtained from the function
609 svm_get_nr_class. The class with the highest probability is
610 returned. For regression/one-class SVM, the array prob_estimates
611 is unchanged and the returned value is the same as that of
612 svm_predict.
613
614- Function: const char *svm_check_parameter(const struct svm_problem *prob,
615 const struct svm_parameter *param);
616
617 This function checks whether the parameters are within the feasible
618 range of the problem. This function should be called before calling
619 svm_train() and svm_cross_validation(). It returns NULL if the
620 parameters are feasible, otherwise an error message is returned.
621
622- Function: int svm_check_probability_model(const struct svm_model *model);
623
624 This function checks whether the model contains required
625 information to do probability estimates. If so, it returns
626 +1. Otherwise, 0 is returned. This function should be called
627 before calling svm_get_svr_probability and
628 svm_predict_probability.
629
630- Function: int svm_save_model(const char *model_file_name,
631 const struct svm_model *model);
632
633 This function saves a model to a file; returns 0 on success, or -1
634 if an error occurs.
635
636- Function: struct svm_model *svm_load_model(const char *model_file_name);
637
638 This function returns a pointer to the model read from the file,
639 or a null pointer if the model could not be loaded.
640
641- Function: void svm_free_model_content(struct svm_model *model_ptr);
642
643 This function frees the memory used by the entries in a model structure.
644
645- Function: void svm_free_and_destroy_model(struct svm_model **model_ptr_ptr);
646
647 This function frees the memory used by a model and destroys the model
648 structure. It is equivalent to svm_destroy_model, which
649 is deprecated after version 3.0.
650
651- Function: void svm_destroy_param(struct svm_parameter *param);
652
653 This function frees the memory used by a parameter set.
654
655- Function: void svm_set_print_string_function(void (*print_func)(const char *));
656
657 Users can specify their output format by a function. Use
658 svm_set_print_string_function(NULL);
659 for default printing to stdout.
660
661Java Version
662============
663
664The pre-compiled java class archive `libsvm.jar' and its source files are
665in the java directory. To run the programs, use
666
667java -classpath libsvm.jar svm_train <arguments>
668java -classpath libsvm.jar svm_predict <arguments>
669java -classpath libsvm.jar svm_toy
670java -classpath libsvm.jar svm_scale <arguments>
671
672Note that you need Java 1.5 (5.0) or above to run it.
673
674You may need to add Java runtime library (like classes.zip) to the classpath.
675You may need to increase maximum Java heap size.
676
677Library usages are similar to the C version. These functions are available:
678
679public class svm {
680 public static final int LIBSVM_VERSION=323;
681 public static svm_model svm_train(svm_problem prob, svm_parameter param);
682 public static void svm_cross_validation(svm_problem prob, svm_parameter param, int nr_fold, double[] target);
683 public static int svm_get_svm_type(svm_model model);
684 public static int svm_get_nr_class(svm_model model);
685 public static void svm_get_labels(svm_model model, int[] label);
686 public static void svm_get_sv_indices(svm_model model, int[] indices);
687 public static int svm_get_nr_sv(svm_model model);
688 public static double svm_get_svr_probability(svm_model model);
689 public static double svm_predict_values(svm_model model, svm_node[] x, double[] dec_values);
690 public static double svm_predict(svm_model model, svm_node[] x);
691 public static double svm_predict_probability(svm_model model, svm_node[] x, double[] prob_estimates);
692 public static void svm_save_model(String model_file_name, svm_model model) throws IOException
693 public static svm_model svm_load_model(String model_file_name) throws IOException
694 public static String svm_check_parameter(svm_problem prob, svm_parameter param);
695 public static int svm_check_probability_model(svm_model model);
696 public static void svm_set_print_string_function(svm_print_interface print_func);
697}
698
699The library is in the "libsvm" package.
700Note that in Java version, svm_node[] is not ended with a node whose index = -1.
701
702Users can specify their output format by
703
704 your_print_func = new svm_print_interface()
705 {
706 public void print(String s)
707 {
708 // your own format
709 }
710 };
711 svm.svm_set_print_string_function(your_print_func);
712
713Building Windows Binaries
714=========================
715
716Windows binaries are available in the directory `windows'. To re-build
717them via Visual C++, use the following steps:
718
7191. Open a DOS command box (or Visual Studio Command Prompt) and change
720to libsvm directory. If environment variables of VC++ have not been
721set, type
722
723""C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\amd64\vcvars64.bat""
724
725You may have to modify the above command according which version of
726VC++ or where it is installed.
727
7282. Type
729
730nmake -f Makefile.win clean all
731
7323. (optional) To build shared library libsvm.dll, type
733
734nmake -f Makefile.win lib
735
7364. (optional) To build 32-bit windows binaries, you must
737 (1) Setup "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\vcvars32.bat" instead of vcvars64.bat
738 (2) Change CFLAGS in Makefile.win: /D _WIN64 to /D _WIN32
739
740Another way is to build them from Visual C++ environment. See details
741in libsvm FAQ.
742
743- Additional Tools: Sub-sampling, Parameter Selection, Format checking, etc.
744============================================================================
745
746See the README file in the tools directory.
747
748MATLAB/OCTAVE Interface
749=======================
750
751Please check the file README in the directory `matlab'.
752
753Python Interface
754================
755
756See the README file in python directory.
757
758Additional Information
759======================
760
761If you find LIBSVM helpful, please cite it as
762
763Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support
764vector machines. ACM Transactions on Intelligent Systems and
765Technology, 2:27:1--27:27, 2011. Software available at
766http://www.csie.ntu.edu.tw/~cjlin/libsvm
767
768LIBSVM implementation document is available at
769http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf
770
771For any questions and comments, please email cjlin@csie.ntu.edu.tw
772
773Acknowledgments:
774This work was supported in part by the National Science
775Council of Taiwan via the grant NSC 89-2213-E-002-013.
776The authors thank their group members and users
777for many helpful discussions and comments. They are listed in
778http://www.csie.ntu.edu.tw/~cjlin/libsvm/acknowledgements
779
780
README.python
1----------------------------------
2--- Python interface of LIBSVM ---
3----------------------------------
4
5Table of Contents
6=================
7
8- Introduction
9- Installation
10- Quick Start
11- Quick Start with Scipy
12- Design Description
13- Data Structures
14- Utility Functions
15- Additional Information
16
17Introduction
18============
19
20Python (http://www.python.org/) is a programming language suitable for rapid
21development. This tool provides a simple Python interface to LIBSVM, a library
22for support vector machines (http://www.csie.ntu.edu.tw/~cjlin/libsvm). The
23interface is very easy to use as the usage is the same as that of LIBSVM. The
24interface is developed with the built-in Python library "ctypes."
25
26Installation
27============
28
29On Unix systems, type
30
31> make
32
33The interface needs only LIBSVM shared library, which is generated by
34the above command. We assume that the shared library is on the LIBSVM
35main directory or in the system path.
36
37For windows, the shared library libsvm.dll for 64-bit python is ready
38in the directory `..\windows'. To regenerate the shared library,
39please follow the instruction of building windows binaries in LIBSVM
40README.
41
42Quick Start
43===========
44
45"Quick Start with Scipy" is in the next section.
46
47There are two levels of usage. The high-level one uses utility functions
48in svmutil.py and the usage is the same as the LIBSVM MATLAB interface.
49
50>>> from svmutil import *
51# Read data in LIBSVM format
52>>> y, x = svm_read_problem('../heart_scale')
53>>> m = svm_train(y[:200], x[:200], '-c 4')
54>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:], m)
55
56# Construct problem in python format
57# Dense data
58>>> y, x = [1,-1], [[1,0,1], [-1,0,-1]]
59# Sparse data
60>>> y, x = [1,-1], [{1:1, 3:1}, {1:-1,3:-1}]
61>>> prob = svm_problem(y, x)
62>>> param = svm_parameter('-t 0 -c 4 -b 1')
63>>> m = svm_train(prob, param)
64
65# Precomputed kernel data (-t 4)
66# Dense data
67>>> y, x = [1,-1], [[1, 2, -2], [2, -2, 2]]
68# Sparse data
69>>> y, x = [1,-1], [{0:1, 1:2, 2:-2}, {0:2, 1:-2, 2:2}]
70# isKernel=True must be set for precomputed kernel
71>>> prob = svm_problem(y, x, isKernel=True)
72>>> param = svm_parameter('-t 4 -c 4 -b 1')
73>>> m = svm_train(prob, param)
74# For the format of precomputed kernel, please read LIBSVM README.
75
76
77# Other utility functions
78>>> svm_save_model('heart_scale.model', m)
79>>> m = svm_load_model('heart_scale.model')
80>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')
81>>> ACC, MSE, SCC = evaluations(y, p_label)
82
83# Getting online help
84>>> help(svm_train)
85
86The low-level use directly calls C interfaces imported by svm.py. Note that
87all arguments and return values are in ctypes format. You need to handle them
88carefully.
89
90>>> from svm import *
91>>> prob = svm_problem([1,-1], [{1:1, 3:1}, {1:-1,3:-1}])
92>>> param = svm_parameter('-c 4')
93>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model
94# Convert a Python-format instance to svm_nodearray, a ctypes structure
95>>> x0, max_idx = gen_svm_nodearray({1:1, 3:1})
96>>> label = libsvm.svm_predict(m, x0)
97
98Quick Start with Scipy
99======================
100
101Make sure you have Scipy installed to proceed in this section.
102If numba (http://numba.pydata.org) is installed, some operations will be much faster.
103
104There are two levels of usage. The high-level one uses utility functions
105in svmutil.py and the usage is the same as the LIBSVM MATLAB interface.
106
107>>> import scipy
108>>> from svmutil import *
109# Read data in LIBSVM format
110>>> y, x = svm_read_problem('../heart_scale', return_scipy = True) # y: ndarray, x: csr_matrix
111>>> m = svm_train(y[:200], x[:200, :], '-c 4')
112>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:, :], m)
113
114# Construct problem in Scipy format
115# Dense data: numpy ndarray
116>>> y, x = scipy.asarray([1,-1]), scipy.asarray([[1,0,1], [-1,0,-1]])
117# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))
118>>> y, x = scipy.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2])))
119>>> prob = svm_problem(y, x)
120>>> param = svm_parameter('-t 0 -c 4 -b 1')
121>>> m = svm_train(prob, param)
122
123# Precomputed kernel data (-t 4)
124# Dense data: numpy ndarray
125>>> y, x = scipy.asarray([1,-1]), scipy.asarray([[1,2,-2], [2,-2,2]])
126# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))
127>>> y, x = scipy.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 2, -2, 2, -2, 2], ([0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2])))
128# isKernel=True must be set for precomputed kernel
129>>> prob = svm_problem(y, x, isKernel=True)
130>>> param = svm_parameter('-t 4 -c 4 -b 1')
131>>> m = svm_train(prob, param)
132# For the format of precomputed kernel, please read LIBSVM README.
133
134# Apply data scaling in Scipy format
135>>> y, x = svm_read_problem('../heart_scale', return_scipy=True)
136>>> scale_param = csr_find_scale_param(x, lower=0)
137>>> scaled_x = csr_scale(x, scale_param)
138
139# Other utility functions
140>>> svm_save_model('heart_scale.model', m)
141>>> m = svm_load_model('heart_scale.model')
142>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')
143>>> ACC, MSE, SCC = evaluations(y, p_label)
144
145# Getting online help
146>>> help(svm_train)
147
148The low-level use directly calls C interfaces imported by svm.py. Note that
149all arguments and return values are in ctypes format. You need to handle them
150carefully.
151
152>>> from svm import *
153>>> prob = svm_problem(scipy.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2]))))
154>>> param = svm_parameter('-c 4')
155>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model
156# Convert a tuple of ndarray (index, data) to feature_nodearray, a ctypes structure
157# Note that index starts from 0, though the following example will be changed to 1:1, 3:1 internally
158>>> x0, max_idx = gen_svm_nodearray((scipy.asarray([0,2]), scipy.asarray([1,1])))
159>>> label = libsvm.svm_predict(m, x0)
160
161Design Description
162==================
163
164There are two files svm.py and svmutil.py, which respectively correspond to
165low-level and high-level use of the interface.
166
167In svm.py, we adopt the Python built-in library "ctypes," so that
168Python can directly access C structures and interface functions defined
169in svm.h.
170
171While advanced users can use structures/functions in svm.py, to
172avoid handling ctypes structures, in svmutil.py we provide some easy-to-use
173functions. The usage is similar to LIBSVM MATLAB interface.
174
175Data Structures
176===============
177
178Four data structures derived from svm.h are svm_node, svm_problem, svm_parameter,
179and svm_model. They all contain fields with the same names in svm.h. Access
180these fields carefully because you directly use a C structure instead of a
181Python object. For svm_model, accessing the field directly is not recommanded.
182Programmers should use the interface functions or methods of svm_model class
183in Python to get the values. The following description introduces additional
184fields and methods.
185
186Before using the data structures, execute the following command to load the
187LIBSVM shared library:
188
189 >>> from svm import *
190
191- class svm_node:
192
193 Construct an svm_node.
194
195 >>> node = svm_node(idx, val)
196
197 idx: an integer indicates the feature index.
198
199 val: a float indicates the feature value.
200
201 Show the index and the value of a node.
202
203 >>> print(node)
204
205- Function: gen_svm_nodearray(xi [,feature_max=None [,isKernel=False]])
206
207 Generate a feature vector from a Python list/tuple/dictionary, numpy ndarray or tuple of (index, data):
208
209 >>> xi_ctype, max_idx = gen_svm_nodearray({1:1, 3:1, 5:-2})
210
211 xi_ctype: the returned svm_nodearray (a ctypes structure)
212
213 max_idx: the maximal feature index of xi
214
215 feature_max: if feature_max is assigned, features with indices larger than
216 feature_max are removed.
217
218 isKernel: if isKernel == True, the list index starts from 0 for precomputed
219 kernel. Otherwise, the list index starts from 1. The default
220 value is False.
221
222- class svm_problem:
223
224 Construct an svm_problem instance
225
226 >>> prob = svm_problem(y, x)
227
228 y: a Python list/tuple/ndarray of l labels (type must be int/double).
229
230 x: 1. a list/tuple of l training instances. Feature vector of
231 each training instance is a list/tuple or dictionary.
232
233 2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
234
235 Note that if your x contains sparse data (i.e., dictionary), the internal
236 ctypes data format is still sparse.
237
238 For pre-computed kernel, the isKernel flag should be set to True:
239
240 >>> prob = svm_problem(y, x, isKernel=True)
241
242 Please read LIBSVM README for more details of pre-computed kernel.
243
244- class svm_parameter:
245
246 Construct an svm_parameter instance
247
248 >>> param = svm_parameter('training_options')
249
250 If 'training_options' is empty, LIBSVM default values are applied.
251
252 Set param to LIBSVM default values.
253
254 >>> param.set_to_default_values()
255
256 Parse a string of options.
257
258 >>> param.parse_options('training_options')
259
260 Show values of parameters.
261
262 >>> print(param)
263
264- class svm_model:
265
266 There are two ways to obtain an instance of svm_model:
267
268 >>> model = svm_train(y, x)
269 >>> model = svm_load_model('model_file_name')
270
271 Note that the returned structure of interface functions
272 libsvm.svm_train and libsvm.svm_load_model is a ctypes pointer of
273 svm_model, which is different from the svm_model object returned
274 by svm_train and svm_load_model in svmutil.py. We provide a
275 function toPyModel for the conversion:
276
277 >>> model_ptr = libsvm.svm_train(prob, param)
278 >>> model = toPyModel(model_ptr)
279
280 If you obtain a model in a way other than the above approaches,
281 handle it carefully to avoid memory leak or segmentation fault.
282
283 Some interface functions to access LIBSVM models are wrapped as
284 members of the class svm_model:
285
286 >>> svm_type = model.get_svm_type()
287 >>> nr_class = model.get_nr_class()
288 >>> svr_probability = model.get_svr_probability()
289 >>> class_labels = model.get_labels()
290 >>> sv_indices = model.get_sv_indices()
291 >>> nr_sv = model.get_nr_sv()
292 >>> is_prob_model = model.is_probability_model()
293 >>> support_vector_coefficients = model.get_sv_coef()
294 >>> support_vectors = model.get_SV()
295
296Utility Functions
297=================
298
299To use utility functions, type
300
301 >>> from svmutil import *
302
303The above command loads
304 svm_train() : train an SVM model
305 svm_predict() : predict testing data
306 svm_read_problem() : read the data from a LIBSVM-format file.
307 svm_load_model() : load a LIBSVM model.
308 svm_save_model() : save model to a file.
309 evaluations() : evaluate prediction results.
310 csr_find_scale_param() : find scaling parameter for data in csr format.
311 csr_scale() : apply data scaling to data in csr format.
312
313- Function: svm_train
314
315 There are three ways to call svm_train()
316
317 >>> model = svm_train(y, x [, 'training_options'])
318 >>> model = svm_train(prob [, 'training_options'])
319 >>> model = svm_train(prob, param)
320
321 y: a list/tuple/ndarray of l training labels (type must be int/double).
322
323 x: 1. a list/tuple of l training instances. Feature vector of
324 each training instance is a list/tuple or dictionary.
325
326 2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
327
328 training_options: a string in the same form as that for LIBSVM command
329 mode.
330
331 prob: an svm_problem instance generated by calling
332 svm_problem(y, x).
333 For pre-computed kernel, you should use
334 svm_problem(y, x, isKernel=True)
335
336 param: an svm_parameter instance generated by calling
337 svm_parameter('training_options')
338
339 model: the returned svm_model instance. See svm.h for details of this
340 structure. If '-v' is specified, cross validation is
341 conducted and the returned model is just a scalar: cross-validation
342 accuracy for classification and mean-squared error for regression.
343
344 To train the same data many times with different
345 parameters, the second and the third ways should be faster..
346
347 Examples:
348
349 >>> y, x = svm_read_problem('../heart_scale')
350 >>> prob = svm_problem(y, x)
351 >>> param = svm_parameter('-s 3 -c 5 -h 0')
352 >>> m = svm_train(y, x, '-c 5')
353 >>> m = svm_train(prob, '-t 2 -c 5')
354 >>> m = svm_train(prob, param)
355 >>> CV_ACC = svm_train(y, x, '-v 3')
356
357- Function: svm_predict
358
359 To predict testing data with a model, use
360
361 >>> p_labs, p_acc, p_vals = svm_predict(y, x, model [,'predicting_options'])
362
363 y: a list/tuple/ndarray of l true labels (type must be int/double).
364 It is used for calculating the accuracy. Use [] if true labels are
365 unavailable.
366
367 x: 1. a list/tuple of l training instances. Feature vector of
368 each training instance is a list/tuple or dictionary.
369
370 2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
371
372 predicting_options: a string of predicting options in the same format as
373 that of LIBSVM.
374
375 model: an svm_model instance.
376
377 p_labels: a list of predicted labels
378
379 p_acc: a tuple including accuracy (for classification), mean
380 squared error, and squared correlation coefficient (for
381 regression).
382
383 p_vals: a list of decision values or probability estimates (if '-b 1'
384 is specified). If k is the number of classes in training data,
385 for decision values, each element includes results of predicting
386 k(k-1)/2 binary-class SVMs. For classification, k = 1 is a
387 special case. Decision value [+1] is returned for each testing
388 instance, instead of an empty list.
389 For probabilities, each element contains k values indicating
390 the probability that the testing instance is in each class.
391 Note that the order of classes is the same as the 'model.label'
392 field in the model structure.
393
394 Example:
395
396 >>> m = svm_train(y, x, '-c 5')
397 >>> p_labels, p_acc, p_vals = svm_predict(y, x, m)
398
399- Functions: svm_read_problem/svm_load_model/svm_save_model
400
401 See the usage by examples:
402
403 >>> y, x = svm_read_problem('data.txt')
404 >>> m = svm_load_model('model_file')
405 >>> svm_save_model('model_file', m)
406
407- Function: evaluations
408
409 Calculate some evaluations using the true values (ty) and the predicted
410 values (pv):
411
412 >>> (ACC, MSE, SCC) = evaluations(ty, pv, useScipy)
413
414 ty: a list/tuple/ndarray of true values.
415
416 pv: a list/tuple/ndarray of predicted values.
417
418 useScipy: convert ty, pv to ndarray, and use scipy functions to do the evaluation
419
420 ACC: accuracy.
421
422 MSE: mean squared error.
423
424 SCC: squared correlation coefficient.
425
426- Function: csr_find_scale_parameter/csr_scale
427
428 Scale data in csr format.
429
430 >>> param = csr_find_scale_param(x [, lower=l, upper=u])
431 >>> x = csr_scale(x, param)
432
433 x: a csr_matrix of data.
434
435 l: x scaling lower limit; default -1.
436
437 u: x scaling upper limit; default 1.
438
439 The scaling process is: x * diag(coef) + ones(l, 1) * offset'
440
441 param: a dictionary of scaling parameters, where param['coef'] = coef and param['offset'] = offset.
442
443 coef: a scipy array of scaling coefficients.
444
445 offset: a scipy array of scaling offsets.
446
447Additional Information
448======================
449
450This interface was written by Hsiang-Fu Yu from Department of Computer
451Science, National Taiwan University. If you find this tool useful, please
452cite LIBSVM as follows
453
454Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support
455vector machines. ACM Transactions on Intelligent Systems and
456Technology, 2:27:1--27:27, 2011. Software available at
457http://www.csie.ntu.edu.tw/~cjlin/libsvm
458
459For any question, please contact Chih-Jen Lin <cjlin@csie.ntu.edu.tw>,
460or check the FAQ page:
461
462http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html
463