1----------------------------------
2--- Python interface of LIBSVM ---
3----------------------------------
4
5Table of Contents
6=================
7
8- Introduction
9- Installation
10- Quick Start
11- Quick Start with Scipy
12- Design Description
13- Data Structures
14- Utility Functions
15- Additional Information
16
17Introduction
18============
19
20Python (http://www.python.org/) is a programming language suitable for rapid
21development. This tool provides a simple Python interface to LIBSVM, a library
22for support vector machines (http://www.csie.ntu.edu.tw/~cjlin/libsvm). The
23interface is very easy to use as the usage is the same as that of LIBSVM. The
24interface is developed with the built-in Python library "ctypes."
25
26Installation
27============
28
29On Unix systems, type
30
31> make
32
33The interface needs only LIBSVM shared library, which is generated by
34the above command. We assume that the shared library is on the LIBSVM
35main directory or in the system path.
36
37For windows, the shared library libsvm.dll for 64-bit python is ready
38in the directory `..\windows'. To regenerate the shared library,
39please follow the instruction of building windows binaries in LIBSVM
40README.
41
42Quick Start
43===========
44
45"Quick Start with Scipy" is in the next section.
46
47There are two levels of usage. The high-level one uses utility
48functions in svmutil.py and commonutil.py (shared with LIBLINEAR and
49imported by svmutil.py). The usage is the same as the LIBSVM MATLAB
50interface.
51
52>>> from svmutil import *
53# Read data in LIBSVM format
54>>> y, x = svm_read_problem('../heart_scale')
55>>> m = svm_train(y[:200], x[:200], '-c 4')
56>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:], m)
57
58# Construct problem in python format
59# Dense data
60>>> y, x = [1,-1], [[1,0,1], [-1,0,-1]]
61# Sparse data
62>>> y, x = [1,-1], [{1:1, 3:1}, {1:-1,3:-1}]
63>>> prob = svm_problem(y, x)
64>>> param = svm_parameter('-t 0 -c 4 -b 1')
65>>> m = svm_train(prob, param)
66
67# Precomputed kernel data (-t 4)
68# Dense data
69>>> y, x = [1,-1], [[1, 2, -2], [2, -2, 2]]
70# Sparse data
71>>> y, x = [1,-1], [{0:1, 1:2, 2:-2}, {0:2, 1:-2, 2:2}]
72# isKernel=True must be set for precomputed kernel
73>>> prob = svm_problem(y, x, isKernel=True)
74>>> param = svm_parameter('-t 4 -c 4 -b 1')
75>>> m = svm_train(prob, param)
76# For the format of precomputed kernel, please read LIBSVM README.
77
78
79# Other utility functions
80>>> svm_save_model('heart_scale.model', m)
81>>> m = svm_load_model('heart_scale.model')
82>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')
83>>> ACC, MSE, SCC = evaluations(y, p_label)
84
85# Getting online help
86>>> help(svm_train)
87
88The low-level use directly calls C interfaces imported by svm.py. Note that
89all arguments and return values are in ctypes format. You need to handle them
90carefully.
91
92>>> from svm import *
93>>> prob = svm_problem([1,-1], [{1:1, 3:1}, {1:-1,3:-1}])
94>>> param = svm_parameter('-c 4')
95>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model
96# Convert a Python-format instance to svm_nodearray, a ctypes structure
97>>> x0, max_idx = gen_svm_nodearray({1:1, 3:1})
98>>> label = libsvm.svm_predict(m, x0)
99
100Quick Start with Scipy
101======================
102
103Make sure you have Scipy installed to proceed in this section.
104If numba (http://numba.pydata.org) is installed, some operations will be much faster.
105
106There are two levels of usage. The high-level one uses utility functions
107in svmutil.py and the usage is the same as the LIBSVM MATLAB interface.
108
109>>> import scipy
110>>> from svmutil import *
111# Read data in LIBSVM format
112>>> y, x = svm_read_problem('../heart_scale', return_scipy = True) # y: ndarray, x: csr_matrix
113>>> m = svm_train(y[:200], x[:200, :], '-c 4')
114>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:, :], m)
115
116# Construct problem in Scipy format
117# Dense data: numpy ndarray
118>>> y, x = scipy.asarray([1,-1]), scipy.asarray([[1,0,1], [-1,0,-1]])
119# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))
120>>> y, x = scipy.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2])))
121>>> prob = svm_problem(y, x)
122>>> param = svm_parameter('-t 0 -c 4 -b 1')
123>>> m = svm_train(prob, param)
124
125# Precomputed kernel data (-t 4)
126# Dense data: numpy ndarray
127>>> y, x = scipy.asarray([1,-1]), scipy.asarray([[1,2,-2], [2,-2,2]])
128# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))
129>>> y, x = scipy.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 2, -2, 2, -2, 2], ([0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2])))
130# isKernel=True must be set for precomputed kernel
131>>> prob = svm_problem(y, x, isKernel=True)
132>>> param = svm_parameter('-t 4 -c 4 -b 1')
133>>> m = svm_train(prob, param)
134# For the format of precomputed kernel, please read LIBSVM README.
135
136# Apply data scaling in Scipy format
137>>> y, x = svm_read_problem('../heart_scale', return_scipy=True)
138>>> scale_param = csr_find_scale_param(x, lower=0)
139>>> scaled_x = csr_scale(x, scale_param)
140
141# Other utility functions
142>>> svm_save_model('heart_scale.model', m)
143>>> m = svm_load_model('heart_scale.model')
144>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')
145>>> ACC, MSE, SCC = evaluations(y, p_label)
146
147# Getting online help
148>>> help(svm_train)
149
150The low-level use directly calls C interfaces imported by svm.py. Note that
151all arguments and return values are in ctypes format. You need to handle them
152carefully.
153
154>>> from svm import *
155>>> prob = svm_problem(scipy.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2]))))
156>>> param = svm_parameter('-c 4')
157>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model
158# Convert a tuple of ndarray (index, data) to feature_nodearray, a ctypes structure
159# Note that index starts from 0, though the following example will be changed to 1:1, 3:1 internally
160>>> x0, max_idx = gen_svm_nodearray((scipy.asarray([0,2]), scipy.asarray([1,1])))
161>>> label = libsvm.svm_predict(m, x0)
162
163Design Description
164==================
165
166There are two files svm.py and svmutil.py, which respectively correspond to
167low-level and high-level use of the interface.
168
169In svm.py, we adopt the Python built-in library "ctypes," so that
170Python can directly access C structures and interface functions defined
171in svm.h.
172
173While advanced users can use structures/functions in svm.py, to
174avoid handling ctypes structures, in svmutil.py we provide some easy-to-use
175functions. The usage is similar to LIBSVM MATLAB interface.
176
177Data Structures
178===============
179
180Four data structures derived from svm.h are svm_node, svm_problem, svm_parameter,
181and svm_model. They all contain fields with the same names in svm.h. Access
182these fields carefully because you directly use a C structure instead of a
183Python object. For svm_model, accessing the field directly is not recommanded.
184Programmers should use the interface functions or methods of svm_model class
185in Python to get the values. The following description introduces additional
186fields and methods.
187
188Before using the data structures, execute the following command to load the
189LIBSVM shared library:
190
191 >>> from svm import *
192
193- class svm_node:
194
195 Construct an svm_node.
196
197 >>> node = svm_node(idx, val)
198
199 idx: an integer indicates the feature index.
200
201 val: a float indicates the feature value.
202
203 Show the index and the value of a node.
204
205 >>> print(node)
206
207- Function: gen_svm_nodearray(xi [,feature_max=None [,isKernel=False]])
208
209 Generate a feature vector from a Python list/tuple/dictionary, numpy ndarray or tuple of (index, data):
210
211 >>> xi_ctype, max_idx = gen_svm_nodearray({1:1, 3:1, 5:-2})
212
213 xi_ctype: the returned svm_nodearray (a ctypes structure)
214
215 max_idx: the maximal feature index of xi
216
217 feature_max: if feature_max is assigned, features with indices larger than
218 feature_max are removed.
219
220 isKernel: if isKernel == True, the list index starts from 0 for precomputed
221 kernel. Otherwise, the list index starts from 1. The default
222 value is False.
223
224- class svm_problem:
225
226 Construct an svm_problem instance
227
228 >>> prob = svm_problem(y, x)
229
230 y: a Python list/tuple/ndarray of l labels (type must be int/double).
231
232 x: 1. a list/tuple of l training instances. Feature vector of
233 each training instance is a list/tuple or dictionary.
234
235 2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
236
237 Note that if your x contains sparse data (i.e., dictionary), the internal
238 ctypes data format is still sparse.
239
240 For pre-computed kernel, the isKernel flag should be set to True:
241
242 >>> prob = svm_problem(y, x, isKernel=True)
243
244 Please read LIBSVM README for more details of pre-computed kernel.
245
246- class svm_parameter:
247
248 Construct an svm_parameter instance
249
250 >>> param = svm_parameter('training_options')
251
252 If 'training_options' is empty, LIBSVM default values are applied.
253
254 Set param to LIBSVM default values.
255
256 >>> param.set_to_default_values()
257
258 Parse a string of options.
259
260 >>> param.parse_options('training_options')
261
262 Show values of parameters.
263
264 >>> print(param)
265
266- class svm_model:
267
268 There are two ways to obtain an instance of svm_model:
269
270 >>> model = svm_train(y, x)
271 >>> model = svm_load_model('model_file_name')
272
273 Note that the returned structure of interface functions
274 libsvm.svm_train and libsvm.svm_load_model is a ctypes pointer of
275 svm_model, which is different from the svm_model object returned
276 by svm_train and svm_load_model in svmutil.py. We provide a
277 function toPyModel for the conversion:
278
279 >>> model_ptr = libsvm.svm_train(prob, param)
280 >>> model = toPyModel(model_ptr)
281
282 If you obtain a model in a way other than the above approaches,
283 handle it carefully to avoid memory leak or segmentation fault.
284
285 Some interface functions to access LIBSVM models are wrapped as
286 members of the class svm_model:
287
288 >>> svm_type = model.get_svm_type()
289 >>> nr_class = model.get_nr_class()
290 >>> svr_probability = model.get_svr_probability()
291 >>> class_labels = model.get_labels()
292 >>> sv_indices = model.get_sv_indices()
293 >>> nr_sv = model.get_nr_sv()
294 >>> is_prob_model = model.is_probability_model()
295 >>> support_vector_coefficients = model.get_sv_coef()
296 >>> support_vectors = model.get_SV()
297
298Utility Functions
299=================
300
301To use utility functions, type
302
303 >>> from svmutil import *
304
305The above command loads
306 svm_train() : train an SVM model
307 svm_predict() : predict testing data
308 svm_read_problem() : read the data from a LIBSVM-format file.
309 svm_load_model() : load a LIBSVM model.
310 svm_save_model() : save model to a file.
311 evaluations() : evaluate prediction results.
312 csr_find_scale_param() : find scaling parameter for data in csr format.
313 csr_scale() : apply data scaling to data in csr format.
314
315- Function: svm_train
316
317 There are three ways to call svm_train()
318
319 >>> model = svm_train(y, x [, 'training_options'])
320 >>> model = svm_train(prob [, 'training_options'])
321 >>> model = svm_train(prob, param)
322
323 y: a list/tuple/ndarray of l training labels (type must be int/double).
324
325 x: 1. a list/tuple of l training instances. Feature vector of
326 each training instance is a list/tuple or dictionary.
327
328 2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
329
330 training_options: a string in the same form as that for LIBSVM command
331 mode.
332
333 prob: an svm_problem instance generated by calling
334 svm_problem(y, x).
335 For pre-computed kernel, you should use
336 svm_problem(y, x, isKernel=True)
337
338 param: an svm_parameter instance generated by calling
339 svm_parameter('training_options')
340
341 model: the returned svm_model instance. See svm.h for details of this
342 structure. If '-v' is specified, cross validation is
343 conducted and the returned model is just a scalar: cross-validation
344 accuracy for classification and mean-squared error for regression.
345
346 To train the same data many times with different
347 parameters, the second and the third ways should be faster..
348
349 Examples:
350
351 >>> y, x = svm_read_problem('../heart_scale')
352 >>> prob = svm_problem(y, x)
353 >>> param = svm_parameter('-s 3 -c 5 -h 0')
354 >>> m = svm_train(y, x, '-c 5')
355 >>> m = svm_train(prob, '-t 2 -c 5')
356 >>> m = svm_train(prob, param)
357 >>> CV_ACC = svm_train(y, x, '-v 3')
358
359- Function: svm_predict
360
361 To predict testing data with a model, use
362
363 >>> p_labs, p_acc, p_vals = svm_predict(y, x, model [,'predicting_options'])
364
365 y: a list/tuple/ndarray of l true labels (type must be int/double).
366 It is used for calculating the accuracy. Use [] if true labels are
367 unavailable.
368
369 x: 1. a list/tuple of l training instances. Feature vector of
370 each training instance is a list/tuple or dictionary.
371
372 2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
373
374 predicting_options: a string of predicting options in the same format as
375 that of LIBSVM.
376
377 model: an svm_model instance.
378
379 p_labels: a list of predicted labels
380
381 p_acc: a tuple including accuracy (for classification), mean
382 squared error, and squared correlation coefficient (for
383 regression).
384
385 p_vals: a list of decision values or probability estimates (if '-b 1'
386 is specified). If k is the number of classes in training data,
387 for decision values, each element includes results of predicting
388 k(k-1)/2 binary-class SVMs. For classification, k = 1 is a
389 special case. Decision value [+1] is returned for each testing
390 instance, instead of an empty list.
391 For probabilities, each element contains k values indicating
392 the probability that the testing instance is in each class.
393 Note that the order of classes is the same as the 'model.label'
394 field in the model structure.
395
396 Example:
397
398 >>> m = svm_train(y, x, '-c 5')
399 >>> p_labels, p_acc, p_vals = svm_predict(y, x, m)
400
401- Functions: svm_read_problem/svm_load_model/svm_save_model
402
403 See the usage by examples:
404
405 >>> y, x = svm_read_problem('data.txt')
406 >>> m = svm_load_model('model_file')
407 >>> svm_save_model('model_file', m)
408
409- Function: evaluations
410
411 Calculate some evaluations using the true values (ty) and the predicted
412 values (pv):
413
414 >>> (ACC, MSE, SCC) = evaluations(ty, pv, useScipy)
415
416 ty: a list/tuple/ndarray of true values.
417
418 pv: a list/tuple/ndarray of predicted values.
419
420 useScipy: convert ty, pv to ndarray, and use scipy functions to do the evaluation
421
422 ACC: accuracy.
423
424 MSE: mean squared error.
425
426 SCC: squared correlation coefficient.
427
428- Function: csr_find_scale_parameter/csr_scale
429
430 Scale data in csr format.
431
432 >>> param = csr_find_scale_param(x [, lower=l, upper=u])
433 >>> x = csr_scale(x, param)
434
435 x: a csr_matrix of data.
436
437 l: x scaling lower limit; default -1.
438
439 u: x scaling upper limit; default 1.
440
441 The scaling process is: x * diag(coef) + ones(l, 1) * offset'
442
443 param: a dictionary of scaling parameters, where param['coef'] = coef and param['offset'] = offset.
444
445 coef: a scipy array of scaling coefficients.
446
447 offset: a scipy array of scaling offsets.
448
449Additional Information
450======================
451
452This interface was written by Hsiang-Fu Yu from Department of Computer
453Science, National Taiwan University. If you find this tool useful, please
454cite LIBSVM as follows
455
456Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support
457vector machines. ACM Transactions on Intelligent Systems and
458Technology, 2:27:1--27:27, 2011. Software available at
459http://www.csie.ntu.edu.tw/~cjlin/libsvm
460
461For any question, please contact Chih-Jen Lin <cjlin@csie.ntu.edu.tw>,
462or check the FAQ page:
463
464http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html
465