1======================================= 2SPARQL Endpoint interface to Python 3======================================= 4 5|Build Status| |PyPi version| 6 7About 8===== 9 10**SPARQLWrapper** is a simple Python wrapper around a `SPARQL <https://www.w3.org/TR/sparql11-overview/>`_ service to 11remotelly execute your queries. It helps in creating the query 12invokation and, possibly, convert the result into a more manageable 13format. 14 15Installation & Distribution 16=========================== 17 18You can install SPARQLWrapper from PyPi:: 19 20 $ pip install sparqlwrapper 21 22You can install SPARQLWrapper from GitHub:: 23 24 $ pip install git+https://github.com/rdflib/sparqlwrapper#egg=sparqlwrapper 25 26You can install SPARQLWrapper from Debian:: 27 28 $ sudo apt-get install python-sparqlwrapper 29 30.. note:: 31 32 Be aware that there could be a gap between the latest version of SPARQLWrapper 33 and the version available as Debian package. 34 35Also, the source code of the package can be downloaded 36in ``.zip`` and ``.tar.gz`` formats from `GitHub SPARQLWrapper releases <https://github.com/RDFLib/sparqlwrapper/releases>`_. 37Documentation is included in the distribution. 38 39 40How to use 41========== 42 43 44First steps 45----------- 46 47The simplest usage of this module looks as follows (using the default, ie, `XML return format <http://www.w3.org/TR/rdf-sparql-XMLres/>`_, and special URI for the 48SPARQL Service):: 49 50 from SPARQLWrapper import SPARQLWrapper 51 52 queryString = "SELECT * WHERE { ?s ?p ?o. }" 53 sparql = SPARQLWrapper("http://example.org/sparql") 54 55 sparql.setQuery(queryString) 56 57 try : 58 ret = sparql.query() 59 # ret is a stream with the results in XML, see <http://www.w3.org/TR/rdf-sparql-XMLres/> 60 except : 61 deal_with_the_exception() 62 63If ``SPARQLWrapper("http://example.org/sparql",returnFormat=SPARQLWrapper.JSON)`` was used, the result would be in 64`JSON format <http://www.w3.org/TR/rdf-sparql-json-res/>`_ instead of XML. 65 66 67SELECT example 68-------------- 69 70.. code:: python 71 72 from SPARQLWrapper import SPARQLWrapper, JSON 73 74 sparql = SPARQLWrapper("http://dbpedia.org/sparql") 75 sparql.setQuery(""" 76 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 77 SELECT ?label 78 WHERE { <http://dbpedia.org/resource/Asturias> rdfs:label ?label } 79 """) 80 sparql.setReturnFormat(JSON) 81 results = sparql.query().convert() 82 83 for result in results["results"]["bindings"]: 84 print(result["label"]["value"]) 85 86 print('---------------------------') 87 88 for result in results["results"]["bindings"]: 89 print('%s: %s' % (result["label"]["xml:lang"], result["label"]["value"])) 90 91ASK example 92----------- 93 94.. code:: python 95 96 from SPARQLWrapper import SPARQLWrapper, XML 97 98 sparql = SPARQLWrapper("http://dbpedia.org/sparql") 99 sparql.setQuery(""" 100 ASK WHERE { 101 <http://dbpedia.org/resource/Asturias> rdfs:label "Asturias"@es 102 } 103 """) 104 sparql.setReturnFormat(XML) 105 results = sparql.query().convert() 106 print(results.toxml()) 107 108CONSTRUCT example 109----------------- 110 111.. code:: python 112 113 from SPARQLWrapper import SPARQLWrapper, RDFXML 114 from rdflib import Graph 115 116 sparql = SPARQLWrapper("http://dbpedia.org/sparql") 117 118 sparql.setQuery(""" 119 PREFIX dbo: <http://dbpedia.org/ontology/> 120 PREFIX schema: <http://schema.org/> 121 122 CONSTRUCT { 123 ?lang a schema:Language ; 124 schema:alternateName ?iso6391Code . 125 } 126 WHERE { 127 ?lang a dbo:Language ; 128 dbo:iso6391Code ?iso6391Code . 129 FILTER (STRLEN(?iso6391Code)=2) # to filter out non-valid values 130 } 131 """) 132 133 sparql.setReturnFormat(RDFXML) 134 results = sparql.query().convert() 135 print(results.serialize(format='xml')) 136 137DESCRIBE example 138---------------- 139 140.. code:: python 141 142 from SPARQLWrapper import SPARQLWrapper, N3 143 from rdflib import Graph 144 145 sparql = SPARQLWrapper("http://dbpedia.org/sparql") 146 147 sparql.setQuery(""" 148 DESCRIBE <http://dbpedia.org/resource/Asturias> 149 """) 150 151 sparql.setReturnFormat(N3) 152 results = sparql.query().convert() 153 g = Graph() 154 g.parse(data=results, format="n3") 155 print(g.serialize(format='n3')) 156 157SPARQL UPDATE example 158--------------------- 159 160.. code:: python 161 162 from SPARQLWrapper import SPARQLWrapper, POST, DIGEST 163 164 sparql = SPARQLWrapper("https://example.org/sparql-auth") 165 166 sparql.setHTTPAuth(DIGEST) 167 sparql.setCredentials("login", "password") 168 sparql.setMethod(POST) 169 170 sparql.setQuery(""" 171 WITH <http://example.graph> 172 DELETE 173 { <http://dbpedia.org/resource/Asturias> rdfs:label "Asturies"@ast } 174 """) 175 176 results = sparql.query() 177 print results.response.read() 178 179SPARQLWrapper2 example 180---------------------- 181 182There is also a ``SPARQLWrapper2`` class that works with JSON SELECT 183results only and wraps the results to make processing of average queries 184a bit simpler. 185 186.. code:: python 187 188 from SPARQLWrapper import SPARQLWrapper2 189 190 sparql = SPARQLWrapper2("http://dbpedia.org/sparql") 191 sparql.setQuery(""" 192 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 193 SELECT ?label 194 WHERE { <http://dbpedia.org/resource/Asturias> rdfs:label ?label } 195 """) 196 197 for result in sparql.query().bindings: 198 print('%s: %s' % (result["label"].lang, result["label"].value)) 199 200Return formats 201------------------------ 202 203The expected return formats differs from the query type (``SELECT``, ``ASK``, ``CONSTRUCT``, ``DESCRIBE``...). 204 205.. note:: From the `SPARQL specification <https://www.w3.org/TR/sparql11-protocol/#query-success>`_, 206 *The response body of a successful query operation with a 2XX response is either:* 207 208 * ``SELECT`` and ``ASK``: a SPARQL Results Document in XML, JSON, or CSV/TSV format. 209 * ``DESCRIBE`` and ``CONSTRUCT``: an RDF graph serialized, for example, in the RDF/XML syntax, or an equivalent RDF graph serialization. 210 211The package, though it does not contain a full SPARQL parser, makes an attempt to determine the query type 212when the query is set. This should work in most of the cases (but there is a possibility to set this manually, in case something 213goes wrong). 214 215Automatic conversion of the results 216----------------------------------- 217 218To make processing somewhat easier, the package can do some conversions automatically from the return result. These are: 219 220* for XML, the `xml.dom.minidom <http://docs.python.org/library/xml.dom.minidom.html>`_ is used to convert the result stream into a ``Python representation of a DOM tree``. 221* for JSON, the `json <https://docs.python.org/library/json.html>`_ package to generate a ``Python dictionary``. Until version 1.3.1, the `simplejson <https://pypi.python.org/pypi/simplejson>`_ package was used. 222* for CSV or TSV, a simple ``string``. 223* For RDF/XML and JSON-LD, the `RDFLib <https://rdflib.readthedocs.io>`_ package is used to convert the result into a ``Graph`` instance. 224* For RDF Turtle/N3, a simple ``string``. 225 226 227There are two ways to generate this conversion: 228 229* use ``ret.convert()`` in the return result from ``sparql.query()`` in the code above 230* use ``sparql.queryAndConvert()`` to get the converted result right away if the intermediate stream is not used 231 232 233For example, in the code below:: 234 235 try : 236 sparql.setReturnFormat(SPARQLWrapper.JSON) 237 ret = sparql.query() 238 dict = ret.convert() 239 except: 240 deal_with_the_exception() 241 242 243the value of ``dict`` is a Python dictionary of the query result, based on the `SPARQL Query Results JSON Format <http://www.w3.org/TR/rdf-sparql-json-res/>`_. 244 245 246Partial interpretation of the results 247------------------------------------- 248 249A further help is to offer an extra, partial interpretation of the results, again to cover 250most of the practical use cases. 251Based on the `SPARQL Query Results JSON Format <http://www.w3.org/TR/rdf-sparql-json-res/>`_, the :class:`SPARQLWrapper.SmartWrapper.Bindings` class 252can perform some simple steps in decoding the JSON return results. If :class:`SPARQLWrapper.SmartWrapper.SPARQLWrapper2` 253is used instead of :class:`SPARQLWrapper.Wrapper.SPARQLWrapper`, this result format is generated. Note that this relies on a JSON format only, 254ie, it has to be checked whether the SPARQL service can return JSON or not. 255 256Here is a simple code that makes use of this feature:: 257 258 from SPARQLWrapper import SPARQLWrapper2 259 260 queryString = "SELECT ?subj ?prop WHERE { ?subj ?prop ?o. }" 261 262 sparql = SPARQLWrapper2("http://example.org/sparql") 263 264 sparql.setQuery(queryString) 265 try : 266 ret = sparql.query() 267 print ret.variables # this is an array consisting of "subj" and "prop" 268 for binding in ret.bindings : 269 # each binding is a dictionary. Let us just print the results 270 print "%s: %s (of type %s)" % ("s",binding[u"subj"].value,binding[u"subj"].type) 271 print "%s: %s (of type %s)" % ("p",binding[u"prop"].value,binding[u"prop"].type) 272 except: 273 deal_with_the_exception() 274 275To make this type of code even easier to realize, the ``[]`` and ``in`` operators are also implemented 276on the result of :class:`SPARQLWrapper.SmartWrapper.Bindings`. This can be used to check and find a particular binding (ie, particular row 277in the return value). This features becomes particularly useful when the ``OPTIONAL`` feature of SPARQL is used. For example:: 278 279 from SPARQLWrapper import SPARQLWrapper2 280 281 queryString = "SELECT ?subj ?o ?opt WHERE { ?subj <http://a.b.c> ?o. OPTIONAL { ?subj <http://d.e.f> ?opt }}" 282 283 sparql = SPARQLWrapper2("http://example.org/sparql") 284 285 sparql.setQuery(queryString) 286 try : 287 ret = sparql.query() 288 print ret.variables # this is an array consisting of "subj", "o", "opt" 289 if (u"subj",u"prop",u"opt") in ret : 290 # there is at least one binding covering the optional "opt", too 291 bindings = ret[u"subj",u"o",u"opt"] 292 # bindings is an array of dictionaries with the full bindings 293 for b in bindings : 294 subj = b[u"subj"].value 295 o = b[u"o"].value 296 opt = b[u"opt"].value 297 # do something nice with subj, o, and opt 298 # another way of accessing to values for a single variable: 299 # take all the bindings of the "subj" 300 subjbind = ret.getValues(u"subj") # an array of Value instances 301 ... 302 except: 303 deal_with_the_exception() 304 305 306GET or POST 307----------- 308 309By default, all SPARQL services are invoked using HTTP **GET** verb. However, 310**POST** might be useful if the size of the query 311extends a reasonable size; this can be set in the query instance. 312 313Note that some combination may not work yet with all SPARQL processors 314(e.g., there are implementations where **POST + JSON return** does not work). 315Hopefully, this problem will eventually disappear. 316 317 318Development 319=========== 320 321Requirements 322------------ 323 324The `RDFLib <https://rdflib.readthedocs.io>`_ package is used for RDF parsing. 325 326This package is imported in a lazy fashion, ie, only when needed. Ie, if the user never intends to use the 327RDF format, the RDFLib package is not imported and the user does not have to install it. 328 329Source code 330----------- 331 332The source distribution contains: 333 334- ``SPARQLWrapper``: the Python package. You should copy the directory 335 somewhere into your PYTHONPATH. Alternatively, you can also run 336 the distutils scripts: ``python setup.py install`` 337 338- ``test``: some unit and integrations tests. In order to run the tests 339 some packages have to be installed before. So please install the packages 340 listed in requirements.development.txt: 341 ``pip install -r requirements.development.txt`` 342 343- ``scripts``: some scripts to run the package against some SPARQL endpoints. 344 345- ``docs``: the documentation. 346 347- ``custom_fixers``: 2to3 custom_fixer in order to fix an issue with urllib2._opener. 348 349Community 350========= 351 352Community support is available through the developer's discussion group `rdflib-dev <http://groups.google.com/d/forum/rdflib-dev>`_. 353The `archives <http://sourceforge.net/mailarchive/forum.php?forum_name=sparql-wrapper-devel>`_. from the old mailing list are still available. 354 355Issues 356====== 357 358Please, `report any issue to github <https://github.com/RDFLib/sparqlwrapper/issues>`_. 359 360Documentation 361============= 362 363The `SPARQLWrapper documentation is available online <https://rdflib.github.io/sparqlwrapper/doc/latest/>`_. 364 365Other interesting documents are the latest `SPARQL 1.1 Specification (W3C Recommendation 21 March 2013) <https://www.w3.org/TR/sparql11-overview/>`_ 366and the initial `SPARQL Specification (W3C Recommendation 15 January 2008) <http://www.w3.org/TR/rdf-sparql-query/>`_. 367 368 369License 370======= 371 372The SPARQLWrapper package is licensed under `W3C license`_. 373 374.. _W3C license: https://www.w3.org/Consortium/Legal/2015/copyright-software-and-document 375 376 377Acknowledgement 378=============== 379 380The package was greatly inspired by `Lee Feigenbaum's similar package for Javascript <http://thefigtrees.net/lee/blog/2006/04/sparql_calendar_demo_a_sparql.html>`_. 381 382Developers involved: 383 384* Ivan Herman <http://www.ivan-herman.net> 385* Sergio Fernández <http://www.wikier.org> 386* Carlos Tejo Alonso <http://www.dayures.net> 387* Alexey Zakhlestin <https://indeyets.ru/> 388 389Organizations involved: 390 391* `World Wide Web Consortium <http://www.w3.org>`_ 392* `Salzburg Research <http://www.salzburgresearch.at>`_ 393* `Foundation CTIC <http://www.fundacionctic.org/>`_ 394 395 396.. |Build Status| image:: https://secure.travis-ci.org/RDFLib/sparqlwrapper.svg?branch=master 397 :target: https://travis-ci.org/RDFLib/sparqlwrapper 398.. |PyPi version| image:: https://badge.fury.io/py/SPARQLWrapper.svg 399 :target: https://pypi.python.org/pypi/SPARQLWrapper 400