1.. _about: 2 3About 4================================================================================ 5 6.. include:: ./workshop/includes/substitutions.rst 7 8What is PDAL? 9-------------------------------------------------------------------------------- 10 11|PDAL| is Point Data Abstraction Library. It is a C/C++ open source library 12and applications for translating and processing `point cloud data`_. It is not 13limited to |LiDAR| data, although the focus and impetus for many of the 14tools in the library have their origins in LiDAR. 15 16.. _`point cloud data`: https://en.wikipedia.org/wiki/Point_cloud 17 18What is its big idea? 19-------------------------------------------------------------------------------- 20 21PDAL allows you to compose :ref:`operations <filters>` on point clouds into 22:ref:`pipelines <pipeline>` of stages. These pipelines can 23be written in a declarative JSON syntax or constructed using the available API. 24 25Why would you want to do that? 26................................................................................ 27 28A task might be to load some `ASPRS LAS`_ (the most common LiDAR binary format) 29data into a database, but you wanted to transform it into a common coordinate 30system along the way. 31 32One option would be to write a specialized monolithic 33program that reads LAS data, reprojects it as necessary, and then handles the 34necessary operations to insert the data in the appropriate format in the 35database. This approach has a distinct disadvantage in that without careful 36planning it could quickly spiral out of control as you add new little tweaks 37and features to the operation. It ends up being very specific, and it 38does not allow you to easily reuse the component that reads the LAS data 39separately from the component that transforms the data. 40 41The PDAL approach is to chain together a set of components, 42each of which encapsulates specific functionality. The components allow for 43reuse, composition, and separation of concerns. PDAL views point cloud 44processing operations as a pipeline composed as a series of stages. You might 45have a simple pipeline composed of a :ref:`LAS Reader <readers.las>` stage, a 46:ref:`Reprojection <filters.reprojection>` stage, and a :ref:`PostgreSQL Writer 47<writers.pgpointcloud>`, for example. Rather than writing a single, monolithic 48specialized program to perform this operation, you can dynamically compose it 49as a sequence of steps or operations. 50 51.. figure:: ./images/las-reproject-pgpointcloud.png 52 53 A simple PDAL pipeline composed of a reader, filter, and writer 54 stages. 55 56A PDAL JSON :ref:`pipeline` that composes this operation to reproject 57and load the data into PostgreSQL might look something like the following: 58 59.. code-block:: json 60 :linenos: 61 :emphasize-lines: 4, 8, 12 62 63 { 64 "pipeline":[ 65 { 66 "type":"readers.las", 67 "filename":"input.las" 68 }, 69 { 70 "type":"filters.reprojection", 71 "out_srs":"EPSG:3857" 72 }, 73 { 74 "type":"writers.pgpointcloud", 75 "connection":"host='localhost' dbname='lidar' user='hobu'", 76 "table":"output", 77 "srid":"3857" 78 } 79 ] 80 } 81 82PDAL can compose intermediate stages for operations such as filtering, 83clipping, tiling, transforming into a processing pipeline and reuse as 84necessary. It allows you to define these pipelines as `JSON`_, and it 85provides a command, :ref:`pipeline_command`, to allow you to execute them. 86 87.. note:: 88 89 Raster processing tools often compose operations with this approach. 90 PDAL conceptually steals its pipeline modeling from |GDAL|'s 91 `Virtual Raster Format`_. 92 93.. _`Virtual Raster Format`: http://www.gdal.org/gdal_vrttut.html 94.. _`JSON`: https://en.wikipedia.org/wiki/JSON 95 96 97How is it different than other tools? 98-------------------------------------------------------------------------------- 99 100LAStools 101................................................................................ 102 103.. index:: LAStools 104 105One of the most common open source processing tool suites available for LiDAR 106processing is `LAStools`_ from `Martin Isenburg`_. PDAL is different in 107philosophy in a number of important ways: 108 1091. All components of PDAL are released as open source software under an 110 `OSI`_-approved license. 1112. PDAL allows application developers to provide proprietary extensions that 112 act as stages in processing pipelines. These might be things like custom 113 format 114 readers, specialized exploitation algorithms, or entire processing pipelines. 1153. PDAL can operate on point cloud data of any format 116 -- not just `ASPRS LAS`_. `LAStools`_ can read and write formats other than 117 LAS, but relates all data to its internal handling of LAS data, limiting 118 it to :ref:`dimension <dimensions>` types provided by the LAS format. 1194. PDAL is coordinated by users with its declarative :ref:`JSON <pipeline>` 120 syntax. LAStools is coordinated by linking lots of small, specialized 121 command line utilities together with intricate arguments. 1225. PDAL is an open source project, with all of its development activities 123 available online at https://github.com/PDAL/PDAL 124 125.. _about_pcl: 126 127PCL 128................................................................................ 129 130.. index:: PCL 131 132`PCL`_ is a complementary, rather than substitute, open source software 133processing suite for point cloud data. The developer community of the PCL 134library is focused on algorithm development, robotic and computer vision, and 135real-time laser scanner processing. PDAL can read and write PCL's PCD format. 136 137Entwine 138................................................................................ 139 140.. index:: Entwine 141 142 143 `Entwine`_ is open source software from Hobu, Inc. that organizes massive 144 point cloud collections into streamable data services. These two software 145 projects allow province-scale LiDAR collections to be organized and served via 146 HTTP clients over the internet. PDAL provides :ref:`readers.ept` to allow 147 users to read data from those `Entwine Point Tile`_ collections that Entwine 148 produces.. 149 150.. _`Entwine Point Tile`: https://entwine.io/entwine-point-tile.html 151.. _`Hobu, Inc.`: https://hobu.co 152 153.. _`Entwine`: https://entwine.io 154.. _`Greyhound`: http://greyhound.io 155 156Potree 157................................................................................ 158 159`Potree`_ is a `WebGL`_ HTML5 point cloud renderer that speaks `ASPRS LAS`_ and 160`LASzip`_ compressed LAS. You can find the software at 161https://github.com/potree/potree/ 162 163.. note:: 164 165 See Potree in action using the USGS 3DEP AWS Public Dataset at 166 https://usgs.entwine.io 167 168.. _`WebGL`: https://en.wikipedia.org/wiki/WebGL 169.. _`Potree`: http://potree.org 170.. _`LASzip`: http://laszip.org 171 172Others 173................................................................................ 174 175.. index:: OrfeoToolbox, libLAS, CloudCompare, Fusion 176 177Other open source point cloud softwares tend to be Desktop GUI, rather than 178library, focused. They include some processing operations, and sometimes they 179even embed tools such as PDAL. We're obviously biased toward PDAL, but you 180might find useful bits of functionality in them. These other tools include: 181 182* `libLAS`_ 183* `CloudCompare`_ 184* `Fusion`_ 185* `OrfeoToolbox`_ 186 187.. _`PCL`: http://pointclouds.org 188.. _`Fusion`: http://www.idaholidar.org/tools/fusion-ldv/ 189.. _`OrfeoToolbox`: https://www.orfeo-toolbox.org/ 190.. _`CloudCompare`: http://www.danielgm.net/cc/ 191.. _`libLAS`: http://liblas.org 192 193.. note:: 194 195 The `libLAS`_ project is an open source project that predates PDAL, and 196 provides some of the processing capabilities provided by PDAL. It is 197 currently in maintenance mode due to its dependence on LAS, the release of 198 relevant LAStools capabilities as open source, and the completion of 199 `Python LAS`_ software. 200 201.. _`Python LAS`: https://pypi.python.org/pypi/laspy/1.4.1 202.. _`ASPRS LAS`: http://www.asprs.org/Committee-General/LASer-LAS-File-Format-Exchange-Activities.html 203.. _`OSI`: https://opensource.org/licenses 204 205.. _`LAStools`: http://lastools.org 206.. _`Martin Isenburg`: https://www.cs.unc.edu/~isenburg/ 207 208Where did PDAL come from? 209-------------------------------------------------------------------------------- 210 211PDAL takes its cue from another very popular open source project -- |GDAL|. 212GDAL is Geospatial Data Abstraction Library, and it is used throughout the geospatial 213software industry to provide translation and processing support for a variety 214of raster and vector formats. PDAL provides the same capability for point 215cloud data types. 216 217PDAL evolved out of the development of database storage and access capabilities 218for the |USACE| `CRREL <http://www.erdc.usace.army.mil/Media/Fact-Sheets/Fact-Sheet-Article-View/Article/476649/remote-sensinggeographic-information-systems-center/>`__ |GRiD| project. Functionality that was creeping into |libLAS| 219was pulled into a new library, and it was designed from the ground up to mimic 220successful extract, transform, and load libraries in the geospatial software 221domain. PDAL has steadily attracted more contributors as other software developers 222use it to provide point cloud data translation and processing capability to 223their software. 224 225 226How is point cloud data different than raster or vector geo data? 227................................................................................ 228 229Point cloud data are indeed very much like the typical vector point data type 230of which many geospatial practitioners are familiar, but their volume causes some 231significant challenges. Besides their `X`, `Y`, and `Z` locations, each point 232often has full attribute information of other things like `Intensity`, `Time`, 233`Red`, `Green`, and `Blue`. 234 235Typical vector coverages of point data might max out at a million or so 236features. Point clouds quickly get into the billions and even trillions, and 237because of this specialized processing and management techniques must be used 238to handle so much data efficiently. 239 240The algorithms used to extract and exploit point cloud data are also significantly 241different than typical vector GIS work flows, and data organization is extremely 242important to be able to efficiently leverage the available computing. These 243characteristics demand a library oriented toward these approaches and PDAL 244achieves it. 245 246.. note:: 247 248 Possible point cloud dimension types provided and supported by PDAL 249 can be found at :ref:`dimensions`. 250 251What tasks are PDAL good at? 252-------------------------------------------------------------------------------- 253 254 255PDAL is great at point cloud data translation work flows. It allows users to 256apply algorithms to data by providing an abstract API to the content -- freeing 257users from worrying about many data format issues. PDAL's format-free worry 258does come with a bit of overhead cost. In most cases this is not significant, 259but for specific processing work flows with specific data, specialized tools 260will certainly outperform it. 261 262In exchange for possible performance penalty or data model impedance, developers 263get the freedom to access data over an abstract API, a multitude of algorithms 264to apply to data within easy reach, and the most complete set of 265point cloud format drivers in the industry. PDAL also provides a straightforward 266command line, and it extends simple generic Python processing through Numpy. These 267features make it attractive to software developers, data managers, and 268scientists. 269 270What are PDAL's weak points? 271-------------------------------------------------------------------------------- 272 273PDAL doesn't provide a friendly GUI interface, it expects that you have the 274confidence to dig into the options of :ref:`filters`, :ref:`readers`, and 275:ref:`writers`. We sometimes forget that you don't always want to read source 276code to figure out how things work. PDAL is an open source project in active 277development, and because of that, we're always working to improve it. Please 278visit :ref:`community` to find out how you can participate if you are 279interested. The project is always looking for contribution, and the mailing 280list is the place to ask for help if you are stuck. 281 282High Level Overview 283-------------------------------------------------------------------------------- 284 285PDAL is first and foremost a software library. A successful software library 286must meet the needs of software developers who use it to provide its software 287capabilities to their own software. In addition to its use as a software 288library, PDAL provides some :ref:`command line applications <apps>` users can 289leverage to conveniently translate, filter, and process data with PDAL. 290Finally, PDAL provides |Python| support in the form of embedded operations 291and Python extensions. 292 293Core C++ Software Library 294................................................................................ 295 296PDAL provides a :ref:`C++ API <api>` software developers can use to provide 297point cloud processing capabilities in their own software. PDAL is 298cross-platform C++, and it can compile and run on Linux, OS X, and Windows. The 299best place to learn how to use PDAL's C++ API is the :ref:`test suite 300<pdal_test>` and its `source code 301<https://github.com/PDAL/PDAL/tree/master/test/unit>`__. 302 303.. seealso:: 304 305 PDAL :ref:`software <reading>` :ref:`development <writing>` 306 :ref:`tutorials <writing-reader>` have more information on how to 307 use the library from a software developer's perspective. 308 309Command Line Utilities 310................................................................................ 311 312.. index:: Command line, Apps 313 314PDAL provides a number of :ref:`applications <apps>` that allow users to 315coordinate and construct point cloud processing work flows. Some key tasks 316users can achieve with these applications include: 317 318* Print :ref:`info <info_command>` about a data set 319* Data :ref:`translation <translate_command>` from one point cloud format to 320 another 321* Application of exploitation algorithms 322 323 * Generate a DTM 324 * Remove noise 325 * Reproject from one coordinate system to another 326 * Classify points as :ref:`ground/not ground <ground_command>` 327 328* :ref:`Merge <merge_command>` or :ref:`split <split_command>` data 329* :ref:`Catalog <tindex_command>` collections of data 330 331.. note:: 332 333 The command line utilities are often simply :ref:`pipeline_command` and 334 :ref:`pipeline` collected into a convenient application. In 335 many cases you can replicate the functionality of an application 336 entirely within a single pipeline. 337 338Python API 339................................................................................ 340 341.. index:: Numpy, Python 342 343PDAL supports both embedding |Python| and extending with |Python|. These 344allow you to dynamically interact with point cloud data in a more 345comfortable and familiar language environment for geospatial practitioners. 346 347.. seealso:: 348 349 The :ref:`python` document contains information on how to 350 install and use the PDAL Python extension. 351 352.. _`Numpy`: http://www.numpy.org/ 353 354Julia Plugin 355................................................................................ 356 357.. index:: Julia 358 359PDAL supports embedding |Julia| filters. These allow you to dynamically interact 360with point cloud data in a more comfortable and familiar language environment 361for geospatial practitioners, while still maintaining high performance. 362 363Additionally the TypedTables.jl, RoamesGeometry.jl and AcceleratedArrays.jl libraries 364provide some very high-level interfaces for writing efficient filters. 365 366.. seealso:: 367 368 The github repo at https://github.com/cognitive-earth/PDAL-julia contains 369 a docker image, build instructions and some sample filters. 370 371 Documentation for the stage :ref:`filters.julia` 372 373 374Conclusion 375-------------------------------------------------------------------------------- 376 377PDAL is an open source project for translating, filtering, and processing 378point cloud data. It provides a C++ API, command line utilities, and Python 379extensions. There are many open source software projects for interacting 380with point cloud data, and PDAL's niche is in processing, translation, 381and automation. 382
