README.md
1# libplacebo
2
3[](https://code.videolan.org/videolan/libplacebo/pipelines)
4[](https://code.videolan.org/videolan/libplacebo/-/jobs/artifacts/master/file/coverage/index.html?job=test-gpu)
5[](#backers)
6[](#sponsors)
7[](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=SFJUTMPSZEAHC)
8[](https://www.patreon.com/haasn)
9
10**libplacebo** is, in a nutshell, the core rendering algorithms and ideas of
11[mpv](https://mpv.io) rewritten as an independent library. As of today,
12libplacebo contains a large assortment of video processing shaders, focusing
13on both quality and performance. These include features such as the following:
14
15- High-quality, optimized **upscaling and downscaling** including support for
16 polar filters ("Jinc"), anti-aliasing, anti-ringing and gamma correct
17 scaling.
18- **Color management** and format conversions for a wide variety of HDR or
19 wide gamut color spaces. This includes support for ICC profiles, ITU-R
20 BT.1886 emulation, colorimetrically accurate clipping, custom 1D/3D LUTs,
21 scene-referred OOTFs (such as HLG), constant luminance formats including
22 ICtCp and a variety of film industry formats ranging from XYZ to Sony's
23 S-Log or Panasonic's V-Gamut.
24- Tunable **debanding** shader. This is based on flash3kyuu, expanded to
25 provide high quality by combining multiple debanding passes.
26- Dynamic **HDR tone mapping**, including shaders for real-time peak and
27 scene-change detection, chroma-preserving (luma-only) tone mapping,
28 highlight desaturation, dynamic exposure control and a variety of
29 industry-standard EETFs including BT.2390.
30- High performance AV1 **film grain synthesis**, allowing media players to
31 offload this part of AV1 decoding from the CPU to the GPU.
32- A **pluggable, extensible custom shader syntax**, equivalent to an improved
33 version of [mpv's `.hook`
34 syntax](https://mpv.io/manual/master/#options-glsl-shaders). This can be
35 used to arbitrarily extend the range of custom shaders to include popular
36 user shaders like RAVU, FSRCNNX, or Anime4K. See the [mpv wiki on user
37 scripts](https://github.com/mpv-player/mpv/wiki/User-Scripts#user-shaders)
38 for more information.
39
40Every attempt was made to provide these features at a **high level of
41abstraction**, taking away all the messy details of GPU programming, color
42spaces, obscure subsampling modes, image metadata manipulation, and so on.
43Expert-level functionality is packed into easy-to-use functions like
44`pl_frame_from_avframe` and `pl_render_image`.
45
46libplacebo currently supports Vulkan (including MoltenVK), OpenGL, and
47Direct3D 11. It contains backwards compatibility code for very old versions of
48GLSL down to GLES 2.0 and OpenGL 1.3 and very old Direct3D feature levels down
49to 9_1.
50
51### Examples
52
53This screenshot from the included [plplay demo program](./demos/plplay.c)
54highlights just some of the features supported by the libplacebo rendering
55code, all of which are adjustable dynamically during video playback.
56
57[<img src="./demos/plplay-screenshot.png" width="200" alt="plplay settings" />](./demos/plplay-screenshot.png)
58
59### History
60
61This project grew out of an interest to accomplish the following goals:
62
63- Clean up mpv's internal [RA](#tier-1-rendering-abstraction) API and make it
64 reusable for other projects, as a general high-level backend-agnostic
65 graphics API wrapper.
66- Provide a standard library of useful GPU-accelerated image processing
67 primitives based on GLSL, so projects like media players or browsers can use
68 them without incurring a heavy dependency on `libmpv`.
69- Rewrite core parts of mpv's GPU-accelerated video renderer on top of
70 redesigned abstractions, in order to modernize it and allow supporting more
71 features.
72
73It has since been adopted by [VLC](https://www.videolan.org/vlc/) as their
74optional Vulkan-based video output path, and is provided as a Vulkan-based
75video filter in the FFmpeg project.
76
77## API Overview
78
79The public API of libplacebo is currently split up into the following
80components, the header files (and documentation) for which are available
81inside the [`src/include/libplacebo`](src/include/libplacebo) directory. The
82API is available in different "tiers", representing levels of abstraction
83inside libplacebo. The APIs in higher tiers depend on those in lower tiers.
84Which tier is used by a user depends on how much power/control they want over
85the actual rendering. The low-level tiers are more suitable for big projects
86that need strong control over the entire rendering pipeline; whereas the
87high-level tiers are more suitable for smaller or simpler projects that want
88libplacebo to take care of everything.
89
90### Tier 0 (logging, raw math primitives)
91
92- `colorspace.h`: A collection of enums and structs for describing color
93 spaces, as well as a collection of helper functions for computing various
94 color space transformation matrices.
95- `common.h`: A collection of miscellaneous utility types and macros that are
96 shared among multiple subsystems. Usually does not need to be included
97 directly.
98- `log.h`: Logging subsystem.
99- `config.h`: Macros defining information about the way libplacebo was built,
100 including the version strings and compiled-in features/dependencies. Usually
101 does not need to be included directly. May be useful for feature tests.
102- `dither.h`: Some helper functions for generating various noise and dithering
103 matrices. Might be useful for somebody else.
104- `filters.h`: A collection of reusable reconstruction filter kernels, which
105 can be used for scaling. The generated weights arrays are semi-tailored to
106 the needs of libplacebo, but may be useful to somebody else regardless. Also
107 contains the structs needed to define a filter kernel for the purposes of
108 libplacebo's upscaling routines.
109
110The API functions in this tier are either used throughout the program
111(context, common etc.) or are low-level implementations of filter kernels,
112color space conversion logic etc.; which are entirely independent of GLSL
113and even the GPU in general.
114
115### Tier 1 (rendering abstraction)
116
117- `gpu.h`: Exports the GPU abstraction API used by libplacebo internally.
118- `swapchain.h`: Exports an API for wrapping platform-specific swapchains and
119 other display APIs. This is the API used to actually queue up rendered
120 frames for presentation (e.g. to a window or display device).
121- `vulkan.h`: GPU API implementation based on Vulkan.
122- `opengl.h`: GPU API implementation based on OpenGL.
123- `d3d11.h`: GPU API implementation based on Direct3D 11.
124- `dummy.h`: Dummy GPI API (interfaces with CPU only, no shader support)
125
126As part of the public API, libplacebo exports a middle-level abstraction for
127dealing with GPU objects and state. Basically, this is the API libplacebo uses
128internally to wrap OpenGL, Vulkan, Direct3D etc. into a single unifying API
129subset that abstracts away state, messy details, synchronization etc. into a
130fairly high-level API suitable for libplacebo's image processing tasks.
131
132It's made public both because it constitutes part of the public API of various
133image processing functions, but also in the hopes that it will be useful for
134other developers of GPU-accelerated image processing software.
135
136### Tier 2 (GLSL generating primitives)
137
138- `shaders.h`: The low-level interface to shader generation. This can be used
139 to generate GLSL stubs suitable for inclusion in other programs, as part of
140 larger shaders. For example, a program might use this interface to generate
141 a specialized tone-mapping function for performing color space conversions,
142 then call that from their own fragment shader code. This abstraction has an
143 optional dependency on `gpu.h`, but can also be used independently from it.
144
145In addition to this low-level interface, there are several available shader
146routines which libplacebo exports:
147
148- `shaders/av1.h`: Helper shaders for AV1 decoding, currently only implements
149 a film grain synthesis shader.
150- `shaders/colorspace.h`: Shader routines for decoding and transforming
151 colors, tone mapping, dithering, and so forth.
152- `shaders/custom.h`: Allows directly ingesting custom GLSL logic into the
153 `pl_shader` abstraction, either as bare GLSL or in [mpv .hook
154 format](https://mpv.io/manual/master/#options-glsl-shaders).
155- `shaders/icc.h`: Shader for ICC profile based color management.
156- `shaders/lut.h`: Code for applying arbitrary 1D/3D LUTs.
157- `shaders/sampling.h`: Shader routines for various algorithms that sample
158 from images, such as debanding and scaling.
159
160### Tier 3 (shader dispatch)
161
162- `dispatch.h`: A higher-level interface to the `pl_shader` system, based on
163 `gpu.h`. This dispatch mechanism generates+executes complete GLSL shaders,
164 subject to the constraints and limitations of the underlying GPU.
165
166This shader dispatch mechanism is designed to be combined with the shader
167processing routines exported by `shaders/*.h`, but takes care of the low-level
168translation of the resulting `pl_shader_res` objects into legal GLSL. It also
169takes care of resource binding, shader input placement, as well as shader
170caching and resource pooling; and makes sure all generated shaders have unique
171identifiers (so they can be freely merged together).
172
173### Tier 4 (high level renderer)
174
175- `renderer.h`: A high-level renderer which combines the shader primitives
176 and dispatch mechanism into a fully-fledged rendering pipeline that takes
177 raw texture data and transforms it into the desired output image.
178- `utils/frame_queue.h`: A high-level frame queuing abstraction. This API
179 can be used to interface with a decoder (or other source of frames), and
180 takes care of translating timestamped frames into a virtual stream of
181 presentation events suitable for use with `renderer.h`, including any extra
182 context required for frame interpolation (`pl_frame_mix`).
183- `utils/upload.h`: A high-level helper for uploading generic data in some
184 user-described format to a plane texture suitable for use with `renderer.h`.
185 These helpers essentially take care of picking/mapping a good image format
186 supported by the GPU. (Note: Eventually, this function will also support
187 on-CPU conversions to a different format where necessary, but for now, it
188 will just fail)
189- `utils/dav1d.h`: High level helper for translating between Dav1dPicture
190 and libplacebo's `pl_frame`. (Single header library)
191- `utils/libav.h`: High-level helpers for interoperation between
192 libplacebo and FFmpeg's libav* abstractions. (Single header library)
193
194This is the "primary" interface to libplacebo, and the one most users will be
195interested in. It takes care of internal details such as degrading to simpler
196algorithms depending on the hardware's capabilities, combining the correct
197sequence of colorspace transformations and shader passes in order to get the
198best overall image quality, and so forth.
199
200## Authors
201
202libplacebo was founded and primarily developed by Niklas Haas
203([@haasn](https://github.com/haasn)), but it would not be possible without the
204contributions of others. Special note also goes out to wm4, the developer of
205mpv, whose ideas helped shape the foundation of the shader dispatch system.
206This library also includes various excerpts from mpv, in particular the filter
207kernel code. For a full list of past contributors to mpv, see the [mpv
208authorship page](https://github.com/mpv-player/mpv/graphs/contributors)
209
210[](https://github.com/haasn/libplacebo/graphs/contributors)
211
212### Backers
213
214Thank you to all our backers! �� [[Become a backer](https://opencollective.com/libplacebo#backer)]
215
216[](https://opencollective.com/libplacebo#backers)
217
218### Sponsors
219
220Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [[Become a sponsor](https://opencollective.com/libplacebo#sponsor)]
221
222[](https://opencollective.com/libplacebo/sponsor/0/website)
223[](https://opencollective.com/libplacebo/sponsor/1/website)
224[](https://opencollective.com/libplacebo/sponsor/2/website)
225[](https://opencollective.com/libplacebo/sponsor/3/website)
226[](https://opencollective.com/libplacebo/sponsor/4/website)
227[](https://opencollective.com/libplacebo/sponsor/5/website)
228[](https://opencollective.com/libplacebo/sponsor/6/website)
229[](https://opencollective.com/libplacebo/sponsor/7/website)
230[](https://opencollective.com/libplacebo/sponsor/8/website)
231[](https://opencollective.com/libplacebo/sponsor/9/website)
232
233### License
234
235libplacebo is currently available under the terms of the LGPLv2.1 (or later)
236license. However, it's possible to release it under a more permissive license
237(e.g. BSD2) if a use case emerges.
238
239Please open an issue if you have a use case for a BSD2-licensed libplacebo.
240
241## Installing
242
243### Gentoo
244
245An ebuild is available as `media-libs/libplacebo` in the gentoo repository.
246
247### Building from source
248
249libplacebo is built using the [meson build system](http://mesonbuild.com/).
250You can build the project using the following steps:
251
252```bash
253$ DIR=./build
254$ meson $DIR
255$ ninja -C$DIR
256```
257
258To rebuild the project on changes, re-run `ninja -Cbuild`. If you wish to
259install the build products to the configured prefix (typically `/usr/local/`),
260you can run `ninja -Cbuild install`. Note that this is normally ill-advised
261except for developers who know what they're doing. Regular users should rely
262on distro packages.
263
264### Dependencies
265
266In principle, libplacebo has no mandatory dependencies - only optional ones.
267However, to get a useful version of libplacebo. you most likely want to build
268with support for either `opengl`, `vulkan` or `d3d11`. libplacebo built without
269these can still be used (e.g. to generate GLSL shaders such as the ones used in
270VLC), but the usefulness is severely impacted since most components will be
271missing, impaired or otherwise not functional.
272
273A full list of optional dependencies each feature requires:
274
275- **glslang**: `glslang` + its related libraries (e.g. `libSPIRV.so`)
276- **lcms**: `liblcms2`
277- **opengl**: `libepoxy`
278- **shaderc**: `libshaderc`
279- **vulkan**: `libvulkan`, `python3-mako`
280
281#### Vulkan support
282
283Because the vulkan backend requires on code generation at compile time,
284`python3-mako` is a hard dependency of the build system. In addition to this,
285the path to the Vulkan registry (`vk.xml`) must be locatable, ideally by
286explicitly providing it via the `-Dvulkan-registry=/path/to/vk.xml` option,
287unless it can be found in one of the built-in hard-coded locations.
288
289### Configuring
290
291To get a list of configuration options supported by libplacebo, after running
292`meson $DIR` you can run `meson configure $DIR`, e.g.:
293
294```bash
295$ meson $DIR
296$ meson configure $DIR
297```
298
299If you want to disable a component, for example Vulkan support, you can
300explicitly set it to `false`, i.e.:
301
302```bash
303$ meson configure $DIR -Dvulkan=disabled -Dshaderc=disabled
304$ ninja -C$DIR
305```
306
307### Testing
308
309To enable building and executing the tests, you need to build with
310`tests` enabled, i.e.:
311
312```bash
313$ meson configure $DIR -Dtests=true
314$ ninja -C$DIR test
315```
316
317### Benchmarking
318
319A naive benchmark suite is provided as an extra test case, disabled by default
320(due to the high execution time required). To enable it, use the `bench`
321option:
322
323```bash
324$ meson configure $DIR -Dbench=true
325$ meson test -C$DIR benchmark --verbose
326```
327
328## Using
329
330For a full documentation of the API, refer to the above [API
331Overview](#api-overview) as well as the [public header
332files](src/include/libplacebo). You can find additional examples of how to use
333the various components in the [demo programs](demos) as well as in the [unit
334tests](src/tests).
335
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}