README.md
1<div align="center">
2 <img src="https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/blob/master/src/doc/Figures/osl-short.png" width=256 height=128>
3
4 <H1>Open Shading Language</H1>
5</div>
6
7[![Open Shading Language Reel 2020](http://img.youtube.com/vi/nmwMz1YnGPA/0.jpg)](https://www.youtube.com/watch?v=nmwMz1YnGPA "Open Shading Language Reel 2020")
8
9[![Build Status](https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/workflows/CI/badge.svg)](https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/actions)
10[![License](https://img.shields.io/badge/license-BSD%203--Clause-blue.svg?style=flat-square)](https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/LICENSE.md)
11[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3061/badge)](https://bestpractices.coreinfrastructure.org/projects/3061)
12
13
14
15Introduction
16------------
17
18Welcome to Open Shading Language!
19
20Open Shading Language (OSL) is a small but rich language for
21programmable shading in advanced renderers and other applications, ideal
22for describing materials, lights, displacement, and pattern generation.
23
24OSL was originally developed by Sony Pictures Imageworks for use in its in-
25house renderer used for feature film animation and visual effects, released
26as open source so it could be used by other visual effects and animation
27studios and rendering software vendors. Now it's the de facto standard
28shading language for VFX and animated features, used across the industry in
29many commercial and studio- proprietary renderers. Because of this, the work
30on OSL received an Academy Award for Technical Achievement in 2017.
31
32OSL is robust and production-proven, and has been used in films as diverse
33as "The Amazing Spider-Man," "Hotel Transylvania," "Edge of Tomorrow", "Ant
34Man", "Finding Dory," and many more. OSL support is in most leading
35renderers used for high-end VFX and animation work. For a full list of films
36and products, see the [filmography](#where-osl-has-been-used).
37
38The OSL code is distributed under the
39["New/3-clause BSD" license](https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/LICENSE.md),
40and the documentation under the [Creative Commons Attribution 4.0 International
41License](https://creativecommons.org/licenses/by/4.0/). In short, you are
42free to use OSL in your own applications, whether they are free or
43commercial, open or proprietary, as well as to modify the OSL code and
44documentation as you desire, provided that you retain the original copyright
45notices as described in the license.
46
47
48How OSL is different
49--------------------
50
51OSL has syntax similar to C, as well as other shading languages.
52However, it is specifically designed for advanced rendering algorithms
53and has features such as radiance closures, BSDFs, and deferred ray
54tracing as first-class concepts.
55
56OSL has several unique characteristics not found in other shading
57languages (certainly not all together). Here are some things you will
58find are different in OSL compared to other languages:
59
60* Surface and volume shaders compute radiance closures, not final colors.
61
62 OSL's surface and volume shaders compute an explicit symbolic
63 description, called a "closure", of the way a surface or volume
64 scatters light, in units of radiance. These radiance closures may be
65 evaluated in particular directions, sampled to find important
66 directions, or saved for later evaluation and re-evaluation.
67 This new approach is ideal for a physically-based renderer that
68 supports ray tracing and global illumination.
69
70 In contrast, other shading languages usually compute just a surface
71 color as visible from a particular direction. These old shaders are
72 "black boxes" that a renderer can do little with but execute to find
73 this one piece of information (for example, there is no effective way
74 to discover from them which directions are important to sample).
75 Furthermore, the physical units of lights and surfaces are often
76 underspecified, making it very difficult to ensure that shaders are
77 behaving in a physically correct manner.
78
79* Surface and volume shaders do not loop over lights or shoot rays.
80
81 There are no "light loops" or explicitly traced illumination rays in
82 OSL surface shaders. Instead, surface shaders compute a radiance
83 closure describing how the surface scatters light, and a part of the
84 renderer called an "integrator" evaluates the closures for a
85 particular set of light sources and determines in which directions
86 rays should be traced. Effects that would ordinarily require explicit
87 ray tracing, such as reflection and refraction, are simply part of the
88 radiance closure and look like any other BSDF.
89
90 Advantages of this approach include that integration and sampling may
91 be batched or re-ordered to increase ray coherence; a "ray budget" can
92 be allocated to optimally sample the BSDF; the closures may be used by
93 for bidirectional ray tracing or Metropolis light transport; and the
94 closures may be rapidly re-evaluated with new lighting without having
95 to re-run the shaders.
96
97* Surface and light shaders are the same thing.
98
99 OSL does not have a separate kind of shader for light sources. Lights
100 are simply surfaces that are emissive, and all lights are area lights.
101
102* Transparency is just another kind of illumination.
103
104 You don't need to explicitly set transparency/opacity variables in the
105 shader. Transparency is just another way for light to interact with a
106 surface, and is included in the main radiance closure computed by a
107 surface shader.
108
109* Renderer outputs (AOV's) may be specified using "light path expressions."
110
111 Sometimes it is desirable to output images containing individual
112 lighting components such as specular, diffuse, reflection, individual
113 lights, etc. In other languages, this is usually accomplished by
114 adding a plethora of "output variables" to the shaders that collect
115 these individual quantities.
116
117 OSL shaders need not be cluttered with any code or output variables to
118 accomplish this. Instead, there is a regular-expression-based
119 notation for describing which light paths should contribute to which
120 outputs. This is all done on the renderer side (though supported by
121 the OSL implementation). If you desire a new output, there is no need
122 to modify the shaders at all; you only need to tell the renderer the
123 new light path expression.
124
125* Shaders are organized into networks.
126
127 OSL shaders are not monolithic, but rather can be organized into
128 networks of shaders (sometimes called a shader group, graph, or DAG),
129 with named outputs of some nodes being connected to named inputs of
130 other nodes within the network. These connections may be done
131 dynamically at render time, and do not affect compilation of
132 individual shader nodes. Furthermore, the individual nodes are
133 evaluated lazily, only when their outputs are "pulled" from the later
134 nodes that depend on them (shader writers may remain blissfully
135 unaware of these details, and write shaders as if everything is
136 evaluated normally).
137
138* Arbitrary derivatives without grids or extra shading points.
139
140 In OSL, you can take derivatives of any computed quantity in a shader,
141 and use arbitrary quantities as texture coordinates and expect correct
142 filtering. This does not require that shaded points be arranged in a
143 rectangular grid, or have any particular connectivity, or that any
144 "extra points" be shaded. This is because derivatives are not
145 computed by finite differences with neighboring points, but rather by
146 "automatic differentiation", computing partial differentials for the
147 variables that lead to derivatives, without any intervention required
148 by the shader writer.
149
150* OSL optimizes aggressively at render time
151
152 OSL uses the LLVM compiler framework to translate shader networks into
153 machine code on the fly (just in time, or "JIT"), and in the process
154 heavily optimizes shaders and networks with full knowledge of the
155 shader parameters and other runtime values that could not have been
156 known when the shaders were compiled from source code. As a result,
157 we are seeing our OSL shading networks execute 25% faster than the
158 equivalent shaders hand-crafted in C! (That's how our old shaders
159 worked in our renderer.)
160
161
162
163What OSL consists of
164--------------------
165
166The OSL open source distribution consists of the following components:
167
168* oslc, a standalone compiler that translates OSL source code into
169 an assembly-like intermediate code (in the form of .oso files).
170
171* liboslc, a library that implements the OSLCompiler class, which
172 contains the guts of the shader compiler, in case anybody needs to
173 embed it into other applications and does not desire for the compiler
174 to be a separate executable.
175
176* liboslquery, a library that implements the OSLQuery class, which
177 allows applications to query information about compiled shaders,
178 including a full list of its parameters, their types, and any metadata
179 associated with them.
180
181* oslinfo, a command-line program that uses liboslquery to print to the
182 console all the relevant information about a shader and its parameters.
183
184* liboslexec, a library that implements the ShadingSystem class, which
185 allows compiled shaders to be executed within an application.
186 Currently, it uses LLVM to JIT compile the shader bytecode to x86
187 instructions.
188
189* testshade, a program that lets you execute a shader (or connected
190 shader network) on a rectangular array of points, and save any of its
191 outputs as images. This allows for verification of shaders (and the
192 shading system) without needing to be integrated into a fully
193 functional renderer, and is the basis for most of our testsuite
194 verification. Along with testrender, testshade is a good example
195 of how to call the OSL libraries.
196
197* testrender, a tiny ray-tracing renderer that uses OSL for shading.
198 Features are very minimal (only spheres are permitted at this time)
199 and there has been no attention to performance, but it demonstrates how
200 the OSL libraries may be integrated into a working renderer, what
201 interfaces the renderer needs to supply, and how the BSDFs/radiance
202 closures should be evaluated and integrated (including with multiple
203 importance sampling).
204
205* A few sample shaders.
206
207* Documentation -- at this point consisting of the OSL language
208 specification (useful for shader writers), but in the future will have
209 detailed documentation about how to integrate the OSL libraries into
210 renderers.
211
212
213
214Where OSL has been used
215-----------------------
216
217*This list only contains films or products whose OSL use is stated or can be
218inferred from public sources, or that we've been told is ok to list here. If
219an OSL-using project is missing and it's not a secret, just email the OSL
220project leader or submit a PR with edits to this file.*
221
222Renderers and other tools with OSL support (in approximate order of
223adding OSL support):
224* Sony Pictures Imageworks: in-house "Arnold" renderer
225* [Blender/Cycles](https://docs.blender.org/manual/en/dev/render/shader_nodes/osl.html)
226* [Chaos Group: V-Ray](https://www.chaosgroup.com/)
227* [Pixar: PhotoRealistic RenderMan RIS](https://renderman.pixar.com)
228* [Isotropix: Clarisse](http://www.isotropix.com/clarisse)
229* [Autodesk Beast](http://www.autodesk.com/products/beast/overview)
230* [Appleseed](http://appleseedhq.net)
231* [Animal Logic: Glimpse renderer](https://www.fxguide.com/featured/a-glimpse-at-animal-logic/)
232* [Image Engine: Gaffer](http://www.gafferhq.org/) (for expressions and deformers)
233* [DNA Research: 3Delight](http://www.3delight.com/)
234* Ubisoft motion picture group's proprietary renderer
235* [Autodesk/SolidAngle: Arnold](https://www.solidangle.com/arnold/)
236* [Autodesk: 3DS Max 2019](https://help.autodesk.com/view/3DSMAX/2019/ENU/?guid=__developer_3ds_max_sdk_features_rendering_osl_html)
237
238Films using OSL (grouped by year of release date):
239* **(2012)**
240 Men in Black 3, The Amazing Spider-Man, Hotel Transylvania
241* **(2013)**
242 Oz the Great and Powerful, Smurfs 2, Cloudy With a Chance of Meatballs 2
243* **(2014)**
244 The Amazing Spider-Man 2, Blended, Edge of Tomorrow, 22 Jump Street,
245 Guardians of the Galaxy, Fury,
246 The Hunger Games: Mockingjay - Part 1, Exodus: Gods and Kings,
247 The Interview
248* **(2015)**
249 American Sniper,
250 Insurgent,
251 Avengers Age of Ultron,
252 Ant Man,
253 Pixels,
254 Mission Impossible: Rogue Nation,
255 Hotel Transylvania 2,
256 Bridge of Spies,
257 James Bond: Spectre,
258 The Hunger Games: Mockingjay - Part 2,
259 Concussion
260* **(2016)**
261 Allegiant,
262 Batman vs Superman: Dawn of Justice,
263 The Huntsman,
264 Angry Birds Movie,
265 Alice Through the Looking Glass,
266 Captain America: Civil War,
267 Finding Dory,
268 Piper,
269 Independence Day: Resurgence,
270 Ghostbusters,
271 Star Trek Beyond,
272 Suicide Squad,
273 Kingsglaive: Final Fantasy XV,
274 Storks,
275 Miss Peregrine's Home for Peculiar Children,
276 Fantastic Beasts and Where to Find Them,
277 Assassin's Creed
278* **(2017)**
279 Lego Batman,
280 The Great Wall,
281 A Cure for Wellness,
282 Logan,
283 Power Rangers,
284 Life,
285 Smurfs: The Lost Village,
286 The Fate of the Furious,
287 Alien Covenant,
288 Guardians of the Galaxy 2,
289 The Mummy,
290 Wonder Woman,
291 Cars 3,
292 Baby Driver,
293 Spider-Man: Homecoming,
294 Dunkirk,
295 The Emoji Movie,
296 Detroit,
297 Kingsman: The Golden Circle,
298 Lego Ninjago Movie,
299 Blade Runner 2049,
300 Geostorm,
301 Coco,
302 Justice League,
303 Thor: Ragnarok
304* **(2018)**
305 Peter Rabbit,
306 Black Panther,
307 Annnihilation,
308 Red Sparrow,
309 Pacific Rim Uprising,
310 Avengers Infinity War,
311 Deadpool 2,
312 Incredibles 2,
313 Jurassic World: Fallen Kingdom,
314 Hotel Transylvania 3: Summer Vacation,
315 Ant Man and the Wasp,
316 Skyscraper,
317 Mission Impossible: Fallout,
318 The Meg,
319 Kin,
320 Smallfoot,
321 Alpha,
322 Venom,
323 First Man,
324 Bad Times at the El Royale,
325 Fantastic Beasts: The Crimes of Grindelwald,
326 Bohemian Rhapsody,
327 Holmes and Watson,
328 Spider-Man: Into the Spider-Verse
329* **(2019)**
330 The Kid Who Would Be King,
331 Alita: Battle Angel,
332 Lego Movie 2,
333 Lucky 13 (Love, Death, and Robots),
334 Captain Marvel,
335 Triple Frontier,
336 Avengers: Endgame,
337 Pokémon Detective Pikachu,
338 Godzilla: King of Monsters,
339 Rim of the World,
340 John Wick 3 Parabellum,
341 Men in Black International,
342 Toy Story 4,
343 Spider-Man: Far From Home,
344 Hobbs & Shaw,
345 Angry Birds 2,
346 The Art of Racing in the Rain,
347 Secret Life of Pets,
348 The Mandalorian (S1),
349 The Dark Crystal: Age of Resistance,
350 The King,
351 Jumanji: The Next Level,
352 Richard Jewell,
353 Game of Thrones (S8),
354 Lost in Space (S1),
355 Togo
356* **(2020)**
357 Underwater,
358 Birds of Prey,
359 Onward,
360 Bloodshot,
361 Greyhound,
362 The Old Guard,
363 Mulan,
364 Tenet,
365 The New Mutants,
366 Artemis Fowl,
367 The Eight Hundred,
368 Over the Moon,
369 Wonder Woman 1984,
370 Soul,
371 The Mandalorian (S2)
372* **(2021 / upcoming)**
373 Chaos Walking,
374 Peter Rabbit 2: The Runaway,
375 The Falcon and the Winder Soldier,
376 Secret Magic Control Agency,
377 The Mitchells vs the Machines,
378 ...
379
380
381Building and Installation
382-------------------------
383
384Please read the [INSTALL.md](INSTALL.md) file for detailed instructions on
385how to build and install OSL.
386
387
388Documentation
389-------------
390
391The OSL language specification can be found at
392[src/doc/osl-languagespec.pdf](src/doc/osl-languagespec.pdf) (in a source
393distribution) or in the share/doc/OSL/osl-languagespec.pdf file of an
394installed binary distribution.
395
396
397Contact & reporting problems
398----------------------------
399
400Simple "how do I...", "I'm having trouble", or "is this a bug" questions are
401best asked on the [osl-dev developer mail
402list](https://lists.aswf.io/g/osl-dev).
403That's where the most people will see it and potentially be able to answer
404your question quickly (more so than a GH "issue").
405
406Bugs, build problems, and discovered vulnerabilities that you are relatively
407certain is a legit problem in the code, and for which you can give clear
408instructions for how to reproduce, should be [reported as
409issues](https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/issues).
410
411If confidentiality precludes a public question or issue, you may contact the
412project administrator privately at [lg@imageworks.com](lg@imageworks.com),
413including for security-related issues.
414
415
416Contributing
417------------
418
419OSL welcomes code contributions, and nearly 50 people have done so over the
420years. We take code contributions via the usual GitHub pull request (PR)
421mechanism. Please see [CONTRIBUTING](CONTRIBUTING.md) for detailed
422instructions.
423
424
425Contacts, Links, and References
426-------------------------------
427
428[OSL GitHub page](https://github.com/AcademySoftwareFoundation/OpenShadingLanguage)
429
430[Read or subscribe to the OSL development mail list](https://lists.aswf.io/g/osl-dev)
431
432[Most recent PDF of the OSL language specification](https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/blob/master/src/doc/osl-languagespec.pdf
433)
434
435[OSL home page at SPI](http://openshadinglanguage.org)
436
437[Sony Pictures Imageworks main open source page](http://opensource.imageworks.com)
438
439
440
441Credits
442-------
443
444The current project leadership is documented in the
445[Governance](GOVERNANCE.md) file.
446
447Many people have contributed features, bug fixes, and other changes to OSL
448over the years: Steve Agland, Shane Ambler, Martijn Berger, Farchad
449Bidgolirad, Nicholas Bishop, Solomon Boulos, Stefan Bruens, Stefan Büttner,
450Matthaus G. Chajdas, Alejandro Conty, Thomas Dinges, Luke Emrose, Louis
451Feng, Mark Final, Henri Fousse, Syoyo Fujita, Tim Grant, Larry Gritz, Derek
452Haase, Sven-Hendrik Haase, John Haddon, Daniel Heckenberg, Thiago Ize, Matt
453Johnson, Ronan Keryell, Chris Kulla, Elvic Liang, Max Liani, Adam Martinez,
454John Mertic, Bastien Montagne, Steena Monteiro, Alexis Oblet, Erich Ocean,
455Mikko Ohtamaa, Jino Park, Alexei Pawlow, Jay Reynolds, Declan Russell,
456Patrick Scheibe, Alex Schworer, Jonathan Scruggs, Sergey Sharybin, Cliff
457Stein, Stephan Steinbach, Esteban Tovagliari, Brecht Van Lommel, Alexander
458von Knorring, Alex Wells, Roman Zulak. (Listed alphabetically; if we've left
459anybody out, it is inadvertent, please let us know.)
460
461We cannot possibly express sufficient gratitude to the managers at Sony
462Pictures Imageworks who allowed this project to proceed, supported it
463wholeheartedly, and permitted us to release the source, especially Rob
464Bredow, Brian Keeney, Barbara Ford, Rene Limberger, Erik Strauss, and Mike
465Ford.
466
467Huge thanks also go to the crack shading team at SPI, and the brave lookdev
468TDs and CG supes willing to use OSL on their shows. They served as our
469guinea pigs, inspiration, testers, and a fantastic source of feedback. And
470of course, the many engineers, TDs, and artists elsewhere who incorporated
471OSL into their products and pipelines, especially the early risk-takers at
472Chaos Group, Double Negative, Pixar, DNA, Isotropix, and Animal Logic. Thank
473you, and we hope we've been responsive to your needs.
474
475OSL was not developed in isolation. We owe a debt to the individuals
476and studios who patiently read early drafts of the language
477specification and gave us very helpful feedback and additional ideas,
478as well as to the continuing contributions and feedback of its current
479developers and users at other VFX and animation studios.
480
481The OSL implementation depends upon several other open source packages,
482all with compatible licenses:
483
484* [OpenImageIO](http://www.openimageio.org)
485* [Boost](http://www.boost.org)
486* [IlmBase](http://www.openexr.com)
487* [LLVM Compiler Infrastructure](http://llvm.org)
488
489OSL's documentation incorporates parts of [Markdeep](https://casual-effects.com/markdeep/)
490(c) 2015-2016, Morgan McGuire, and [highlight.js](https://highlightjs.org/)
491(c) 2006, Ivan Sagalaev, both distributed under BSD licenses.
492