Toonz Shader Fxs Manual

==========================================================

 1. Introduction

Toonz 7.1 allows users to write new Fxs using GLSL (the

OpenGL Shading Language).

Shader Fx interfaces are read once at Toonz's startup,

but the underlying fx algorithm can be modified in

real time to ease the fx creation process.

Users reading these notes for the first time may want to

refer to the official GLSL guide at:

  http://www.opengl.org/documentation/glsl/

Up-and-running examples of GLSL (fragment) shader programs

can be found at the GLSL sanbox gallery, from which some of

the provided examples are adapted from (requires a

WebGL-compatible web browser, such as Firefox or Google

Chrome):

  http://glsl.heroku.com/

Further examples can be found at the beautiful gallery at:

  https://www.shadertoy.com/

==========================================================

 2. Requirements

The most recent version of your graphics drivers, as well

as a fairly recent graphics card.

Specifically, graphics drivers must support OpenGL 2.1,

Transform Feedback and Pixel Buffers (either as a built-in

feature or through extensions).

==========================================================

 3. Limitations

Shader fxs are rendered on the GPU, meaning that they are

typically executed in a massively parallel fashion - ie fast.

However, since most systems only adopt one GPU, only one

Shader fx is allowed to be rendered at the same time.

This means that Shader Fxs do not take advantage of multiple

rendering threads in a Toonz rendering process like common

CPU-based fxs do.

Shader Fx are intended to apply a fragment shader on the

output surface for the fx. In other words, each output pixel

is processed separately using the supplied fragment shader

program.

This prevents the implementation of more complex output

patterns that span multiple pixels at the same time.

Furthermore, there is no way to specify intermediate buffer

objects to read or write data to - which is often a common

need when writing fxs.

==========================================================

 3. Implementing a Shader Fx

In order to implement a shader fx it's currently necessary

to either create or edit the following files:

  a. <toonz path="" stuff="">/config/current.txt</toonz>

	This file hosts the associations between fxs and

	their parameters and the names displayed in the GUI

	(which are not locale-dependent).

  b. <toonz path="" stuff="">/profiles/layouts/fxs/fxs.lst</toonz>

	The list of fxs as displayed in the right-click

	contextual menus like "Add Fx" or "Insert Fx"

  c. <toonz path="" stuff="">/profiles/layouts/fxs/<your fx="" name="" shader="">.xml</your></toonz>

	Parameters tabbing in the Fx Parameters Editor

  d. <toonz library="" path="">/shaders/<your fx="" name="" shader="">.xml</your></toonz>

	The Shader Fx interface.

  e. The actual shader program files

Please, observe that the paths and names outside brackets

are mandatory.

Apart from point (d) and (e) discussed separately, it is best

to locate existing entries and emulate their behavior.

You can typically find related entries by searching "Shader"

in each file.

==========================================================

 4. The Shader Interface File

The Shader Fx Interface file at (3.d) is an xml document that

defines the main properties of the fx.

Specifically:

  a. Shader program files to be compiled at run-time

  b. Input ports for the fx

  c. Parameters

  d. Restrictions to the class of world/output coordinates

     transforms handled by the fx

The file is read once when Toonz starts, so any modification

will not be recognized until Toonz is restarted.

The complete recognized file structure is as follows:

<mainprogram>                       // (4.a) The applied fragment shader</mainprogram>

  <name></name>

    SHADER_myShaderName             // Internal name of the fx (mandatory, a simple app-unique literal id)

  <programfile></programfile>

    "programs/myShader.frag"        // The shader program file (3.e), relative to the

                      // path of the interface file.

<inputports>                        // (4.b) - Only a *fixed* number of ports allowed</inputports>

  <inputport>                       // A first port</inputport>

    "Source"                        // The displayed port name

  <inputport>                       // Second port</inputport>

    "Control"

  <portsprogram>                    // (4.a) Vertex shader used to acquire the geometry of</portsprogram>

    <name>                          // input images. See (5.b).</name>

      SHADER_myShader_ports         // The unique id for the vertex shader program (mandatory)

    <programfile></programfile>

      "programs/myShader_ports.vert"

<bboxprogram>                       // (4.a) Vertex shader used to calculate the fx's bbox.</bboxprogram>

  <name>                            //  See (5.c).</name>

    SHADER_myShader_bbox

  <programfile></programfile>

    "programs/myShader_bbox.vert"

<handledworldtransforms>            // (4.d) Optional, see (5.a)</handledworldtransforms>

  isotropic                         // May be either 'any' (default) or 'isotropic'.

           // Isotropic transforms exclude shears and non-uniform scales.

<parameters>                        // (4.c)</parameters>

  <parameter></parameter>

    float radius                    // Parameter declaration

    <default>                       // Additional Parameter attributes (can be omitted)</default>

      10                            // The parameter default

    <range></range>

      0 20                          // The parameter range

    <concept></concept>

      length                        // The parameter concept type - or, how it is represented

                          // by the Toonz GUI

  <parameter></parameter>

    float angle

    <concept></concept>

      angle_ui                      // Concepts of type <concept type="">_ui are editable in</concept>

      <name>                        // camera stand</name>

        "My Angle"

<concept>                           // Composite parameter concepts can be formed by 2 or</concept>

                                    // more parameters

  polar_ui

  <name></name>

    "My Polar Coordinates"

  <parameter>                       // List of involved parameters</parameter>

    radius

  <parameter></parameter>

    angle

----------------------------------------------------------

4.1. Parameter Declarations

Parameters are introduced by a declaration typically matching

the corresponding GLSL variable declaration.

The complete recognized list of supported parameter types is:

  bool, float, vec2, int, ivec2, rgb, rgba

The 'rgb' and 'rgba' types map to GLSL 'vec3' and 'vec4'

variables respectively, but are displayed with the appropriate

color editors by Toonz - plus, the range of their components

automatically maps from [0, 255] in Toonz and the Shader

Interface file to [0.0, 1.0] in the corresponding shader program

files.

----------------------------------------------------------

4.2. Parameter Concepts

Parameter 'concepts' are additional parameter properties that

regard the way Toonz represents a certain parameter type.

For example, a 'float' variable type may either indicate

an angle, the length of a segment, a percentage value,

and more.

Fx writers may want to explicitly specify a parameter concept

for the following reasons:

  a. Impose a measure to the parameter (e.g. degrees, inches, %)

  b. Make the parameter editable in camera-stand

The complete list of supported parameter concepts is the following:

  percent            - Displayed with the percentage '%' unit

  length             - Displayed in length units (inches, mm, cm, etc..)

  angle              - Displayed in angular units 'ďż˝'

  point              - A vec2 displayed in length units

  radius_ui          - Like length, displaying a radius in camstand. May compose with a point (the center)

  width_ui           - Like length, displaying a vertical line width. May compose with the line's angle.

  angle_ui           - Like angle, displaying it in camstand

  point_ui           - Like point, in camstand

  xy_ui              - Composes two float types in a point

  vector_ui          - Composes two float types in an 'arrow'-like vector

  polar_ui           - Like vector_ui, from a length and an angle

  size_ui            - Displays a square indicating a size. May compose width and height in a rect.

  quad_ui            - Composes 4 points in a quadrilateral

  rect_ui            - Composes width, height, and the optional center point in a rect

==========================================================

 5. Shader program files

A shader program file is a simple text file containing the

actual algorithms of a shader fx.

In the current implementation of Toonz Shader Fxs, there are

3 possible shader program files that need to be specified:

  a. The main fragment shader program, responsible of

     executing the code that actually renders the fx

  b. An optional vertex shader program to calculate the 

     geometries of contents required from input ports

  c. An optional vertex shader program to calculate the

     bounding box of the fx output

----------------------------------------------------------

 5.a. The 'MainProgram' Fragment Shader

The main program is in practice a standard GLSL fragment

shader - however, Toonz will provide it a set of additional

uniform input variables that must be addressed to correctly

compute the desired output.

The complete list of additional variables always supplied

by Toonz is:

  uniform mat3    worldToOutput;

  uniform mat3    outputToWorld;

These matrix variables describe the affine transforms mapping

output coordinates to Toonz's world coordinates, and vice-versa.

They include an additional coordinate as an OpenGL version-portable

way to perform translations by natural multiplication - transforming

a point is then done like:

  vec2 worldPoint = (outputToWorld * vec3(outPoint, 1.0)).xy

Fx parameters are typically intended in world coordinates,

and should be adjusted through these transforms - for example,

a camstand-displayed radius value must be multiplied by the

'worldToOutput' scale factors in order to get the corresponding

value in output coordinates.

World/Output transforms may be restricted to a specific sub-class

of affine transforms by specifying so in the Shader Interface File.

Restricting to isotropic transforms may be useful to simplify

cases where angular values are taken into account, since this

transforms class preserves angles by allowing only uniform scales,

rotations and translations. Non-uniform scales and shears are

later applied by Toonz on the produced fx output if necessary.

In case input ports have been specified, we also have:

  uniform sampler2D inputImage[n];

  uniform mat3      outputToInput[n];

  uniform mat3      inputToOutput[n];

The sampler variables correspond to the input content to the

fx. The matrix variables are the reference transforms from

output to input variables, and vice-versa.

Additional uniform variables corresponding to fx parameters

will also be supplied by Toonz. For example, if a "float radius"

parameter was specified, a corresponding

  uniform float radius;

input variable will be provided to the program.

WARNING: Toonz requires that *output* colors must be 

         'premultiplied' - that is, common RGB components

         (in the range [0, 1]) must be stored multiplied

         by their alpha component.

----------------------------------------------------------

 5.b. The optional 'PortsProgram' Vertex Shader

The shader program (b) is required in case an fx specifies

input ports, AND it needs to calculate some input content

in a different region than the required output.

It can be neglected otherwise.

For example, a blur fx requires that input contents outside

the required output rectangle are 'blurred in' it.

The 'PortsProgram' vertex shader is a one-shot shader

run by Toonz on a single dummy vertex - which uses

OpenGL 3.0's "Transform Feedback" extension to return a

set of predefined 'varying' output variables

The complete set of variables supplied by Toonz and required

in output by the program is:

  uniform mat3    worldToOutput;

  uniform mat3    outputToWorld;

  uniform vec4    outputRect;

  varying vec4    inputRect[portsCount];

  varying vec4    worldToInput[portsCount];

The transforms are intended in the same way as (5.a).

The outputRect and inputRect[] variables store the

(left, bottom, right, top) rect components in output

and input coordinates respectively.

Parameter input variables are obviously also supplied.

WARNING: *All* the required output variables must be

	 declared AND filled with values.

         There is no recognized default for them, and the

         fx will (silently) fail to render if some are not

         assigned.

----------------------------------------------------------

 5.c. The optional 'BBoxProgram' Vertex Shader

Some fx may be able to restrict their opaque renderable

area inside a rect.

For example, blurring an image will 'blur out' the image

content by the specified blur radius. Beyond that, the fx

will render full transparent pixels. Thus, the bounding

box of the fx in this case will be calculated as the

input bounding box, enlarged by the blur radius.

The default output bounding box is assumed to be infinite;

if that is the case, the BBoxProgram can be omitted.

Fx writers may want to supply an explicit program to

calculate the bounding box of the fx, given its input

bounding boxes. This is be useful in Toonz's rendering

pipeline because the software is then allowed to

restrict memory allocation (and fxs calculations)

for the output image to said output bounding box, resulting

in less memory consumption and increased speed.

The complete set of variables supplied by Toonz and required

in output by the program is:

  uniform vec4 infiniteRect;

  uniform vec4 inputBBox[portsCount];

  varying vec4 outputBBox;

The infiniteRect variable should be used to identify both

input and output infinite bboxes.