Curv 0.6: Reactive values work in all circumstances where they should. Relies on new compiler/partial evaluator, new type system. Update Array documentation: definition of ranked array, ranked function. * `list[i1,i2,...]` support reactive args * `list[i1,i2,...] := ...` in full generality * design a general set of adverbs for pipeline programming * Add each_left and each_right from K. What names do I choose? * '`foo` y' is equivalent to 'x->foo[x,y]', as a form of partial application. Use cases: A>>`foo`B, map(`foo`B), etc. * Infix `foo` should have precedence, so that A `mod` B == 0 works. Use A>>`union`B in a shape pipeline instead of A`union`B. * Prefix `foo` must have higher precedence than (for pipelines), lower precedence than , higher precedence than (for consistency with infix having precedence ). Best option is , which contains the other high precedence prefix operators. * I would also like postfix `foo`, also for partial application. Precedence? It's probably ' `foo`'. All builtin numeric/boolean operations are now vectorized. Document this. Call_Expr -- could vectorize the function argument. Q'NIAL, FP All expression types supported by SubCurv also support reactive arguments. Expressions that don't support reactive arguments: select[a,b,c] dot[a,b] and[...] and or[...] implement custom reduction code for bool32 and bvec. How? * A functional Prim interface, returns an optional SC_Value. SC_Value sc_try_reduce(SC_Frame& f, SC_Value a, const Context& cx) SubCurv: sum[a,b,c,d] is t=[a,b]+[c,d];t[0]+t[1], not (a+b+c+d)? rename sc_vec_element -> sc_list_elem fix internals to support general struc list fix callers to support general struc list update sc_eval_index*_expr() functions to support all struc types sc index an array using a struc index value add Reactive_Value::at(i,cx), call from list_elem and list_at replace idchr() -- use is_C_identifier()? Array_Op and Prim should export the same interface. Remove boilerplate and duplication from PrimType classes. _Prim and _PrimType classes (different names for different signatures) migrate all array primitives with operator syntax to Prim migrate all array primitives with function syntax to Prim and[...] and or[...] implement custom reduction code for bool32 and bvec. How? 1. Define And_Function directly. Define ::sc_call_expr() to handle special cases first before invoking the general case. It calls sc_reduce, but this contains an if that handles a List_Expr arg before evaluating the arg and handling the value. It's at the value handling point that I want to insert code. * Maybe sc_reduce_expr calls sc_reduce_value. I want to override sc_reduce_value to execute my code then call the default version. 2. It is cleaner to define the reducer in And_Prim. * Maybe And_Prim inherits sc_reduce_value from Binary_Prim, then overrides it while calling the default version. Yeah, but the default sc_reduce_value needs to call Prim::sc_check_args and Prim::sc_call. That's recursive, so we need to tie the knot and use OOP + a vtable. * Or tie the knot by passing the default sc_reducer as a function argument to Prim::sc_reduce_value. * Maybe And_Prim defines sc_try_reduce, which defaults to do nothing, inherited from Binary_PrimType. * A procedural interface, returns true on success, mutates a SC_Value& r to return a result. bool sc_try_reduce(SC_Frame& f, SC_Value& a, const Context& cx) if (a.type.is_bool32()) a = ...; return true; return false; * A functional interface, returns an optional SC_Value. SC_Value sc_try_reduce(SC_Frame& f, SC_Value a, const Context& cx) if (a.type.is_bool32()) return ...; return {}; Soft Typing ----------- Assignment of types to reactive expressions uses "soft typing", where type conflicts produce an Error type and don't automatically cause the program to abort. sc_result_type computes a soft type for the reactive expression, it is not as restrictive as sc_check_args. Figure out the type system and then fix this. Here's a bbox value: [[x0,y0,z0],[x1,y1,z1]] * reactive_num It is given the type Array[2]Vec3. Notice this isn't a struc type. Binary_Num_SCMat_Prim::sc_result_type() is rigged to return non-struc types so that this case won't generate a compile error. Non-struc values aren't supported by SubCurv, but this case works out because it is a bbox value, and we aren't compiling this expression into GLSL. Standardize format of Prim error messages ----------------------------------------- Binary_Array_Op::domain_error: use Prim::name rather than dig into cx.syntax is_C_identifier in symbol.h + if (is_C_identifier(Prim::name())) { + return Exception(cx, + stringify(Prim::name(),"[",x,",",y,"]: invalid argument")); + } else { + return Exception(cx, + stringify(x," ",Prim::name()," ",y,": invalid argument")); Standardized error messages in Prim. For the same Prim, errors can occur: * in a Call_Expr function call expression * in a Unary_Prim_Expr or Binary_Prim_Expr (part of a reactive expr) ERROR: 2 + #blue: invalid arguments ERROR: max[2,#blue]: invalid argument pair ERROR: argument #1 of max: [2,#blue]: invalid argument pair * Errors produced by sc_check_args, from 2 SC_Values and a cx... what do they look like? In some cases I use At_Index(cx, 0 or 1) but does that work in all cases? * Errors resulting from sc_result_type() failures? Maybe there should be no errors. ISSUES: * rx->expr() makes more logical sense than rx->expr(ph). That's in curv5. The 'ph' argument is used when a Uniform Variable value is converted to an expression operand, but that isn't the same as the source location where the Uniform Variable was initially referenced by name. * list_elem() takes an At_Syntax& cx argument. Which affects upstream APIs. How is it used? What are the actual requirements? * syntax_ of Call_Expr: it's a call site, needs to be passed as argument. * syntax_ of Constant '[i]'. This should be provided by the caller, explaining what the index is, or where it was computed. * cx argument of Reactive_Expression constructor. It's used to throw an exception if Operation arg isn't 'pure', but that's an internal compiler error. Maybe we don't need this: call 'die' instead? * The List_Pattern::exec() call to list_elem(): * The result call_site: * The index call_site: * I think these are both satisfied by the syntax_ of the List_Pattern element being matched. So, list_elem(val,i,const Phrase& syntax). List_Pattern::exec() has an 'argcx' argument that describes the argument value to be decomposed as a list. Does this context have any relevance to the information we must pass to list_elem()? If an error is reported later, when the result of list_elem() is used as an argument, what information should that error/stack trace contain? * At_Index does not return an At_Syntax, affecting List_Pattern::exec(). A Reactive_Expression contains an Operation which has a syntax_ field. * The syntax_ is the call site from which the Operation was invoked: either a location in Curv source code (the analytic case), or a description of the call site in C++ native code (the synthetic case). * For an expression, the syntax_ shows where the value was computed, and the context in which it was being used. What if these are different? The value is computed, stored in a variable/data structure, then used in an argument context that throws an error? The site where the argument is computed is valuable information: nice if we can show that in error messages. * The purpose of syntax_ is to be included in a stack trace, so we can see where an error occurred. The only proper use is syntax_.location(), when a Phrase inside a Context is converted to a stack trace. * Due to the way that Reactive_Expressions are constructed, each Operation in the expression tree may have a call site that is disjoint from its parent or its children. (This doesn't happen in the analytic case.) * Therefore, we shouldn't make too many assumptions about syntax_ objects. We shouldn't assume they were constructed analytically. At present, Reactive_Value::expr(ph) requires a Phrase argument. Hence we need a Phrase during general expression evaluation. Ideas: 1. In Pattern::exec(Value*,Value,Context&cx,Frame&), change 'Context&cx' to 'At_Syntax&cx' so I can use cx.syntax(). Code impact: Initially it requires At_Index and At_Field to be derived from At_Syntax. 2. In Pattern::exec, add 'const Phrase& valph' argument. The underlying problem is that a Uniform_Variable value requires a Phrase for conversion to a Constant each time it is referenced in an arithmetic expression. * When will this Phrase be used? What is the best choice of Phrase? I think the Phrase could be used if an arithmetic expression fails to compile in the Shape Compiler, in which case the Phrase points to the bad operand. But this shouldn't happen, we do type checking right now when we construct a reactive expression. It seems like the Phrase won't be used. 3. Store the parameter name Identifier phrase in the Uniform_Variable when it is constructed. Looks practical, changes are local. Operations on abstract list values (Lists and Reactive_Values). bool is_list(val) -- already called islist(val) in math.h unsigned list_count(val) Value list_elem(val, unsigned i, const Context& cx) Value list_elem_unsafe(val, unsigned i) No virtual functions (for cache locality) in the fast case of a List. This should be faster than virtual functions in Ref_Value. list_elem(val,i) may construct an Operation, if the list is reactive. What syntax object(s) do I use? The syntax_ member of an Operation denotes the call site for that operation, either in Curv source code, or in C++ native code. Consider this API: list_elem(val,i,const At_Syntax& cx) where 'cx' is the call site for the 'val[i]' expression. If val is a reactive list, then we need to synthesize a call site for the index 'i' as well. Old answer: Synthetic operation nodes are generated by Curv primitives written in C++. Their syntax object encodes the name of this primitive. `Synthetic_Phrase` is a subclass of Phrase: so you can test if a Meaning is synthetic. Why is this okay? In a reactive value, the source location of each node can potentially be different and disconnected from the locations of the parent and children. Who will access these synthetic phrases? Is this just a placeholder to avoid storing a null pointer, or will it be used? What about errors and stack traces? Using the `time` global reactive variable, you can evaluate a reactive value in the interpreter, substituting a time value. Even though the reactive operation tree is type checked, run time errors can still occur. Eg, 0/0 throws an error. A reactive value could contain the subexpression `time/0`, or `(time-1)/0`. What does the error and stack trace look like? ERROR: 0 / 0: domain error at reactive value: 1| (time-1)/0 at file "foo": 42| ...... With the above design, reactive values don't need source locations. But synthetic phrases might still be useful in the debugger? Perhaps Value::list_elem and List_Pattern::match are native primitives that should each have their own Frame? Or, their own VM opcode? Who will call Value::list_elem(), and what does the op tree look like? List_Pattern::exec(). The argument A being matched is a reactive list, and we have an At_Syntax for the argument expression. The pattern is `[a,b]`, so `a` is bound to `A[0]`. What is the syntax object for `A[0]`? What is the syntax object for `[0]` and `0`? The syntax_ member of an Operation specifies where the Operation is being called from. If Value::list_elem is a primitive Operation, called from List_Pattern::match, then the former Op needs a syntax_ describing the call site within the latter. Therefore, Value::list_elem(i) needs an additional argument that describes the caller, from which a Phrase can be derived. I figure this additional argument has type At_Syntax or Frame. Value::list_elem(i) may construct an Operation, if the list is reactive. What syntax object do I use? 1. Synthetic operation nodes encode the name of the C++ function that generates them in the Phrase object. `Synthetic_Phrase` is a subclass of Phrase: so you can test if a Meaning is synthetic. static auto origin = make(__func__); make(origin, ...) 2. API is list_elem(unsigned, At_Syntax). 3. Copy the syntax from the reactive list. In what circumstances is a reactive value's syntax object used? * In the design for the `time` global reactive variable, you can evaluate a reactive value in the interpreter, substituting a time value. Even though the reactive operation tree is type checked, run time errors can still occur. Eg, 0/0 throws an error. A reactive value could contain the subexpression `time/0`, or `(time-1)/0`. What does the error and stack trace look like? ERROR: 0 / 0: domain error at reactive value: 1| (time-1)/0 at file "foo": 42| ...... With the above design, reactive values don't need source locations. * In a reactive value, the source location of each node can potentially be different and disconnected from the locations of the parent and children. Some of the nodes are synthetic. For these, we could encode the name of the C++ function (__func__) that generates them, in the Phrase object. * Who will call Value::list_elem(), and what At_Syntax info is available? * List_Pattern::exec(). The argument A being matched is a reactive list, and we have an At_Syntax for the argument expression. The pattern is `[a,b]`, so `a` is bound to `A[0]`. What is the syntax object for `A[0]`? What is the syntax object for `[0]` and `0`? * Support reactive values in Binary_Array_Op using Infix_Op_Expr. * This will require defining Add_Expr = Infix_Op_Expr. Same class is used everywhere, so that + expressions hash consistently. * So now, Binary_Array_Op::reactive_op hard codes Infix_Op_Expr as the expression form of the binary add operation. * Replace add(a,b,cx) with Add_Op::op(a,b,cx). * Replace Add_Expr with Infix_Op_Expr Add Operation::print(), for printing function and reactive values. new-core Prim API: * It separates 'analysis' (type inference and optimization) from emitting GLSL/C++ code. * The analysis code can be automated. Eg, Binary_Num_Prim provides generic analysis code, so Add_Prim can leave it out. What remains is sc_emit(). * sc_emit() outputs an expression, not a statement. Do this later, once we are building the IR tree. * The result type is needed in type inference and in code generation. It is part of the Prim superclass (eg, Binary_Num_Prim). * The old Prim API has `make_expr()`, which encodes the Operation class of an operator like `+` (aka Add_Expr). This shouldn't be needed. * It is used by Binary_Array_Op::op and Unary_Array_Op::op to generate reactive values. * From this context, given a Prim, we should be able to construct a unary or binary expression class. Eg, Binary_Op_Expr. vectorized 'equal', 'unequal' * support broadcasting in SubCurv: scalar->vector, bool->bool32, vec->mat SubCurv: * Add matrix support to `dot` * transpose * a[i] if a is bool32 or matrix SubCurv tests for dyadic primitives: op[a,b] -- list literal op A -- two element array op M -- mat2 op V -- vec2 SubCurv tests for the following: `bit`: support bool vectors `and`, `or`: bool vectors work now (fix bad codegen) `!`: support bool and bool32 vectors Constructing SC typed values from Curv const values: * matrixes are now constructed * general boolean arrays are now constructed vectorized relations select internals, maybe: SC_Type::Array(base, d1, d2)? document the changes to SubCurv. synonyms for struc: * A 'struc' is a first class value in SubCurv: it can be passed as an argument to a function, returned as a result, stored in a variable, compared for equality. A struc is a scalar (a number or boolean), or it is a small array of scalars. Strucs must be small for efficiency reasons: because they have to fit in GPU register memory, and because they have copy semantics; there is no support for dynamic memory allocation or general pointers in SubCurv. * Other names for 'struc': * element * base value (base type) * small value (small type) * local value (large arrays are stored in global memory and are accessed by reference, pulling parts of the array into the local processor cache). Names that use 'struc' or refer to it: is_struc() is_num_struc() is_bool_struc() sc_struc_unify() sc_eval_bool_struc() sc_struc_unify() sc_convert_scalar_to_struc() SC_Type::Base_Type