1Curv 0.6: Reactive values work in all circumstances where they should. 2Relies on new compiler/partial evaluator, new type system. 3 4Update Array documentation: definition of ranked array, ranked function. 5 6* `list[i1,i2,...]` support reactive args 7* `list[i1,i2,...] := ...` in full generality 8* design a general set of adverbs for pipeline programming 9 * Add each_left and each_right from K. What names do I choose? 10* '`foo` y' is equivalent to 'x->foo[x,y]', as a form of partial application. 11 Use cases: A>>`foo`B, map(`foo`B), etc. 12 * Infix `foo` should have <sum> precedence, so that A `mod` B == 0 works. 13 Use A>>`union`B in a shape pipeline instead of A`union`B. 14 * Prefix `foo` must have higher precedence than <pipeline> (for pipelines), 15 lower precedence than <postfix>, higher precedence than <sum> (for 16 consistency with infix having precedence <sum>). Best option is <power>, 17 which contains the other high precedence prefix operators. 18 * I would also like postfix `foo`, also for partial application. Precedence? 19 It's probably '<sum> `foo`'. 20 21All builtin numeric/boolean operations are now vectorized. Document this. 22 Call_Expr -- could vectorize the function argument. Q'NIAL, FP 23All expression types supported by SubCurv also support reactive arguments. 24Expressions that don't support reactive arguments: 25 select[a,b,c] 26 dot[a,b] 27and[...] and or[...] implement custom reduction code for bool32 and bvec. How? 28 * A functional Prim interface, returns an optional SC_Value. 29 SC_Value sc_try_reduce(SC_Frame& f, SC_Value a, const Context& cx) 30SubCurv: sum[a,b,c,d] is t=[a,b]+[c,d];t[0]+t[1], not (a+b+c+d)? 31 32rename sc_vec_element -> sc_list_elem 33 fix internals to support general struc list 34 fix callers to support general struc list 35update sc_eval_index*_expr() functions to support all struc types 36sc index an array using a struc index value 37add Reactive_Value::at(i,cx), call from list_elem and list_at 38replace idchr() -- use is_C_identifier()? 39Array_Op and Prim should export the same interface. 40Remove boilerplate and duplication from PrimType classes. 41_Prim and _PrimType classes (different names for different signatures) 42migrate all array primitives with operator syntax to Prim 43migrate all array primitives with function syntax to Prim 44 45and[...] and or[...] implement custom reduction code for bool32 and bvec. How? 46 1. Define And_Function directly. Define ::sc_call_expr() to handle 47 special cases first before invoking the general case. It calls sc_reduce, 48 but this contains an if that handles a List_Expr arg before evaluating 49 the arg and handling the value. It's at the value handling point that I want 50 to insert code. 51 * Maybe sc_reduce_expr calls sc_reduce_value. I want to override 52 sc_reduce_value to execute my code then call the default version. 53 2. It is cleaner to define the reducer in And_Prim. 54 * Maybe And_Prim inherits sc_reduce_value from Binary_Prim, then overrides it 55 while calling the default version. Yeah, but the default sc_reduce_value 56 needs to call Prim::sc_check_args and Prim::sc_call. That's recursive, so we 57 need to tie the knot and use OOP + a vtable. 58 * Or tie the knot by passing the default sc_reducer as a function argument 59 to Prim::sc_reduce_value. 60 * Maybe And_Prim defines sc_try_reduce, which defaults to do nothing, 61 inherited from Binary_PrimType. 62 * A procedural interface, returns true on success, mutates a SC_Value& r 63 to return a result. 64 bool sc_try_reduce(SC_Frame& f, SC_Value& a, const Context& cx) 65 if (a.type.is_bool32()) 66 a = ...; 67 return true; 68 return false; 69 * A functional interface, returns an optional SC_Value. 70 SC_Value sc_try_reduce(SC_Frame& f, SC_Value a, const Context& cx) 71 if (a.type.is_bool32()) 72 return ...; 73 return {}; 74 75Soft Typing 76----------- 77Assignment of types to reactive expressions uses "soft typing", where type 78conflicts produce an Error type and don't automatically cause the program to 79abort. sc_result_type computes a soft type for the reactive expression, it 80is not as restrictive as sc_check_args. Figure out the type system and then 81fix this. 82 Here's a bbox value: 83 [[x0,y0,z0],[x1,y1,z1]] * reactive_num 84 It is given the type Array[2]Vec3. Notice this isn't a struc type. 85 Binary_Num_SCMat_Prim::sc_result_type() is rigged to return non-struc types 86 so that this case won't generate a compile error. Non-struc values aren't 87 supported by SubCurv, but this case works out because it is a bbox value, 88 and we aren't compiling this expression into GLSL. 89 90Standardize format of Prim error messages 91----------------------------------------- 92Binary_Array_Op::domain_error: use Prim::name rather than dig into cx.syntax 93 is_C_identifier in symbol.h 94+ if (is_C_identifier(Prim::name())) { 95+ return Exception(cx, 96+ stringify(Prim::name(),"[",x,",",y,"]: invalid argument")); 97+ } else { 98+ return Exception(cx, 99+ stringify(x," ",Prim::name()," ",y,": invalid argument")); 100 Standardized error messages in Prim. For the same Prim, errors can occur: 101 * in a Call_Expr function call expression 102 * in a Unary_Prim_Expr or Binary_Prim_Expr (part of a reactive expr) 103 ERROR: 2 + #blue: invalid arguments 104 <points at Binary_Prim_Expr> 105 ERROR: max[2,#blue]: invalid argument pair 106 <points at Binary_Prim_Expr or Call_Expr> 107 ERROR: argument #1 of max: [2,#blue]: invalid argument pair 108 <points at argument of Call_Expr> 109* Errors produced by sc_check_args, from 2 SC_Values and a cx... what do they 110 look like? In some cases I use At_Index(cx, 0 or 1) but does that work in 111 all cases? 112* Errors resulting from sc_result_type() failures? Maybe there should be no 113 errors. 114 115ISSUES: 116 * rx->expr() makes more logical sense than rx->expr(ph). That's in curv5. 117 The 'ph' argument is used when a Uniform Variable value is converted to an 118 expression operand, but that isn't the same as the source location where 119 the Uniform Variable was initially referenced by name. 120 * list_elem() takes an At_Syntax& cx argument. Which affects upstream APIs. 121 How is it used? What are the actual requirements? 122 * syntax_ of Call_Expr: it's a call site, needs to be passed as argument. 123 * syntax_ of Constant '[i]'. This should be provided by the caller, 124 explaining what the index is, or where it was computed. 125 * cx argument of Reactive_Expression constructor. It's used to throw an 126 exception if Operation arg isn't 'pure', but that's an internal compiler 127 error. Maybe we don't need this: call 'die' instead? 128 * The List_Pattern::exec() call to list_elem(): 129 * The result call_site: 130 * The index call_site: 131 * I think these are both satisfied by the syntax_ of the List_Pattern 132 element being matched. So, list_elem(val,i,const Phrase& syntax). 133 List_Pattern::exec() has an 'argcx' argument that describes the argument 134 value to be decomposed as a list. Does this context have any relevance to 135 the information we must pass to list_elem()? If an error is reported later, 136 when the result of list_elem() is used as an argument, what information 137 should that error/stack trace contain? 138 * At_Index does not return an At_Syntax, affecting List_Pattern::exec(). 139A Reactive_Expression contains an Operation which has a syntax_ field. 140 * The syntax_ is the call site from which the Operation was invoked: either 141 a location in Curv source code (the analytic case), or a description of the 142 call site in C++ native code (the synthetic case). 143 * For an expression, the syntax_ shows where the value was computed, and the 144 context in which it was being used. What if these are different? The value 145 is computed, stored in a variable/data structure, then used in an argument 146 context that throws an error? The site where the argument is computed is 147 valuable information: nice if we can show that in error messages. 148 * The purpose of syntax_ is to be included in a stack trace, so we can see 149 where an error occurred. The only proper use is syntax_.location(), when 150 a Phrase inside a Context is converted to a stack trace. 151 * Due to the way that Reactive_Expressions are constructed, each Operation 152 in the expression tree may have a call site that is disjoint from its parent 153 or its children. (This doesn't happen in the analytic case.) 154 * Therefore, we shouldn't make too many assumptions about syntax_ objects. 155 We shouldn't assume they were constructed analytically. 156 157 158At present, Reactive_Value::expr(ph) requires a Phrase argument. Hence we need 159a Phrase during general expression evaluation. Ideas: 160 1. In Pattern::exec(Value*,Value,Context&cx,Frame&), change 161 'Context&cx' to 'At_Syntax&cx' so I can use cx.syntax(). Code impact: 162 Initially it requires At_Index and At_Field to be derived from At_Syntax. 163 2. In Pattern::exec, add 'const Phrase& valph' argument. 164The underlying problem is that a Uniform_Variable value requires a Phrase for 165conversion to a Constant each time it is referenced in an arithmetic expression. 166 * When will this Phrase be used? What is the best choice of Phrase? 167I think the Phrase could be used if an arithmetic expression fails to compile 168in the Shape Compiler, in which case the Phrase points to the bad operand. 169But this shouldn't happen, we do type checking right now when we construct 170a reactive expression. It seems like the Phrase won't be used. 171 3. Store the parameter name Identifier phrase in the Uniform_Variable when it 172 is constructed. Looks practical, changes are local. 173 174Operations on abstract list values (Lists and Reactive_Values). 175 bool is_list(val) -- already called islist(val) in math.h 176 unsigned list_count(val) 177 Value list_elem(val, unsigned i, const Context& cx) 178 Value list_elem_unsafe(val, unsigned i) 179No virtual functions (for cache locality) in the fast case of a List. 180This should be faster than virtual functions in Ref_Value. 181 182list_elem(val,i) may construct an Operation, if the list is reactive. 183What syntax object(s) do I use? The syntax_ member of an Operation denotes the 184call site for that operation, either in Curv source code, or in C++ native code. 185 Consider this API: list_elem(val,i,const At_Syntax& cx) 186 where 'cx' is the call site for the 'val[i]' expression. 187 If val is a reactive list, then we need to synthesize a call site 188 for the index 'i' as well. 189 190Old answer: 191 Synthetic operation nodes are generated by Curv primitives written in C++. 192 Their syntax object encodes the name of this primitive. `Synthetic_Phrase` 193 is a subclass of Phrase: so you can test if a Meaning is synthetic. 194Why is this okay? 195 In a reactive value, the source location of each node can potentially be 196 different and disconnected from the locations of the parent and children. 197Who will access these synthetic phrases? Is this just a placeholder to avoid 198storing a null pointer, or will it be used? 199What about errors and stack traces? 200 Using the `time` global reactive variable, you can evaluate a reactive value 201 in the interpreter, substituting a time value. Even though the reactive 202 operation tree is type checked, run time errors can still occur. 203 Eg, 0/0 throws an error. A reactive value could contain the subexpression 204 `time/0`, or `(time-1)/0`. What does the error and stack trace look like? 205 ERROR: 0 / 0: domain error 206 at reactive value: 207 1| (time-1)/0 208 at file "foo": 209 42| ...<some expression yielding a reactive value>... 210 With the above design, reactive values don't need source locations. 211But synthetic phrases might still be useful in the debugger? 212 Perhaps Value::list_elem and List_Pattern::match are native primitives 213 that should each have their own Frame? Or, their own VM opcode? 214Who will call Value::list_elem(), and what does the op tree look like? 215 List_Pattern::exec(). The argument A being matched is a reactive list, and 216 we have an At_Syntax for the argument expression. The pattern is `[a,b]`, 217 so `a` is bound to `A[0]`. What is the syntax object for `A[0]`? 218 What is the syntax object for `[0]` and `0`? 219The syntax_ member of an Operation specifies where the Operation is being 220called from. If Value::list_elem is a primitive Operation, called from 221List_Pattern::match, then the former Op needs a syntax_ describing the call 222site within the latter. Therefore, Value::list_elem(i) needs an additional 223argument that describes the caller, from which a Phrase can be derived. 224I figure this additional argument has type At_Syntax or Frame. 225 226 227Value::list_elem(i) may construct an Operation, if the list is reactive. 228What syntax object do I use? 229 1. Synthetic operation nodes encode the name of the C++ function that 230 generates them in the Phrase object. `Synthetic_Phrase` is a subclass 231 of Phrase: so you can test if a Meaning is synthetic. 232 static auto origin = make<Synthetic_Phrase>(__func__); 233 make<Some_Expr>(origin, ...) 234 2. API is list_elem(unsigned, At_Syntax). 235 3. Copy the syntax from the reactive list. 236In what circumstances is a reactive value's syntax object used? 237 * In the design for the `time` global reactive variable, you can evaluate a 238 reactive value in the interpreter, substituting a time value. Even though 239 the reactive operation tree is type checked, run time errors can still occur. 240 Eg, 0/0 throws an error. A reactive value could contain the subexpression 241 `time/0`, or `(time-1)/0`. What does the error and stack trace look like? 242 ERROR: 0 / 0: domain error 243 at reactive value: 244 1| (time-1)/0 245 at file "foo": 246 42| ...<some expression yielding a reactive value>... 247 With the above design, reactive values don't need source locations. 248 * In a reactive value, the source location of each node can potentially be 249 different and disconnected from the locations of the parent and children. 250 Some of the nodes are synthetic. For these, we could encode the name of the 251 C++ function (__func__) that generates them, in the Phrase object. 252 * Who will call Value::list_elem(), and what At_Syntax info is available? 253 * List_Pattern::exec(). The argument A being matched is a reactive list, and 254 we have an At_Syntax for the argument expression. The pattern is `[a,b]`, 255 so `a` is bound to `A[0]`. What is the syntax object for `A[0]`? 256 What is the syntax object for `[0]` and `0`? 257 258 * Support reactive values in Binary_Array_Op using Infix_Op_Expr. 259 * This will require defining Add_Expr = Infix_Op_Expr<Add_Op>. 260 Same class is used everywhere, so that + expressions hash consistently. 261 * So now, Binary_Array_Op::reactive_op hard codes Infix_Op_Expr<Add_Op> 262 as the expression form of the binary add operation. 263 * Replace add(a,b,cx) with Add_Op::op(a,b,cx). 264 * Replace Add_Expr with Infix_Op_Expr<Add_Op> 265 266Add Operation::print(), for printing function and reactive values. 267 268new-core Prim API: 269 * It separates 'analysis' (type inference and optimization) from emitting 270 GLSL/C++ code. 271 * The analysis code can be automated. Eg, Binary_Num_Prim provides generic 272 analysis code, so Add_Prim can leave it out. What remains is sc_emit(). 273 * sc_emit() outputs an expression, not a statement. 274 Do this later, once we are building the IR tree. 275 * The result type is needed in type inference and in code generation. 276 It is part of the Prim superclass (eg, Binary_Num_Prim). 277 * The old Prim API has `make_expr()`, which encodes the Operation class of 278 an operator like `+` (aka Add_Expr). This shouldn't be needed. 279 * It is used by Binary_Array_Op::op and Unary_Array_Op::op to generate 280 reactive values. 281 * From this context, given a Prim, we should be able to construct 282 a unary or binary expression class. Eg, Binary_Op_Expr<Op>. 283 284vectorized 'equal', 'unequal' 285 * support broadcasting in SubCurv: scalar->vector, bool->bool32, vec->mat 286 287SubCurv: 288* Add matrix support to `dot` 289* transpose 290* a[i] if a is bool32 or matrix 291 292SubCurv tests for dyadic primitives: 293 op[a,b] -- list literal 294 op A -- two element array 295 op M -- mat2 296 op V -- vec2 297 298SubCurv tests for the following: 299 `bit`: support bool vectors 300 `and`, `or`: bool vectors work now (fix bad codegen) 301 `!`: support bool and bool32 vectors 302 Constructing SC typed values from Curv const values: 303 * matrixes are now constructed 304 * general boolean arrays are now constructed 305 vectorized relations 306 select 307 308internals, maybe: 309 SC_Type::Array(base, d1, d2)? 310 311document the changes to SubCurv. 312 313synonyms for struc: 314* A 'struc' is a first class value in SubCurv: it can be passed as an argument 315 to a function, returned as a result, stored in a variable, compared for 316 equality. A struc is a scalar (a number or boolean), or it is a small array 317 of scalars. Strucs must be small for efficiency reasons: because they have to 318 fit in GPU register memory, and because they have copy semantics; there is 319 no support for dynamic memory allocation or general pointers in SubCurv. 320* Other names for 'struc': 321 * element 322 * base value (base type) 323 * small value (small type) 324 * local value (large arrays are stored in global memory and are accessed 325 by reference, pulling parts of the array into the local processor cache). 326Names that use 'struc' or refer to it: 327 is_struc() 328 is_num_struc() 329 is_bool_struc() 330 sc_struc_unify() 331 sc_eval_bool_struc() 332 sc_struc_unify() 333 sc_convert_scalar_to_struc() 334 SC_Type::Base_Type 335