1This is gnuastro.info, produced by makeinfo version 6.8 from 2gnuastro.texi. 3 4This book documents version 0.16 of the GNU Astronomy Utilities 5(Gnuastro). Gnuastro provides various programs and libraries for 6astronomical data manipulation and analysis. 7 8 Copyright © 2015-2021, Free Software Foundation, Inc. 9 10 Permission is granted to copy, distribute and/or modify this 11 document under the terms of the GNU Free Documentation License, 12 Version 1.3 or any later version published by the Free Software 13 Foundation; with no Invariant Sections, no Front-Cover Texts, and 14 no Back-Cover Texts. A copy of the license is included in the 15 section entitled “GNU Free Documentation License”. 16INFO-DIR-SECTION Astronomy 17START-INFO-DIR-ENTRY 18* Gnuastro: (gnuastro). GNU Astronomy Utilities. 19* libgnuastro: (gnuastro)Gnuastro library. Full Gnuastro library doc. 20 21* help-gnuastro: (gnuastro)help-gnuastro mailing list. Getting help. 22 23* bug-gnuastro: (gnuastro)Report a bug. How to report bugs 24 25* Arithmetic: (gnuastro)Arithmetic. Arithmetic operations on pixels. 26* astarithmetic: (gnuastro)Invoking astarithmetic. Options to Arithmetic. 27 28* BuildProgram: (gnuastro)BuildProgram. Compile and run programs using Gnuastro’s library. 29* astbuildprog: (gnuastro)Invoking astbuildprog. Options to BuildProgram. 30 31* ConvertType: (gnuastro)ConvertType. Convert different file types. 32* astconvertt: (gnuastro)Invoking astconvertt. Options to ConvertType. 33 34* Convolve: (gnuastro)Convolve. Convolve an input file with kernel. 35* astconvolve: (gnuastro)Invoking astconvolve. Options to Convolve. 36 37* CosmicCalculator: (gnuastro)CosmicCalculator. For cosmological params. 38* astcosmiccal: (gnuastro)Invoking astcosmiccal. Options to CosmicCalculator. 39 40* Crop: (gnuastro)Crop. Crop region(s) from image(s). 41* astcrop: (gnuastro)Invoking astcrop. Options to Crop. 42 43* Fits: (gnuastro)Fits. View and manipulate FITS extensions and keywords. 44* astfits: (gnuastro)Invoking astfits. Options to Fits. 45 46* MakeCatalog: (gnuastro)MakeCatalog. Make a catalog from labeled image. 47* astmkcatalog: (gnuastro)Invoking astmkcatalog. Options to MakeCatalog. 48 49* MakeNoise: (gnuastro)MakeNoise. Make (add) noise to an image. 50* astmknoise: (gnuastro)Invoking astmknoise. Options to MakeNoise. 51 52* MakeProfiles: (gnuastro)MakeProfiles. Make mock profiles. 53* astmkprof: (gnuastro)Invoking astmkprof. Options to MakeProfiles. 54 55* Match: (gnuastro)Match. Match two separate catalogs. 56* astmatch: (gnuastro)Invoking astmatch. Options to Match. 57 58* NoiseChisel: (gnuastro)NoiseChisel. Detect signal in noise. 59* astnoisechisel: (gnuastro)Invoking astnoisechisel. Options to NoiseChisel. 60 61* Segment: (gnuastro)Segment. Segment detections based on signal structure. 62* astsegment: (gnuastro)Invoking astsegment. Options to Segment. 63 64* Query: (gnuastro)Query. Access remote databases for downloading data. 65* astquery: (gnuastro)Invoking astquery. Options to Query. 66 67* Statistics: (gnuastro)Statistics. Get image Statistics. 68* aststatistics: (gnuastro)Invoking aststatistics. Options to Statistics. 69 70* Table: (gnuastro)Table. Read and write FITS binary or ASCII tables. 71* asttable: (gnuastro)Invoking asttable. Options to Table. 72 73* Warp: (gnuastro)Warp. Warp a dataset to a new grid. 74* astwarp: (gnuastro)Invoking astwarp. Options to Warp. 75 76* astscript: (gnuastro)Installed scripts. Gnuastro’s installed scripts. 77* astscript-sort-by-night: (gnuastro)Invoking astscript-sort-by-night. Options to this script 78* astscript-radial-profile: (gnuastro)Invoking astscript-radial-profile. Options to this script 79* astscript-ds9-region: (gnuastro)Invoking astscript-ds9-region. Options to this script 80 81END-INFO-DIR-ENTRY 82 83 84File: gnuastro.info, Node: Bounding box, Next: Polygons, Prev: Tessellation library, Up: Gnuastro library 85 8611.3.16 Bounding box (‘box.h’) 87------------------------------ 88 89Functions related to reporting the bounding box of certain inputs are 90declared in ‘gnuastro/box.h’. All coordinates in this header are in the 91FITS format (first axis is the horizontal and the second axis is 92vertical). 93 94 -- Function: 95 void 96 gal_box_bound_ellipse_extent (double a, double b, double 97 theta_deg, double *extent) 98 Return the maximum extent along each dimension of the given ellipse 99 from the center of the ellipse. Therefore this is half the extent 100 of the box in each dimension. ‘a’ is the ellipse semi-major axis, 101 ‘b’ is the semi-minor axis, ‘theta_deg’ is the position angle in 102 degrees. The extent in each dimension is in floating point format 103 and stored in ‘extent’ which must already be allocated before this 104 function. 105 106 -- Function: 107 void 108 gal_box_bound_ellipse (double a, double b, double theta_deg, 109 long *width) 110 Any ellipse can be enclosed into a rectangular box. This function 111 will write the height and width of that box where ‘width’ points 112 to. It assumes the center of the ellipse is located within the 113 central pixel of the box. ‘a’ is the ellipse semi-major axis 114 length, ‘b’ is the semi-minor axis, ‘theta_deg’ is the position 115 angle in degrees. The ‘width’ array will contain the output size 116 in long integer type. ‘width[0]’, and ‘width[1]’ are the number of 117 pixels along the first and second FITS axis. Since the ellipse 118 center is assumed to be in the center of the box, all the values in 119 ‘width’ will be an odd integer. 120 121 -- Function: 122 void 123 gal_box_bound_ellipsoid_extent (double *semiaxes, double 124 *euler_deg, double *extent) 125 Return the maximum extent along each dimension of the given 126 ellipsoid from its center. Therefore this is half the extent of 127 the box in each dimension. The semi-axis lengths of the ellipsoid 128 must be present in the 3 element ‘semiaxis’ array. The ‘euler_deg’ 129 array contains the three ellipsoid Euler angles in degrees. For a 130 description of the Euler angles, see description of 131 ‘gal_box_bound_ellipsoid’ below. The extent in each dimension is 132 in floating point format and stored in ‘extent’ which must already 133 be allocated before this function. 134 135 -- Function: 136 void 137 gal_box_bound_ellipsoid (double *semiaxes, double *euler_deg, 138 long *width) 139 Any ellipsoid can be enclosed into a rectangular volume/box. The 140 purpose of this function is to give the integer size/width of that 141 box. The semi-axes lengths of the ellipse must be in the 142 ‘semiaxes’ array (with three elements). The major axis length must 143 be the first element of ‘semiaxes’. The only other condition is 144 that the next two semi-axes must both be smaller than the first. 145 The orientation of the major axis is defined through three proper 146 Euler angles (ZXZ order in degrees) that are given in the 147 ‘euler_deg’ array. The ‘width’ array will contain the output size 148 in long integer type (in FITS axis order). Since the ellipsoid 149 center is assumed to be in the center of the box, all the values in 150 ‘width’ will be an odd integer. 151 152 The proper Euler angles can be defined in many ways (which axes to 153 rotate about). For a full description of the Euler angles, please 154 see Wikipedia (https://en.wikipedia.org/wiki/Euler_angles). Here 155 we adopt the ZXZ (or $Z_1X_2Z_3$) proper Euler angles were the 156 first rotation is done around the Z axis, the second one about the 157 (rotated) X axis and the third about the (rotated) Z axis. 158 159 -- Function: 160 void 161 gal_box_border_from_center (double center, size_t ndim, long 162 *width, long *fpixel, long *lpixel) 163 Given the center coordinates in ‘center’ and the ‘width’ (along 164 each dimension) of a box, return the coordinates of the first 165 (‘fpixel’) and last (‘lpixel’) pixels. All arrays must have ‘ndim’ 166 elements (one for each dimension). 167 168 -- Function: 169 int 170 gal_box_overlap (long *naxes, long *fpixel_i, long *lpixel_i, 171 long *fpixel_o, long *lpixel_o, size_t ndim) 172 An ‘ndim’-dimensional dataset of size ‘naxes’ (along each 173 dimension, in FITS order) and a box with first and last (inclusive) 174 coordinate of ‘fpixel_i’ and ‘lpixel_i’ is given. This box doesn’t 175 necessarily have to lie within the dataset, it can be outside of 176 it, or only partially overlap. This function will change the 177 values of ‘fpixel_i’ and ‘lpixel_i’ to exactly cover the overlap in 178 the input dataset’s coordinates. 179 180 This function will return 1 if there is an overlap and 0 if there 181 isn’t. When there is an overlap, the coordinates of the first and 182 last pixels of the overlap will be put in ‘fpixel_o’ and 183 ‘lpixel_o’. 184 185 186File: gnuastro.info, Node: Polygons, Next: Qsort functions, Prev: Bounding box, Up: Gnuastro library 187 18811.3.17 Polygons (‘polygon.h’) 189------------------------------ 190 191Polygons are commonly necessary in image processing. For example in 192Crop they are used for cutting out non-rectangular regions of a image 193(see *note Crop::), and in Warp, for mapping different pixel grids over 194each other (see *note Warp::). 195 196 Polygons come in two classes: convex and concave (or generally, 197non-convex!), see below for a demonstration. Convex polygons are those 198where all inner angles are less than 180 degrees. By contrast, a convex 199polygon is one where an inner angle may be more than 180 degress. 200 201 Concave Polygon Convex Polygon 202 203 D --------C D------------- C 204 \ | E / | 205 \E | \ | 206 / | \ | 207 A--------B A ----------B 208 209 In all the functions here the vertices (and points) are defined as an 210array. So a polygon with 4 vertices will be identified with an array of 2118 elements with the first two elements keeping the 2D coordinates of the 212first vertice and so on. 213 214 -- Macro: GAL_POLYGON_MAX_CORNERS 215 The largest number of vertices a polygon can have in this library. 216 217 -- Macro: GAL_POLYGON_ROUND_ERR 218 We have to consider floating point round-off errors when dealing 219 with polygons. For example we will take ‘A’ as the maximum of ‘A’ 220 and ‘B’ when ‘A>B-GAL_POLYGON_ROUND_ERR’. 221 222 -- Function: 223 void 224 gal_polygon_vertices_sort_convex (double *in, size_t n, size_t 225 *ordinds) 226 We have a simple polygon (that can result from projection, so its 227 edges don’t collide or it doesn’t have holes) and we want to order 228 its corners in an anticlockwise fashion. This is necessary for 229 clipping it and finding its area later. The input vertices can 230 have practically any order. 231 232 The input (‘in’) is an array containing the coordinates (two 233 values) of each vertice. ‘n’ is the number of corners. So ‘in’ 234 should have ‘2*n’ elements. The output (‘ordinds’) is an array 235 with ‘n’ elements specifying the indices in order. This array must 236 have been allocated before calling this function. The indexes are 237 output for more generic usage, for example in a homographic 238 transform (necessary in warping an image, see *note Warping 239 basics::), the necessary order of vertices is the same for all the 240 pixels. In other words, only the positions of the vertices change, 241 not the way they need to be ordered. Therefore, this function 242 would only be necessary once. 243 244 As a summary, the input is unchanged, only ‘n’ values will be put 245 in the ‘ordinds’ array. Such that calling the input coordinates in 246 the following fashion will give an anti-clockwise order when there 247 are 4 vertices: 248 249 1st vertice: in[ordinds[0]*2], in[ordinds[0]*2+1] 250 2nd vertice: in[ordinds[1]*2], in[ordinds[1]*2+1] 251 3rd vertice: in[ordinds[2]*2], in[ordinds[2]*2+1] 252 4th vertice: in[ordinds[3]*2], in[ordinds[3]*2+1] 253 254 The implementation of this is very similar to the Graham scan in 255 finding the Convex Hull. However, in projection we will never have 256 a concave polygon (the left condition below, where this algorithm 257 will get to E before D), we will always have a convex polygon 258 (right case) or E won’t exist! This is because we are always going 259 to be calculating the area of the overlap between a quadrilateral 260 and the pixel grid or the quadrilateral its self. 261 262 The ‘GAL_POLYGON_MAX_CORNERS’ macro is defined so there will be no 263 need to allocate these temporary arrays separately. Since we are 264 dealing with pixels, the polygon can’t really have too many 265 vertices. 266 267 -- Function: 268 int 269 gal_polygon_is_convex (double *v, size_t n) 270 Returns ‘1’ if the polygon is convex with vertices defined by ‘v’ 271 and ‘0’ if it is a concave polygon. Note that the vertices of the 272 polygon should be sorted in an anti-clockwise manner. 273 274 -- Function: 275 double 276 gal_polygon_area (double *v, size_t n) 277 Find the area of a polygon with vertices defined in ‘v’. ‘v’ 278 points to an array of doubles which keep the positions of the 279 vertices such that ‘v[0]’ and ‘v[1]’ are the positions of the first 280 vertice to be considered. 281 282 -- Function: 283 int 284 gal_polygon_is_inside (double *v, double *p, size_t n) 285 Returns ‘0’ if point ‘p’ in inside a polygon, either convex or 286 concave. The vertices of the polygon are defined by ‘v’ and ‘0’ 287 otherwise, they have to be ordered in an anti-clockwise manner. 288 This function uses the winding number algorithm 289 (https://en.wikipedia.org/wiki/Point_in_polygon#Winding_number_algorithm), 290 to check the points. Note that this is a generic function (working 291 on both concave and convex polygons, so if you know before-hand 292 that your polygon is convex, it is much more efficient to use 293 ‘gal_polygon_is_inside_convex’. 294 295 -- Function: 296 int 297 gal_polygon_is_inside_convex (double *v, double *p, size_t n) 298 Return ‘1’ if the point ‘p’ is within the polygon whose vertices 299 are defined by ‘v’. The polygon is assumed to be convex, for a 300 more generic function that deals with concave and convex polygons, 301 see ‘gal_polygon_is_inside’. Note that the vertices of the polygon 302 have to be sorted in an anti-clock-wise manner. 303 304 -- Function: 305 int 306 gal_polygon_ppropin (double *v, double *p, size_t n) 307 Similar to ‘gal_polygon_is_inside_convex’, except that if the point 308 ‘p’ is on one of the edges of a polygon, this will return ‘0’. 309 310 -- Function: 311 int 312 gal_polygon_is_counterclockwise (double *v, size_t n) 313 Returns ‘1’ if the sorted polygon has a counter-clockwise 314 orientation and ‘0’ otherwise. This function uses the concept of 315 “winding”, which defines the relative order in which the vertices 316 of a polygon are listed to determine the orientation of vertices. 317 For complex polygons (where edges, or sides, intersect), the most 318 significant orientation is returned. In a complex polygon, when 319 the alternative windings are equal (for example an ‘8’-shape) it 320 will return ‘1’ (as if it was counter-clockwise). Note that the 321 polygon vertices have to be sorted before calling this function. 322 323 -- Function: 324 int 325 gal_polygon_to_counterclockwise (double *v, size_t n) 326 Arrange the vertices of the sorted polygon in place, to be in a 327 counter-clockwise direction. If the input polygon already has a 328 counter-clockwise direction it won’t touch the input. The return 329 value is ‘1’ on successful execution. This function is just a 330 wrapper over ‘gal_polygon_is_counterclockwise’, and will reverse 331 the order of the vertices when necessary necessary. 332 333 -- Function: 334 void 335 gal_polygon_clip (double *s, size_t n, double *c, size_t m, 336 double *o, size_t *numcrn) 337 Clip (find the overlap of) two polygons. This function uses the 338 Sutherland-Hodgman 339 (https://en.wikipedia.org/wiki/Sutherland%E2%80%93Hodgman_algorithm) 340 polygon clipping algorithm. Note that the vertices of both 341 polygons have to be sorted in an anti-clock-wise manner. 342 343 -- Function: 344 void 345 gal_polygon_vertices_sort (double *vertices, size_t n, size_t 346 *ordinds) 347 Sort the indices of the un-ordered ‘vertices’ array to a 348 counter-clockwise polygon in the already allocated space of 349 ‘ordinds’. It is assumed that there are ‘n’ vertices, and thus 350 that ‘vertices’ contains ‘2*n’ elements where the two coordinates 351 of the first vertice occupy the first two elements of the array and 352 so on. 353 354 The polygon can be both concave and convex (see the start of this 355 section). However, note that for concave polygons there is no 356 unique sort from an un-ordered set of vertices. So after this 357 function you may want to use ‘gal_polygon_is_convex’ and print a 358 warning to check the output if the polygon was concave. 359 360 Note that the contents of the ‘vertices’ array are left untouched 361 by this function. If you want to write the ordered vertice 362 coordinates in another array with the same size, you can use a loop 363 like this: 364 365 for(i=0;i<n;++i) 366 { 367 ordered[i*2 ] = vertices[ ordinds[i]*2 ]; 368 ordered[i*2+1] = vertices[ ordinds[i]*2 + 1]; 369 } 370 371 In this algorithm, we find the rightmost and leftmost points (based 372 on their x-coordinate) and use the diagonal vector between those 373 points to group the points in arrays based on their position with 374 respect to this vector. For anticlockwise sorting, all the points 375 below the vector are sorted by their ascending x-coordinates and 376 points above the vector are sorted in decreasing order using 377 ‘qsort’. Finally, both these arrays are merged together to get the 378 final sorted array of points, from which the points are indexed 379 into the ‘ordinds’ using linear search. 380 381 382File: gnuastro.info, Node: Qsort functions, Next: K-d tree, Prev: Polygons, Up: Gnuastro library 383 38411.3.18 Qsort functions (‘qsort.h’) 385----------------------------------- 386 387When sorting a dataset is necessary, the C programming language provides 388the ‘qsort’ (Quick sort) function. ‘qsort’ is a generic function which 389allows you to sort any kind of data structure (not just a single array 390of numbers). To define “greater” and “smaller” (for sorting), ‘qsort’ 391needs another function, even for simple numerical types. The functions 392introduced in this section are to passed onto ‘qsort’. 393 394 Note that larger and smaller operators are not defined on NaN 395elements. Therefore, if the input array is a floating point type, and 396contains NaN values, the relevant functions of this section are going to 397put the NaN elements at the end of the list (after the sorted non-NaN 398elements), irrespective of the requested sorting order (increasing or 399decreasing). 400 401 The first class of functions below (with ‘TYPE’ in their names) can 402be used for sorting a simple numeric array. Just replace ‘TYPE’ with 403the dataset’s numeric datatype. The second set of functions can be used 404to sort indices (leave the actual numbers untouched). To use the second 405set of functions, a global variable or structure are also necessary as 406described below. 407 408 -- Global variable: gal_qsort_index_single 409 Pointer to an array (for example ‘float *’ or ‘int *’) to use as a 410 reference in ‘gal_qsort_index_single_TYPE_d’ or 411 ‘gal_qsort_index_single_TYPE_i’, see the explanation of these 412 functions for more. Note that if _more than one_ array is to be 413 sorted in a multi-threaded operation, these functions will not work 414 as expected. However, when all the threads just sort the indices 415 based on a _single array_, this global variable can safely be used 416 in a multi-threaded scenario. 417 418 -- Type (C struct): gal_qsort_index_multi 419 Structure to get the sorted indices of multiple datasets on 420 multiple threads with ‘gal_qsort_index_multi_d’ or 421 ‘gal_qsort_index_multi_i’. Note that the ‘values’ array will not 422 be changed by these functions, it is only read. Therefore all the 423 ‘values’ elements in the (to be sorted) array of 424 ‘gal_qsort_index_multi’ must point to the same place. 425 426 struct gal_qsort_index_multi 427 { 428 float *values; /* Array of values (same in all). */ 429 size_t index; /* Index of each element to be sorted. */ 430 }; 431 432 -- Function: 433 int 434 gal_qsort_TYPE_d (const void *a, const void *b) 435 When passed to ‘qsort’, this function will sort a ‘TYPE’ array in 436 decreasing order (first element will be the largest). Please 437 replace ‘TYPE’ (in the function name) with one of the *note Numeric 438 data types::, for example ‘gal_qsort_int32_d’, or 439 ‘gal_qsort_float64_d’. 440 441 -- Function: 442 int 443 gal_qsort_TYPE_i (const void *a, const void *b) 444 When passed to ‘qsort’, this function will sort a ‘TYPE’ array in 445 increasing order (first element will be the smallest). Please 446 replace ‘TYPE’ (in the function name) with one of the *note Numeric 447 data types::, for example ‘gal_qsort_int32_i’, or 448 ‘gal_qsort_float64_i’. 449 450 -- Function: 451 int 452 gal_qsort_index_single_TYPE_d (const void *a, const void *b) 453 When passed to ‘qsort’, this function will sort a ‘size_t’ array 454 based on decreasing values in the ‘gal_qsort_index_single’. The 455 global ‘gal_qsort_index_single’ pointer has a ‘void *’ pointer 456 which will be cast to the proper type based on this function: for 457 example ‘gal_qsort_index_single_uint16_d’ will cast the array to an 458 unsigned 16-bit integer type. The array that 459 ‘gal_qsort_index_single’ points to will not be changed, it is only 460 read. For example, see this demo program: 461 462 #include <stdio.h> 463 #include <stdlib.h> /* qsort is defined in stdlib.h. */ 464 #include <gnuastro/qsort.h> 465 466 int 467 main (void) 468 { 469 size_t s[4]={0, 1, 2, 3}; 470 float f[4]={1.3,0.2,1.8,0.1}; 471 gal_qsort_index_single=f; 472 qsort(s, 4, sizeof(size_t), gal_qsort_index_single_float_d); 473 printf("%zu, %zu, %zu, %zu\n", s[0], s[1], s[2], s[3]); 474 return EXIT_SUCCESS; 475 } 476 477 The output will be: ‘2, 0, 1, 3’. 478 479 -- Function: 480 int 481 gal_qsort_index_single_TYPE_i (const void *a, const void *b) 482 Similar to ‘gal_qsort_index_single_TYPE_d’, but will sort the 483 indexes such that the values of ‘gal_qsort_index_single’ can be 484 parsed in increasing order. 485 486 -- Function: 487 int 488 gal_qsort_index_multi_d (const void *a, const void *b) 489 When passed to ‘qsort’ with an array of ‘gal_qsort_index_multi’, 490 this function will sort the array based on the values of the given 491 indices. The sorting will be ordered according to the ‘values’ 492 pointer of ‘gal_qsort_index_multi’. Note that ‘values’ must point 493 to the same place in all the structures of the 494 ‘gal_qsort_index_multi’ array. 495 496 This function is only useful when the indices of multiple arrays on 497 multiple threads are to be sorted. If your program is single 498 threaded, or all the indices belong to a single array (sorting 499 different sub-sets of indices in a single array on multiple 500 threads), it is recommended to use ‘gal_qsort_index_single_d’. 501 502 -- Function: 503 int 504 gal_qsort_index_multi_i (const void *a, const void *b) 505 Similar to ‘gal_qsort_index_multi_d’, but the result will be sorted 506 in increasing order (first element will have the smallest value). 507 508 509File: gnuastro.info, Node: K-d tree, Next: Permutations, Prev: Qsort functions, Up: Gnuastro library 510 51111.3.19 K-d tree (‘kdtree.h’) 512----------------------------- 513 514K-d tree is a space-partitioning binary search tree for organizing 515points in a k-dimensional space. They are a very useful data structure 516for multidimensional searches like range searches and nearest neighbor 517searches. For a more formal and complete introduction see the Wikipedia 518page (https://en.wikipedia.org/wiki/K-d_tree). 519 520 Each non-leaf node in a k-d tree divides the space into two parts, 521known as half-spaces. To select the top/root node for partitioning, we 522find the median of the points and make a hyperplane normal to the first 523dimension. The points to the left of this space are represented by the 524left subtree of that node and points to the right of the space are 525represented by the right subtree. This is then repeated for all the 526points in the input, thus associating a “left” and “right” branch for 527each input point. 528 529 Gnuastro uses the standard algorithms of the k-d tree with one small 530difference that makes it much more memory and CPU optimized. The set of 531input points that define the tree nodes are given as a list of 532Gnuastro’s data container type, see *note List of gal_data_t::. Each 533‘gal_data_t’ in the list represents the point’s coordinate in one 534dimension, and the first element in the list is the first dimension. 535Hence the number of data values in each ‘gal_data_t’ (which must be 536equal in all of them) represents the number of points. This is the same 537format that Gnuastro’s Table reading/writing functions read/write 538columns in tables, see *note Table input output::. 539 540 The output k-d tree is a list of two ‘gal_data_t’s, representing the 541input’s row-number (or index, counting from 0) of the left and right 542subtrees of each row. Each ‘gal_data_t’ thus has the same number of 543rows (or points) as the input, but only containing integers with a type 544of ‘uint32_t’ (unsigned 32-bit integer). If a node has no left, or 545right subtree, then ‘GAL_BLANK_UINT32’ will be used. Below you can see 546the simple tree for 2D points from Wikipedia. The input point 547coordinates are represented as two input ‘gal_data_t’s (‘X’ and ‘Y’, 548where ‘X->next=Y’ and ‘Y->next=NULL’). If you had three dimensional 549points, you could define an extra ‘gal_data_t’ such that ‘Y->next=Z’ and 550‘Z->next=NULL’. The output is always a list of two ‘gal_data_t’s, where 551the first one contains the index of the left sub-tree in the input, and 552the second one, the index of the right subtree. The index of the root 553node (‘0’ in the case below(1)) is also returned as a single number. 554 555 INDEX INPUT OUTPUT K-D Tree 556 (as guide) X --> Y LEFT --> RIGHT (visualized) 557 ---------- ------- -------------- ------------------ 558 0 5 4 1 2 (5,4) 559 1 2 3 BLANK 4 / \ 560 2 7 2 5 3 (2,3) \ 561 3 9 6 BLANK BLANK \ (7,2) 562 4 4 7 BLANK BLANK (4,7) / \ 563 5 8 1 BLANK BLANK (8,1) (9,6) 564 565 This format is therefore scalable to any number of dimensions: the 566number of dimensions are determined from the number of nodes in the 567input list of ‘gal_data_t’s (for example, using ‘gal_list_data_number’). 568In Gnuastro’s k-d tree implementation, there are thus no special 569structures to keep every tree node (which would take extra memory and 570would need to be moved around as the tree is being created). Everything 571is done internally on the index of each point in the input dataset: the 572only thing that is flipped/sorted during tree creation is the index to 573the input row for any number of dimensions. As a result, Gnuastro’s k-d 574tree implementation is very memory and CPU efficient and its two output 575columns can directly be written into a standard table (without having to 576define any special binary format). 577 578 -- Function: 579 gal_data_t * 580 gal_kdtree_create (gal_data_t *coords_raw, size_t *root) 581 Create a k-d tree in a bottom-up manner (from leaves to the root). 582 This function returns two ‘gal_data_t’s connected as a list, see 583 description above. The first dataset contains the indexes of left 584 and right nodes of the subtrees for each input node. The index of 585 the root node is written into the memory that ‘root’ points to. 586 ‘coords_raw’ is the list of the input points (one ‘gal_data_t’ per 587 dimension, see above). 588 589 For example, assume you have the simple set of points below (from 590 the visualized example at the start of this section) in a 591 plain-text file called ‘coordinates.txt’: 592 593 $ cat coordinates.txt 594 5 4 595 2 3 596 7 2 597 9 6 598 4 7 599 8 1 600 601 With the program below, you can calculate the kd-tree, and write it 602 in a FITS file (while keeping the root index as a FITS keyword 603 inside of it). 604 605 #include <stdio.h> 606 #include <gnuastro/table.h> 607 #include <gnuastro/kdtree.h> 608 609 int 610 main (void) 611 { 612 gal_data_t *input, *kdtree; 613 char kdtreefile[]="kd-tree.fits"; 614 char inputfile[]="coordinates.txt"; 615 616 /* To write the root within the saved file. */ 617 size_t root; 618 char *unit="index"; 619 char *keyname="KDTROOT"; 620 gal_fits_list_key_t *keylist=NULL; 621 char *comment="k-d tree root index (counting from 0)."; 622 623 /* Read the input table. Note: this assumes the table only 624 * contains your input point coordinates (one column for each 625 * dimension). If it contains more columns with other properties 626 * for each point, you can specify which columns to read by 627 * name or number, see the documentation of 'gal_table_read'. */ 628 input=gal_table_read(inputfile, "1", NULL, NULL, 629 GAL_TABLE_SEARCH_NAME, 0, -1, 0, NULL); 630 631 /* Construct a k-d tree. The index of root is stored in `root` */ 632 kdtree=gal_kdtree_create(input, &root); 633 634 /* Write the k-d tree to a file and write root index and input 635 * name as FITS keywords ('gal_table_write' frees 'keylist').*/ 636 gal_fits_key_list_title_add(&keylist, "k-d tree parameters", 0); 637 gal_fits_key_write_filename("KDTIN", inputfile, &keylist, 0); 638 gal_fits_key_list_add_end(&keylist, GAL_TYPE_SIZE_T, keyname, 0, 639 &root, 0, comment, 0, unit, 0); 640 gal_table_write(kdtree, &keylist, NULL, GAL_TABLE_FORMAT_BFITS, 641 kdtreefile, "kdtree", 0); 642 643 /* Clean up and return. */ 644 gal_list_data_free(input); 645 gal_list_data_free(kdtree); 646 return EXIT_SUCCESS; 647 } 648 649 You can inspect the saved k-d tree FITS table with Gnuastro’s *note 650 Table:: (first command below), and you can see the keywords 651 containing the root index with *note Fits:: (second command below): 652 653 asttable kd-tree.fits 654 astfits kd-tree.fits -h1 655 656 -- Function: 657 size_t 658 gal_kdtree_nearest_neighbour (gal_data_t *coords_raw, 659 gal_data_t *kdtree, size_t root, double *point, double 660 *least_dist) 661 Returns the index of the nearest input point to the query point 662 (‘point’, assumed to be an array with same number of elements as 663 ‘gal_data_t’s in ‘coords_raw’). The distance between the query 664 point and its nearest neighbor is stored in the space that 665 ‘least_dist’ points to. This search is efficient due to the 666 constant checking for the presence of possible best points in other 667 branches. If it isn’t possible for the other branch to have a 668 better nearest neighbor, that branch is not searched. 669 670 As an example, let’s use the k-d tree that was created in the 671 example of ‘gal_kdtree_create’ (above) and find the nearest row to 672 a given coordinate (‘point’). This will be a very common scenario, 673 especially in large and multi-dimensional datasets where the k-d 674 tree creation can take long and you don’t want to re-create the k-d 675 tree every time. In the ‘gal_kdtree_create’ example output, we 676 also wrote the k-d tree root index as a FITS keyword (‘KDTROOT’), 677 so after loading the two table data (input coordinates and k-d 678 tree), we’ll read the root from the FITS keyword. This is a very 679 simple example, but the scalability is clear: for example it is 680 trivial to parallelize (see *note Library demo - multi-threaded 681 operation::). 682 683 #include <stdio.h> 684 #include <gnuastro/table.h> 685 #include <gnuastro/kdtree.h> 686 687 int 688 main (void) 689 { 690 /* INPUT: desired point. */ 691 double point[2]={8.9,5.9}; 692 693 /* Same as example in description of 'gal_kdtree_create'. */ 694 gal_data_t *input, *kdtree; 695 char kdtreefile[]="kd-tree.fits"; 696 char inputfile[]="coordinates.txt"; 697 698 /* Processing variables of this function. */ 699 char kdtreehdu[]="1"; 700 double *in_x, *in_y, least_dist; 701 size_t root, nkeys=1, nearest_index; 702 gal_data_t *rkey, *keysll=gal_data_array_calloc(nkeys); 703 704 /* Read the input coordinates, see comments in example of 705 * 'gal_kdtree_create' for more. */ 706 input=gal_table_read(inputfile, "1", NULL, NULL, 707 GAL_TABLE_SEARCH_NAME, 0, -1, 0, NULL); 708 709 /* Read the k-d tree contents (created before). */ 710 kdtree=gal_table_read(kdtreefile, "1", NULL, NULL, 711 GAL_TABLE_SEARCH_NAME, 0, -1, 0, NULL); 712 713 /* Read the k-d tree root index from the header keyword. 714 * See example in description of 'gal_fits_key_read_from_ptr'.*/ 715 keysll[0].name="KDTROOT"; 716 keysll[0].type=GAL_TYPE_SIZE_T; 717 gal_fits_key_read(kdtreefile, kdtreehdu, keysll, 0, 0); 718 keysll[0].name=NULL; /* Since we didn't allocate it. */ 719 rkey=gal_data_copy_to_new_type(&keysll[0], GAL_TYPE_SIZE_T); 720 root=((size_t *)(rkey->array))[0]; 721 722 /* Find the nearest neighbour of the point. */ 723 nearest_index=gal_kdtree_nearest_neighbour(input, kdtree, root, 724 point, &least_dist); 725 726 /* Print the results. */ 727 in_x=input->array; 728 in_y=input->next->array; 729 printf("(%g, %g): nearest is (%g, %g), with a distance of %g\n", 730 point[0], point[1], in_x[nearest_index], 731 in_y[nearest_index], least_dist); 732 733 /* Clean up and return. */ 734 gal_data_free(rkey); 735 gal_list_data_free(input); 736 gal_list_data_free(kdtree); 737 gal_data_array_free(keysll, nkeys, 1); 738 return EXIT_SUCCESS; 739 } 740 741 ---------- Footnotes ---------- 742 743 (1) This example input table is the same as the example in Wikipedia 744(as of December 2020). However, on the Wikipedia output, the root node 745is (7,2), not (5,4). The difference is primarily because there are 6 746rows and the median element of an even number of elements can vary by 747integer calculation strategies. Here we use 0-based indexes for finding 748median and round to the smaller integer. 749 750 751File: gnuastro.info, Node: Permutations, Next: Matching, Prev: K-d tree, Up: Gnuastro library 752 75311.3.20 Permutations (‘permutation.h’) 754-------------------------------------- 755 756Permutation is the technical name for re-ordering of values. The need 757for permutations occurs a lot during (mainly low-level) processing. To 758do permutation, you must provide two inputs: an array of values (that 759you want to re-order in place) and a permutation array which contains 760the new index of each element (let’s call it ‘perm’). The diagram below 761shows the input array before and after the re-ordering. 762 763 permute: AFTER[ i ] = BEFORE[ perm[i] ] i = 0 .. N-1 764 inverse: AFTER[ perm[i] ] = BEFORE[ i ] i = 0 .. N-1 765 766 The functions here are a re-implementation of the GNU Scientific 767Library’s ‘gsl_permute’ function. The reason we didn’t use that 768function was that it uses system-specific types (like ‘long’ and ‘int’) 769which can have different widths on different systems, hence are not 770easily convertible to Gnuastro’s fixed width types (see *note Numeric 771data types::). There is also a separate function for each type, heavily 772using macros to allow a ‘base’ function to work on all the types. Thus 773it is hard to read/understand. Hence, Gnuastro contains a re-write of 774their steps in a new type-agnostic method which is a single function 775that can work on any type. 776 777 As described in GSL’s source code and manual, this implementation 778comes from Donald Knuth’s _Art of computer programming_ book, in the 779"Sorting and Searching" chapter of Volume 3 (3rd ed). Exercise 10 of 780Section 5.2 defines the problem and in the answers, Knuth describes the 781solution. So if you are interested, please have a look there for more. 782 783 We are in contact with the GSL developers and in the future(1) we 784will submit these implementations to GSL. If they are finally 785incorporated there, we will delete this section in future versions. 786 787 -- Function: 788 void 789 gal_permutation_check (size_t *permutation, size_t size) 790 Print how ‘permutation’ will re-order an array that has ‘size’ 791 elements for each element in one one line. 792 793 -- Function: 794 void 795 gal_permutation_apply (gal_data_t *input, size_t *permutation) 796 Apply ‘permutation’ on the ‘input’ dataset (can have any type), see 797 above for the definition of permutation. 798 799 -- Function: 800 void 801 gal_permutation_apply_inverse (gal_data_t *input, size_t 802 *permutation) 803 Apply the inverse of ‘permutation’ on the ‘input’ dataset (can have 804 any type), see above for the definition of permutation. 805 806 ---------- Footnotes ---------- 807 808 (1) Gnuastro’s Task 14497 (http://savannah.gnu.org/task/?14497). If 809this task is still “postponed” when you are reading this and you are 810interested to help, your help would be very welcome. Both Gnuastro and 811GSL developers are very busy, hence both would appreciate your help. 812 813 814File: gnuastro.info, Node: Matching, Next: Statistical operations, Prev: Permutations, Up: Gnuastro library 815 81611.3.21 Matching (‘match.h’) 817---------------------------- 818 819Matching is often necessary when two measurements of the same points 820have been done using different instruments (or hardware), different 821software or different configurations of the same software. In other 822words, you have two catalogs or tables and each has N columns containing 823the N-dimensional “positional” values of each point. Each can have 824other columns too, for example one can have brightness measurements in 825one filter, and another can have brightness measurements in another 826filter as well as morphology measurements or etc. 827 828 The matching functions here will use the positional columns to find 829the permutation necessary to apply to both tables. This will enable you 830to match by the positions, then apply the permutation to the brightness 831or morphology columns in the example above. The input and output data 832formats of the functions below are the some and described below before 833the actual functions. Each function also has extra arguments due to the 834particular algorithm it uses for the matching. 835 836 The two inputs of the functions (‘coord1’ and ‘coord2’) must be *note 837List of gal_data_t::. Each ‘gal_data_t’ node in ‘coord1’ or ‘coord2’ 838should be a single dimensional dataset (column in a table) and all the 839nodes must have the same number of elements (rows). In other words, 840each column can be visualized as having the coordinates of each point in 841its respective dimension. The dimensions of the coordinates is 842determined by the number of ‘gal_data_t’ nodes in the two input lists 843(which must be equal). The number of rows (or the number of elements in 844each ‘gal_data_t’) in the columns of ‘coord1’ and ‘coord2’ can be 845different. All these functions will all be satisfied if you use 846‘gal_table_read’ to read the two coordinate columns, see *note Table 847input output::. 848 849 The functions below return a simply-linked list of three 1D datasets 850(see *note List of gal_data_t::), let’s call the returned dataset ‘ret’. 851The first two (‘ret’ and ‘ret->next’) are permutations. In other words, 852the ‘array’ elements of both have a type of ‘size_t’, see *note 853Permutations::. The third node (‘ret->next->next’) is the calculated 854distance for that match and its array has a type of ‘double’. The 855number of matches will be put in the space pointed by the ‘nummatched’ 856argument. If there wasn’t any match, this function will return ‘NULL’. 857 858 The two permutations can be applied to the rows of the two inputs: 859the first one (‘ret’) should be applied to the rows of the table 860containing ‘coord1’ and the second one (‘ret->next’) to the table 861containing ‘coord2’. After applying the returned permutations to the 862inputs, the top ‘nummatched’ elements of both will match with each 863other. The ordering of the rest of the elements is undefined (depends 864on the matching funciton used). The third node is the distances between 865the respective match (which may be elliptical distance, see discussion 866of “aperture” below). 867 868 The functions will not simply return the nearest neighbor as a match. 869The nearest neighbor may be too far to be a meaningful. They will check 870the distance between the distance of the nearest neighbor of each point 871and only return a match for it it is within an acceptable N-dimensional 872distance (or “aperture”). The matching aperture is defined by the 873‘aperture’ array that is an input argument to the functions. If several 874points of one catalog lie within this aperture of a point in the other, 875the nearest is defined as the match. In a 2D situation (where the input 876lists have two nodes), for the most generic case, it must have three 877elements: the major axis length, axis ratio and position angle (see 878*note Defining an ellipse and ellipsoid::). If ‘aperture[1]==1’, the 879aperture will be a circle of radius ‘aperture[0]’ and the third value 880won’t be used. When the aperture is an ellipse, distances between the 881points are also calculated in the respective elliptical distances 882($r_{el}$ in *note Defining an ellipse and ellipsoid::). 883 884 -- Function: 885 gal_data_t * 886 gal_match_coordinates (gal_data_t *coord1, gal_data_t *coord2, 887 double *aperture, int sorted_by_first, int inplace, size_t 888 minmapsize, int quietmmap, size_t *nummatched) 889 890 Use a basic sort-based match to find the matching points of two 891 input coordinates. See the descriptions above on the format of the 892 inputs and outputs. To speed up the search, this function will 893 sort the input coordinates by their first column (first axis). If 894 _both_ are already sorted by their first column, you can avoid the 895 sorting step by giving a non-zero value to ‘sorted_by_first’. 896 897 When sorting is necessary and ‘inplace’ is non-zero, the actual 898 input columns will be sorted. Otherwise, an internal copy of the 899 inputs will be made, used (sorted) and later freed before 900 returning. Therefore, when ‘inplace==0’, inputs will remain 901 untouched, but this function will take more time and memory. If 902 internal allocation is necessary and the space is larger than 903 ‘minmapsize’, the space will be not allocated in the RAM, but in a 904 file, see description of ‘--minmapsize’ and ‘--quietmmap’ in *note 905 Processing options::. 906 907 *Output permutations ignore internal sorting*: the output 908 permutations will correspond to the initial inputs. Therefore, 909 even when ‘inplace!=0’ (and this function re-arranges the inputs in 910 place), the output permutation will correspond to original 911 (possibly non-sorted) inputs. 912 913 The reason for this is that you rarely want to permute the actual 914 positional columns after the match. Usually, you also have other 915 columns (for example the brightness, morphology and etc) and you 916 want to find how they differ between the objects that match. Once 917 you have the permutations, they can be applied to those other 918 columns (see *note Permutations::) and the higher-level processing 919 can continue. So if you don’t need the coordinate columns for the 920 rest of your analysis, it is better to set ‘inplace=1’. 921 922 923File: gnuastro.info, Node: Statistical operations, Next: Binary datasets, Prev: Matching, Up: Gnuastro library 924 92511.3.22 Statistical operations (‘statistics.h’) 926----------------------------------------------- 927 928After reading a dataset into memory from a file or fully simulating it 929with another process, the most common processes that will be done on it 930are statistical operations to let you quantify different aspects of the 931data. the functions in this section describe Gnuastro’s current set of 932tools for this job. All these functions can work on any numeric data 933type natively (see *note Numeric data types::) and can also work on 934tiles over a dataset. Hence the inputs and outputs are in Gnuastro’s 935*note Generic data container::. 936 937 -- Macro: GAL_STATISTICS_SIG_CLIP_MAX_CONVERGE 938 The maximum number of clips, when $\sigma$-clipping should be done 939 by convergence. If the clipping does not converge before making 940 this many clips, all $\sigma$-clipping outputs will be NaN. 941 942 -- Macro: GAL_STATISTICS_MODE_GOOD_SYM 943 The minimum acceptable symmetricity of the mode calculation. If 944 the symmetricity of the derived mode is less than this value, all 945 the returned values by ‘gal_statistics_mode’ will have a value of 946 NaN. 947 948 -- Macro: GAL_STATISTICS_BINS_INVALID 949 -- Macro: GAL_STATISTICS_BINS_REGULAR 950 -- Macro: GAL_STATISTICS_BINS_IRREGULAR 951 Macros used to identify if the regularity of the bins when defining 952 bins. 953 954 -- Function: 955 gal_data_t * 956 gal_statistics_number (gal_data_t *input) 957 Return a single-element dataset with type ‘size_t’ which contains 958 the number of non-blank elements in ‘input’. 959 960 -- Function: 961 gal_data_t * 962 gal_statistics_minimum (gal_data_t *input) 963 Return a single-element dataset containing the minimum non-blank 964 value in ‘input’. The numerical datatype of the output is the same 965 as ‘input’. 966 967 -- Function: 968 gal_data_t * 969 gal_statistics_maximum (gal_data_t *input) 970 Return a single-element dataset containing the maximum non-blank 971 value in ‘input’. The numerical datatype of the output is the same 972 as ‘input’. 973 974 -- Function: 975 gal_data_t * 976 gal_statistics_sum (gal_data_t *input) 977 Return a single-element (‘double’ or ‘float64’) dataset containing 978 the sum of the non-blank values in ‘input’. 979 980 -- Function: 981 gal_data_t * 982 gal_statistics_mean (gal_data_t *input) 983 Return a single-element (‘double’ or ‘float64’) dataset containing 984 the mean of the non-blank values in ‘input’. 985 986 -- Function: 987 gal_data_t * 988 gal_statistics_std (gal_data_t *input) 989 Return a single-element (‘double’ or ‘float64’) dataset containing 990 the standard deviation of the non-blank values in ‘input’. 991 992 -- Function: 993 gal_data_t * 994 gal_statistics_mean_std (gal_data_t *input) 995 Return a two-element (‘double’ or ‘float64’) dataset containing the 996 mean and standard deviation of the non-blank values in ‘input’. 997 The first element of the returned dataset is the mean and the 998 second is the standard deviation. 999 1000 This function will calculate both values in one pass over the 1001 dataset. Hence when both the mean and standard deviation of a 1002 dataset are necessary, this function is much more efficient than 1003 calling ‘gal_statistics_mean’ and ‘gal_statistics_std’ separately. 1004 1005 -- Function: 1006 gal_data_t * 1007 gal_statistics_median (gal_data_t *input, int inplace) 1008 Return a single-element dataset containing the median of the 1009 non-blank values in ‘input’. The numerical datatype of the output 1010 is the same as ‘input’. 1011 1012 Calculating the median involves sorting the dataset and removing 1013 blank values, for better performance (and less memory usage), you 1014 can give a non-zero value to the ‘inplace’ argument. In this case, 1015 the sorting and removal of blank elements will be done directly on 1016 the input dataset. However, after this function the original 1017 dataset may have changed (if it wasn’t sorted or had blank values). 1018 1019 -- Function: 1020 size_t 1021 gal_statistics_quantile_index (size_t size, double quantile) 1022 Return the index of the element that has a quantile of ‘quantile’ 1023 assuming the dataset has ‘size’ elements. 1024 1025 -- Function: 1026 gal_data_t * 1027 gal_statistics_quantile (gal_data_t *input, double quantile, 1028 int inplace) 1029 Return a single-element dataset containing the value with in a 1030 quantile ‘quantile’ of the non-blank values in ‘input’. The 1031 numerical datatype of the output is the same as ‘input’. See 1032 ‘gal_statistics_median’ for a description of ‘inplace’. 1033 1034 -- Function: 1035 size_t 1036 gal_statistics_quantile_function_index (gal_data_t *input, 1037 gal_data_t *value, int inplace) 1038 Return the index of the quantile function (inverse quantile) of 1039 ‘input’ at ‘value’. In other words, this function will return the 1040 index of the nearest element (of a sorted and non-blank) ‘input’ to 1041 ‘value’. If the value is outside the range of the input, then this 1042 function will return ‘GAL_BLANK_SIZE_T’. 1043 1044 -- Function: 1045 gal_data_t * 1046 gal_statistics_quantile_function (gal_data_t *input, 1047 gal_data_t *value, int inplace) 1048 1049 Return a single-element dataset containing the quantile function of 1050 the non-blank values in ‘input’ at ‘value’ (a single-element 1051 dataset). The numerical data type is of the returned dataset is 1052 ‘float64’ (or ‘double’). In other words, this function will return 1053 the quantile of ‘value’ in ‘input’. ‘value’ has to have the same 1054 type as ‘input’. See ‘gal_statistics_median’ for a description of 1055 ‘inplace’. 1056 1057 When all elements are blank, the returned value will be NaN. If the 1058 value is smaller than the input’s smallest element, the returned 1059 value will be negative infinity. If the value is larger than the 1060 input’s largest element, then the returned value will be positive 1061 infinity 1062 1063 -- Function: 1064 gal_data_t * 1065 gal_statistics_unique (gal_data_t *input, int inplace) 1066 Return a 1D dataset with the same numeric data type as the input, 1067 but only containing its unique elements and without any (possible) 1068 blank/NaN elements. Note that the input’s number of dimensions is 1069 irrelevant for this function. If ‘inplace’ is not zero, then the 1070 unique values will over-write the allocated space of the input, 1071 otherwise a new space will be allocated and the input will not be 1072 touched. 1073 1074 -- Function: 1075 gal_data_t * 1076 gal_statistics_mode (gal_data_t *input, float mirrordist, int 1077 inplace) 1078 Return a four-element (‘double’ or ‘float64’) dataset that contains 1079 the mode of the ‘input’ distribution. This function implements the 1080 non-parametric algorithm to find the mode that is described in 1081 Appendix C of Akhlaghi and Ichikawa [2015] 1082 (https://arxiv.org/abs/1505.01664). 1083 1084 In short it compares the actual distribution and its “mirror 1085 distribution” to find the mode. In order to be efficient, you can 1086 determine how far the comparison goes away from the mirror through 1087 the ‘mirrordist’ parameter (think of it as a multiple of 1088 sigma/error). See ‘gal_statistics_median’ for a description of 1089 ‘inplace’. 1090 1091 The output array has the following elements (in the given order, 1092 note that counting in C starts from 0). 1093 array[0]: mode 1094 array[1]: mode quantile. 1095 array[2]: symmetricity. 1096 array[3]: value at the end of symmetricity. 1097 1098 -- Function: 1099 gal_data_t * 1100 gal_statistics_mode_mirror_plots (gal_data_t *input, 1101 gal_data_t *value, size_t numbins, int inplace, double 1102 *mirror_val) 1103 Make a mirrored histogram and cumulative frequency plot (with 1104 ‘numbins’) with the mirror distribution of the ‘input’ having a 1105 value in ‘value’. If all the input elements are blank, or the 1106 mirror value is outside the range of the input, this function will 1107 return a ‘NULL’ pointer. 1108 1109 The output is a list of data structures (see *note List of 1110 gal_data_t::): the first is the bins with one bin at the mirror 1111 point, the second is the histogram with a maximum of one and the 1112 third is the cumulative frequency plot (with a maximum of one). 1113 1114 -- Function: 1115 int 1116 gal_statistics_is_sorted (gal_data_t *input, int updateflags) 1117 Return ‘0’ if the input is not sorted, if it is sorted, this 1118 function will return ‘1’ and ‘2’ if it is increasing or decreasing, 1119 respectively. This function will abort with an error if ‘input’ 1120 has zero elements and will return ‘1’ (sorted, increasing) when 1121 there is only one element. This function will only look into the 1122 dataset if the ‘GAL_DATA_FLAG_SORT_CH’ bit of ‘input->flag’ is ‘0’, 1123 see *note Generic data container::. 1124 1125 When the flags don’t indicate a previous check _and_ ‘updateflags’ 1126 is non-zero, this function will set the flags appropriately to 1127 avoid having to re-check the dataset in future calls (this can be 1128 very useful when repeated checks are necessary). When 1129 ‘updateflags==0’, this function has no side-effects on the dataset: 1130 it will not toggle the flags. 1131 1132 If you want to re-check a dataset with the blank-value-check flag 1133 already set (for example if you have made changes to it), then 1134 explicitly set the ‘GAL_DATA_FLAG_SORT_CH’ bit to zero before 1135 calling this function. When there are no other flags, you can 1136 simply set the flags to zero (with ‘input->flag=0’), otherwise you 1137 can use this expression: 1138 1139 input->flag &= ~GAL_DATA_FLAG_SORT_CH; 1140 1141 -- Function: 1142 void 1143 gal_statistics_sort_increasing (gal_data_t *input) 1144 Sort the input dataset (in place) in an increasing order and toggle 1145 the sort-related bit flags accordingly. 1146 1147 -- Function: 1148 void 1149 gal_statistics_sort_decreasing (gal_data_t *input) 1150 Sort the input dataset (in place) in a decreasing order and toggle 1151 the sort-related bit flags accordingly. 1152 1153 -- Function: 1154 gal_data_t * 1155 gal_statistics_no_blank_sorted (gal_data_t *input, int 1156 inplace) 1157 Remove all the blanks and sort the input dataset. If ‘inplace’ is 1158 non-zero this will happen on the input dataset (in the allocated 1159 space of the input dataset). However, if ‘inplace’ is zero, this 1160 function will allocate a new copy of the dataset and work on that. 1161 Therefore if ‘inplace==0’, the input dataset will be modified. 1162 1163 This function uses the bit flags of the input, so if you have 1164 modified the dataset, set ‘input->flag=0’ before calling this 1165 function. Also note that ‘inplace’ is only for the dataset 1166 elements. Therefore even when ‘inplace==0’, if the input is 1167 already sorted _and_ has no blank values, then the flags will be 1168 updated to show this. 1169 1170 If all the elements were blank, then the returned dataset’s ‘size’ 1171 will be zero. This is thus a good parameter to check after calling 1172 this function to see if there actually were any non-blank elements 1173 in the input or not and take the appropriate measure. This can 1174 help avoid strange bugs in later steps. The flags of a zero-sized 1175 returned dataset will indicate that it has no blanks and is sorted 1176 in an increasing order. Even if having blank values or being 1177 sorted is not defined on a zero-element dataset, it is up to the 1178 caller to choose what they will do with a zero-element dataset. 1179 The flags have to be set after this function any way. 1180 1181 -- Function: 1182 gal_data_t * 1183 gal_statistics_regular_bins (gal_data_t *input, gal_data_t 1184 *inrange, size_t numbins, double onebinstart) 1185 Generate an array of regularly spaced elements as a 1D array 1186 (column) of type ‘double’ (i.e., ‘float64’, it has to be double to 1187 account for small differences on the bin edges). The input 1188 arguments are described below 1189 1190 ‘input’ 1191 The dataset you want to apply the bins to. This is only 1192 necessary if the range argument is not complete, see below. 1193 If ‘inrange’ has all the necessary information, you can pass a 1194 ‘NULL’ pointer for this. 1195 1196 ‘inrange’ 1197 This dataset keeps the desired range along each dimension of 1198 the input data structure, it has to be in ‘float’ (i.e., 1199 ‘float32’) type. 1200 1201 • If you want the full range of the dataset (in any 1202 dimensions, then just set ‘inrange’ to ‘NULL’ and the 1203 range will be specified from the minimum and maximum 1204 value of the dataset (‘input’ cannot be ‘NULL’ in this 1205 case). 1206 1207 • If there is one element for each dimension in range, then 1208 it is viewed as a quantile (Q), and the range will be: ‘Q 1209 to 1-Q’. 1210 1211 • If there are two elements for each dimension in range, 1212 then they are assumed to be your desired minimum and 1213 maximum values. When either of the two are NaN, the 1214 minimum and maximum will be calculated for it. 1215 1216 ‘numbins’ 1217 The number of bins: must be larger than 0. 1218 1219 ‘onebinstart’ 1220 A desired value for onebinstart. Note that with this option, 1221 the bins won’t start and end exactly on the given range 1222 values, it will be slightly shifted to accommodate this 1223 request. 1224 1225 -- Function: 1226 gal_data_t * 1227 gal_statistics_histogram (gal_data_t *input, gal_data_t *bins, 1228 int normalize, int maxone) 1229 Make a histogram of all the elements in the given dataset with bin 1230 values that are defined in the ‘inbins’ structure (see 1231 ‘gal_statistics_regular_bins’, they currently have to be equally 1232 spaced). ‘inbins’ is not mandatory, if you pass a ‘NULL’ pointer, 1233 the bins structure will be built within this function based on the 1234 ‘numbins’ input. As a result, when you have already defined the 1235 bins, ‘numbins’ is not used. 1236 1237 Let’s write the center of the $i$th element of the bin array as 1238 $b_i$, and the fixed half-bin width as $h$. Then element $j$ of 1239 the input array ($in_j$) will be counted in $b_i$ if $(b_i-h) \le 1240 in_j < (b_i+h)$. However, if $in_j$ is somewhere in the last bin, 1241 the condition changes to $(b_i-h) \le in_j \le (b_i+h)$. 1242 1243 If ‘normalize!=0’, the histogram will be “normalized” such that the 1244 sum of the counts column will be one. In other words, all the 1245 counts in every bin will be divided by the total number of counts. 1246 If ‘maxone!=0’, the histogram’s maximum count will be 1. In other 1247 words, the counts in every bin will be divided by the value of the 1248 maximum. 1249 1250 -- Function: 1251 gal_data_t * 1252 gal_statistics_histogram2d (gal_data_t *input, gal_data_t 1253 *bins) 1254 This function is very similar to ‘gal_statistics_histogram’, but 1255 will build a 2D histogram (count how many of the elements of 1256 ‘input’ have a within a 2D box. The bins comprising the first 1257 dimension of the 2D box are defined by ‘bins’. The bins of the 1258 second dimension are defined by ‘bins->next’ (‘bins’ is a *note 1259 List of gal_data_t::). Both the ‘bin’ and ‘bin->next’ can be 1260 created with ‘gal_statistics_regular_bins’. 1261 1262 This function returns a list of ‘gal_data_t’ with three 1263 nodes/columns, so you can directly write them into a table (see 1264 *note Table input output::). Assuming ‘bins’ has $N1$ bins and 1265 ‘bins->next’ has $N2$ bins, each node/column of the returned output 1266 is a 1D array with $N1\times N2$ elements. The first and second 1267 columns are the center of the 2D bin along the first and second 1268 dimensions and have a ‘double’ data type. The third column is the 1269 2D histogram (the number of input elements that have a value within 1270 that 2D bin) and has a ‘uint32’ data type (see *note Numeric data 1271 types::). 1272 1273 -- Function: 1274 gal_data_t * 1275 gal_statistics_cfp (gal_data_t *input, gal_data_t *bins, int 1276 normalize) 1277 Make a cumulative frequency plot (CFP) of all the elements in 1278 ‘input’ with bin values that are defined in the ‘bins’ structure 1279 (see ‘gal_statistics_regular_bins’). 1280 1281 The CFP is built from the histogram: in each bin, the value is the 1282 sum of all previous bins in the histogram. Thus, if you have 1283 already calculated the histogram before calling this function, you 1284 can pass it onto this function as the data structure in 1285 ‘bins->next’ (see ‘List of gal_data_t’). If ‘bin->next!=NULL’, 1286 then it is assumed to be the histogram. If it is ‘NULL’, then the 1287 histogram will be calculated internally and freed after the job is 1288 finished. 1289 1290 When a histogram is given and it is normalized, the CFP will also 1291 be normalized (even if the normalized flag is not set here): note 1292 that a normalized CFP’s maximum value is 1. 1293 1294 -- Function: 1295 gal_data_t * 1296 gal_statistics_sigma_clip (gal_data_t *input, float multip, 1297 float param, int inplace, int quiet) 1298 Apply $\sigma$-clipping on a given dataset and return a dataset 1299 that contains the results. For a description of $\sigma$-clipping 1300 see *note Sigma clipping::. ‘multip’ is the multiple of the 1301 standard deviation (or $\sigma$, that is used to define outliers in 1302 each round of clipping). 1303 1304 The role of ‘param’ is determined based on its value. If ‘param’ 1305 is larger than ‘1’ (one), it must be an integer and will be 1306 interpreted as the number clips to do. If it is less than ‘1’ 1307 (one), it is interpreted as the tolerance level to stop the 1308 iteration. 1309 1310 The returned dataset (let’s call it ‘out’) contains a four-element 1311 array with type ‘GAL_TYPE_FLOAT32’. The final number of clips is 1312 stored in the ‘out->status’. 1313 float *array=out->array; 1314 array[0]: Number of points used. 1315 array[1]: Median. 1316 array[2]: Mean. 1317 array[3]: Standard deviation. 1318 1319 If the $\sigma$-clipping doesn’t converge or all input elements are 1320 blank, then this function will return NaN values for all the 1321 elements above. 1322 1323 -- Function: 1324 gal_data_t * 1325 gal_statistics_outlier_bydistance (int pos1_neg0, gal_data_t 1326 *input, size_t window_size, float sigma, float sigclip_multip, 1327 float sigclip_param, int inplace, int quiet) 1328 1329 Find the first positive outlier (if ‘pos1_neg0!=0’) in the ‘input’ 1330 distribution. When ‘pos1_neg0==0’, the same algorithm goes to the 1331 start of the dataset. The returned dataset contains a single 1332 element: the first positive outlier. It is one of the dataset’s 1333 elements, in the same type as the input. If the process fails for 1334 any reason (for example no outlier was found), a ‘NULL’ pointer 1335 will be returned. 1336 1337 All (possibly existing) blank elements are first removed from the 1338 input dataset, then it is sorted. A sliding window of 1339 ‘window_size’ elements is parsed over the dataset. Starting from 1340 the ‘window_size’-th element of the dataset, in the direction of 1341 increasing values. This window is used as a reference. The first 1342 element where the distance to the previous (sorted) element is 1343 ‘sigma’ units away from the distribution of distances in its window 1344 is considered an outlier and returned by this function. 1345 1346 Formally, if we assume there are $N$ non-blank elements. They are 1347 first sorted. Searching for the outlier starts on element $W$. 1348 Let’s take $v_i$ to be the $i$-th element of the sorted input (with 1349 no blank values) and $m$ and $\sigma$ as the $\sigma$-clipped 1350 median and standard deviation from the distances of the previous 1351 $W$ elements (not including $v_i$). If the value given to ‘sigma’ 1352 is displayed with $s$, the $i$-th element is considered as an 1353 outlier when the condition below is true. 1354 1355 $${(v_i-v_{i-1})-m\over \sigma}>s$$ 1356 1357 The ‘sigclip_multip’ and ‘sigclip_param’ arguments specify the 1358 properties of the $\sigma$-clipping (see *note Sigma clipping:: for 1359 more). You see that by this definition, the outlier cannot be any 1360 of the lower half elements. The advantage of this algorithm 1361 compared to $\sigma$-clippign is that it only looks backwards (in 1362 the sorted array) and parses it in one direction. 1363 1364 If ‘inplace!=0’, the removing of blank elements and sorting will be 1365 done within the input dataset’s allocated space. Otherwise, this 1366 function will internally allocate (and later free) the necessary 1367 space to keep the intermediate space that this process requires. 1368 1369 If ‘quiet!=0’, this function will report the parameters every time 1370 it moves the window as a separate line with several columns. The 1371 first column is the value, the second (in square brackets) is the 1372 sorted index, the third is the distance of this element from the 1373 previous one. The Fourth and fifth (in parenthesis) are the median 1374 and standard deviation of the $\sigma$-clipped distribution within 1375 the window and the last column is the difference between the third 1376 and fourth, divided by the fifth. 1377 1378 -- Function: 1379 gal_data_t * 1380 gal_statistics_outlier_flat_cfp (gal_data_t *input, size_t 1381 numprev, float sigclip_multip, float sigclip_param, float 1382 thresh, size_t numcontig, int inplace, int quiet, size_t 1383 *index) 1384 1385 Return the first element in the given dataset where the cumulative 1386 frequency plot first becomes significantly flat for a sufficient 1387 number of elements. The returned dataset only has one element 1388 (with the same type as the input). If ‘index!=NULL’, the index 1389 (counting from zero, after sorting the dataset and removing any 1390 blanks) is written in the space that ‘index’ points to. If no 1391 sufficiently flat portion is found, the returned pointer will be 1392 ‘NULL’. 1393 1394 The flatness on the cumulative frequency plot is defined like this 1395 (see *note Histogram and Cumulative Frequency Plot::): on the 1396 sorted dataset, for every point ($a_i$), we calculate 1397 $d_i=a_{i+2}-a_{i-2}$. This done on the first $N$ elements (value 1398 of ‘numprev’). After element $a_{N+2}$, we start estimating the 1399 flatness as follows: for every element we use the $N$, $d_i$ 1400 measurements before it as the reference. Let’s call this set $D_i$ 1401 for element $i$. The $\sigma$-clipped median ($m$) and standard 1402 deviation ($s$) of $D_i$ are then calculated. The 1403 $\sigma$-clipping can be configured with the two ‘sigclip_param’ 1404 and ‘sigclip_multip’ arguments. 1405 1406 Taking $t$ as the significance threshold (value to ‘thresh’), a 1407 point is considered flat when $a_i>m+t\sigma$. But a single point 1408 satisfying this condition will probably just be due to noise. To 1409 make a more robust estimate, this significance/condition has to 1410 hold for ‘numcontig’ contiguous elements after $a_i$. When this is 1411 satisfied, $a_i$ is returned as the point where the distribution’s 1412 cumulative frequency plot becomes flat. 1413 1414 To get a good estimate of $m$ and $s$, it is thus recommended to 1415 set ‘numprev’ as large as possible. However, be careful not to set 1416 it too high: the checks in the paragraph above are not done on the 1417 first ‘numprev’ elements and this function assumes the flatness 1418 occurs after them. Also, be sure that the value to ‘numcontig’ is 1419 much less than ‘numprev’, otherwise $\sigma$-clipping may not be 1420 able to remove the immediate outliers in $D_i$ near the boundary of 1421 the flat region. 1422 1423 When ‘quiet==0’, the basic measurements done on each element are 1424 printed on the command-line (good for finding the best parameters). 1425 When ‘inplace!=0’, the sorting and removal of blank elements is 1426 done on the input dataset, so the input may be altered after this 1427 function. 1428 1429 1430File: gnuastro.info, Node: Binary datasets, Next: Labeled datasets, Prev: Statistical operations, Up: Gnuastro library 1431 143211.3.23 Binary datasets (‘binary.h’) 1433------------------------------------ 1434 1435Binary datasets only have two (usable) values: 0 (also known as 1436background) or 1 (also known as foreground). They are created after 1437some binary classification is applied to the dataset. The most common 1438is thresholding: for example in an image, pixels with a value above the 1439threshold are given a value of 1 and those with a value less than the 1440threshold are assigned a value of 0. 1441 1442 Since there is only two values, in the processing of binary images, 1443you are usually concerned with the positioning of an element and its 1444vicinity (neighbors). When a dataset has more than one dimension, 1445multiple classes of immediate neighbors (that are touching the element) 1446can be defined for each data-element. To separate these different 1447classes of immediate neighbors, we define _connectivity_. 1448 1449 The classification is done by the distance from element center to the 1450neighbor’s center. The nearest immediate neighbors have a connectivity 1451of 1, the second nearest class of neighbors have a connectivity of 2 and 1452so on. In total, the largest possible connectivity for data with ‘ndim’ 1453dimensions is ‘ndim’. For example in a 2D dataset, 4-connected 1454neighbors (that share an edge and have a distance of 1 pixel) have a 1455connectivity of 1. The other 4 neighbors that only share a vertice 1456(with a distance of $\sqrt{2}$ pixels) have a connectivity of 2. 1457Conventionally, the class of connectivity-2 neighbors also includes the 1458connectivity 1 neighbors, so for example we call them 8-connected 1459neighbors in 2D datasets. 1460 1461 Ideally, one bit is sufficient for each element of a binary dataset. 1462However, CPUs are not designed to work on individual bits, the smallest 1463unit of memory addresses is a byte (containing 8 bits on modern CPUs). 1464Therefore, in Gnuastro, the type used for binary dataset is ‘uint8_t’ 1465(see *note Numeric data types::). Although it does take 8-times more 1466memory, this choice offers much better performance and the some extra 1467(useful) features. 1468 1469 The advantage of using a full byte for each element of a binary 1470dataset is that you can also have other values (that will be ignored in 1471the processing). One such common “other” value in real datasets is a 1472blank value (to mark regions that should not be processed because there 1473is no data). The constant ‘GAL_BLANK_UINT8’ value must be used in these 1474cases (see *note Library blank values::). Another is some temporary 1475value(s) that can be given to a processed pixel to avoid having another 1476copy of the dataset as in ‘GAL_BINARY_TMP_VALUE’ that is described 1477below. 1478 1479 -- Macro: GAL_BINARY_TMP_VALUE 1480 The functions described below work on a ‘uint8_t’ type dataset with 1481 values of 1 or 0 (no other pixel will be touched). However, in 1482 some cases, it is necessary to put temporary values in each element 1483 during the processing of the functions. This temporary value has a 1484 special meaning for the operation and will be operated on. So if 1485 your input datasets have values other than 0 and 1 that you don’t 1486 want these functions to work on, be sure they are not equal to this 1487 macro’s value. Note that this value is also different from 1488 ‘GAL_BLANK_UINT8’, so your input datasets may also contain blank 1489 elements. 1490 1491 -- Function: 1492 gal_data_t * 1493 gal_binary_erode (gal_data_t *input, size_t num, int 1494 connectivity, int inplace) 1495 Do ‘num’ erosions on the ‘connectivity’-connected neighbors of 1496 ‘input’ (see above for the definition of connectivity). 1497 1498 If ‘inplace’ is non-zero _and_ the input’s type is 1499 ‘GAL_TYPE_UINT8’, then the erosion will be done within the input 1500 dataset and the returned pointer will be ‘input’. Otherwise, 1501 ‘input’ is copied (and converted if necessary) to ‘GAL_TYPE_UINT8’ 1502 and erosion will be done on this new dataset which will also be 1503 returned. This function will only work on the elements with a 1504 value of 1 or 0. It will leave all the rest unchanged. 1505 1506 Erosion (inverse of dilation) is an operation in mathematical 1507 morphology where each foreground pixel that is touching a 1508 background pixel is flipped (changed to background). The 1509 ‘connectivity’ value determines the definition of “touching”. 1510 Erosion will thus decrease the area of the foreground regions by 1511 one layer of pixels. 1512 1513 -- Function: 1514 gal_data_t * 1515 gal_binary_dilate (gal_data_t *input, size_t num, int 1516 connectivity, int inplace) 1517 Do ‘num’ dilations on the ‘connectivity’-connected neighbors of 1518 ‘input’ (see above for the definition of connectivity). For more 1519 on ‘inplace’ and the output, see ‘gal_binary_erode’. 1520 1521 Dilation (inverse of erosion) is an operation in mathematical 1522 morphology where each background pixel that is touching a 1523 foreground pixel is flipped (changed to foreground). The 1524 ‘connectivity’ value determines the definition of “touching”. 1525 Dilation will thus increase the area of the foreground regions by 1526 one layer of pixels. 1527 1528 -- Function: 1529 gal_data_t * 1530 gal_binary_open (gal_data_t *input, size_t num, int 1531 connectivity, int inplace) 1532 Do ‘num’ openings on the ‘connectivity’-connected neighbors of 1533 ‘input’ (see above for the definition of connectivity). For more 1534 on ‘inplace’ and the output, see ‘gal_binary_erode’. 1535 1536 Opening is an operation in mathematical morphology which is defined 1537 as erosion followed by dilation (see above for the definitions of 1538 erosion and dilation). Opening will thus remove the outer 1539 structure of the foreground. In this implementation, ‘num’ 1540 erosions are going to be applied on the dataset, then ‘num’ 1541 dilations. 1542 1543 -- Function: 1544 size_t 1545 gal_binary_connected_components (gal_data_t *binary, 1546 gal_data_t **out, int connectivity) 1547 Return the number of connected components in ‘binary’ through the 1548 breadth first search algorithm (finding all pixels belonging to one 1549 component before going on to the next). Connection between two 1550 pixels is defined based on the value to ‘connectivity’. ‘out’ is a 1551 dataset with the same size as ‘binary’ with ‘GAL_TYPE_INT32’ type. 1552 Every pixel in ‘out’ will have the label of the connected component 1553 it belongs to. The labeling of connected components starts from 1, 1554 so a label of zero is given to the input’s background pixels. 1555 1556 When ‘*out!=NULL’ (its space is already allocated), it will be 1557 cleared (to zero) at the start of this function. Otherwise, when 1558 ‘*out==NULL’, the necessary dataset to keep the output will be 1559 allocated by this function. 1560 1561 ‘binary’ must have a type of ‘GAL_TYPE_UINT8’, otherwise this 1562 function will abort with an error. Other than blank pixels (with a 1563 value of ‘GAL_BLANK_UINT8’ defined in *note Library blank 1564 values::), all other non-zero pixels in ‘binary’ will be considered 1565 as foreground (and will be labeled). Blank pixels in the input 1566 will also be blank in the output. 1567 1568 -- Function: 1569 gal_data_t * 1570 gal_binary_connected_indexs(gal_data_t *binary, int 1571 connectivity) 1572 Build a ‘gal_data_t’ linked list, where each node of the list 1573 contains an array with indices of the connected regions. Therefore 1574 the arrays of each node can have a different size. Note that the 1575 indices will only be calculated on the pixels with a value of 1 and 1576 internally, it will temporarily change the values to 2 (and return 1577 them back to 1 in the end). 1578 1579 -- Function: 1580 gal_data_t * 1581 gal_binary_connected_adjacency_matrix (gal_data_t *adjacency, 1582 size_t *numconnected) 1583 Find the number of connected labels and new labels based on an 1584 adjacency matrix, which must be a square binary array (type 1585 ‘GAL_TYPE_UINT8’). The returned dataset is a list of new labels 1586 for each old label. In other words, this function will find the 1587 objects that are connected (possibly through a third object) and in 1588 the output array, the respective elements for all input labels is 1589 going to have the same value. The total number of connected labels 1590 is put into the space that ‘numconnected’ points to. 1591 1592 An adjacency matrix defines connection between two labels. For 1593 example, let’s assume we have 5 labels and we know that labels 1 1594 and 5 are connected to label 3, but are not connected with each 1595 other. Also, labels 2 and 4 are not touching any other label. So 1596 in total we have 3 final labels: one combined object (merged from 1597 labels 1, 3, and 5) and the initial labels 2 and 4. The input 1598 adjacency matrix would look like this (note the extra row and 1599 column for a label 0 which is ignored): 1600 1601 INPUT OUTPUT 1602 ===== ====== 1603 in_lab 1 2 3 4 5 | 1604 | numconnected = 3 1605 0 0 0 0 0 0 | 1606 in_lab 1 --> 0 0 0 1 0 0 | 1607 in_lab 2 --> 0 0 0 0 0 0 | Returned: new labels for the 1608 in_lab 3 --> 0 1 0 0 0 1 | 5 initial objects 1609 in_lab 4 --> 0 0 0 0 0 0 | | 0 | 1 | 2 | 1 | 3 | 1 | 1610 in_lab 5 --> 0 0 0 1 0 0 | 1611 1612 Although the adjacency matrix as used here is symmetric, currently 1613 this function assumes that it is filled on both sides of the 1614 diagonal. 1615 1616 -- Function: 1617 gal_data_t * 1618 gal_binary_connected_adjacency_list (gal_list_sizet_t 1619 **listarr, size_t number, size_t minmapsize, int quietmmap, 1620 size_t *numconnected) 1621 Find the number of connected labels and new labels based on an 1622 adjacency list. The output of this function is identical to that 1623 of ‘gal_binary_connected_adjacency_matrix’. But the major 1624 difference is that it uses a list of connected labels to each label 1625 instead of a square adjacency matrix. This is done because when 1626 the number of labels becomes very large (for example on the scale 1627 of 100,000), the adjacency matrix can consume more than 10GB of 1628 RAM! 1629 1630 The input list has the following format: it is an array of pointers 1631 to ‘gal_list_sizet_t *’ (or ‘gal_list_sizet_t **’). The array has 1632 ‘number’ elements and each ‘listarr[i]’ is a linked list of 1633 ‘gal_list_sizet_t *’. As a demonstration, the input of the same 1634 example in ‘gal_binary_connected_adjacency_matrix’ would look like 1635 below and the output of this function will be identical to there. 1636 1637 listarr[0] = NULL 1638 listarr[1] = 3 1639 listarr[2] = NULL 1640 listarr[3] = 1 -> 5 1641 listarr[4] = NULL 1642 listarr[5] = 3 1643 1644 From this example, it is already clear that this method will 1645 consume far less memory. But because it needs to parse lists (and 1646 not easily jump between array elements), it can be slower. But in 1647 scenarios where there are too many objects (that may exceed the 1648 whole system’s RAM+SWAP), this option is a good alternative and the 1649 drop in processing speed is worth getting the job done. 1650 1651 Similar to ‘gal_binary_connected_adjacency_matrix’, this function 1652 will write the final number of connected labels in ‘numconnected’. 1653 But since it takes no ‘gal_data_t *’ argument (where it can inherit 1654 the ‘minmapsize’ and ‘quietmmap’ parameters), it also needs these 1655 as input. For more on ‘minmapsize’ and ‘quietmmap’, see *note 1656 Memory management::. 1657 1658 -- Function: 1659 gal_data_t * 1660 gal_binary_holes_label (gal_data_t *input, int connectivity, 1661 size_t *numholes) 1662 Label all the holes in the foreground (non-zero elements in input) 1663 as independent regions. Holes are background regions (zero-valued 1664 in input) that are fully surrounded by the foreground, as defined 1665 by ‘connectivity’. The returned dataset has a 32-bit signed 1666 integer type with the size of the input. All holes in the input 1667 will have labels/counters greater or equal to ‘1’. The rest of the 1668 background regions will still have a value of ‘0’ and the initial 1669 foreground pixels will have a value of ‘-1’. The total number of 1670 holes will be written where ‘numholes’ points to. 1671 1672 -- Function: 1673 void 1674 gal_binary_holes_fill (gal_data_t *input, int connectivity, 1675 size_t maxsize) 1676 Fill all the holes (0 valued pixels surrounded by 1 valued pixels) 1677 of the binary ‘input’ dataset. The connectivity of the holes can 1678 be set with ‘connectivity’. Holes larger than ‘maxsize’ are not 1679 filled. This function currently only works on a 2D dataset. 1680 1681 1682File: gnuastro.info, Node: Labeled datasets, Next: Convolution functions, Prev: Binary datasets, Up: Gnuastro library 1683 168411.3.24 Labeled datasets (‘label.h’) 1685------------------------------------ 1686 1687A labeled dataset is one where each element/pixel has an integer label 1688(or counter). The label identifies the group/class that the element 1689belongs to. This form of labeling allows the higher-level study of all 1690pixels within a certain class. 1691 1692 For example, to detect objects/targets in an image/dataset, you can 1693apply a threshold to separate the noise from the signal (to detect 1694diffuse signal, a threshold is useless and more advanced methods are 1695necessary, for example *note NoiseChisel::). But the output of 1696detection is a binary dataset (which is just a very low-level labeling 1697of ‘0’ for noise and ‘1’ for signal). 1698 1699 The raw detection map is therefore hardly useful for any kind of 1700analysis on objects/targets in the image. One solution is to use a 1701connected-components algorithm (see ‘gal_binary_connected_components’ in 1702*note Binary datasets::). It is a simple and useful way to 1703separate/label connected patches in the foreground. This higher-level 1704(but still elementary) labeling therefore allows you to count how many 1705connected patches of signal there are in the dataset and is a major 1706improvement compared to the raw detection. 1707 1708 However, when your objects/targets are touching, the simple connected 1709components algorithm is not enough and a still higher-level labeling 1710mechanism is necessary. This brings us to the necessity of the 1711functions in this part of Gnuastro’s library. The main inputs to the 1712functions in this section are already labeled datasets (for example with 1713the connected components algorithm above). 1714 1715 Each of the labeled regions are independent of each other (the labels 1716specify different classes of targets). Therefore, especially in large 1717datasets, it is often useful to process each label on independent CPU 1718threads in parallel rather than in series. Therefore the functions of 1719this section actually use an array of pixel/element indices (belonging 1720to each label/class) as the main identifier of a region. Using indices 1721will also allow processing of overlapping labels (for example in 1722deblending problems). Just note that overlapping labels are not yet 1723implemented, but planned. You can use ‘gal_label_indexs’ to generate 1724lists of indices belonging to separate classes from the labeled input. 1725 1726 -- Macro: GAL_LABEL_INIT 1727 -- Macro: GAL_LABEL_RIVER 1728 -- Macro: GAL_LABEL_TMPCHECK 1729 Special negative integer values used internally by some of the 1730 functions in this section. Recall that meaningful labels are 1731 considered to be positive integers ($\geq1$). Zero is 1732 conventionally kept for regions with no labels, therefore negative 1733 integers can be used for any extra classification in the labeled 1734 datasets. 1735 1736 -- Function: 1737 gal_data_t * 1738 gal_label_indexs (gal_data_t *labels, size_t numlabs, size_t 1739 minmapsize, int quietmmap) 1740 1741 Return an array of ‘gal_data_t’ containers, each containing the 1742 pixel indices of the respective label (see *note Generic data 1743 container::). ‘labels’ contains the label of each element and has 1744 to have an ‘GAL_TYPE_INT32’ type (see *note Library data types::). 1745 Only positive (greater than zero) values in ‘labels’ will be 1746 used/indexed, other elements will be ignored. 1747 1748 Meaningful labels start from ‘1’ and not ‘0’, therefore the output 1749 array of ‘gal_data_t’ will contain ‘numlabs+1’ elements. The first 1750 (zero-th) element of the output (‘indexs[0]’ in the example below) 1751 will be initialized to a dataset with zero elements. This will 1752 allow easy (non-confusing) access to the indices of each 1753 (meaningful) label. 1754 1755 ‘numlabs’ is the number of labels in the dataset. If it is given a 1756 value of zero, then the maximum value in the input (largest label) 1757 will be found and used. Therefore if it is given, but smaller than 1758 the actual number of labels, this function may/will crash (it will 1759 write in unallocated space). ‘numlabs’ is therefore useful in a 1760 highly optimized/checked environment. 1761 1762 For example, if the returned array is called ‘indexs’, then 1763 ‘indexs[10].size’ contains the number of elements that have a label 1764 of ‘10’ in ‘labels’ and ‘indexs[10].array’ is an array (after 1765 casting to ‘size_t *’) containing the indices of each one of those 1766 elements/pixels. 1767 1768 By _index_ we mean the 1D position: the input number of dimensions 1769 is irrelevant (any dimensionality is supported). In other words, 1770 each element’s index is the number of elements/pixels between it 1771 and the dataset’s first element/pixel. Therefore it is always 1772 greater or equal to zero and stored in ‘size_t’ type. 1773 1774 -- Function: 1775 size_t 1776 gal_label_watershed (gal_data_t *values, gal_data_t *indexs, 1777 gal_data_t *label, size_t *topinds, int min0_max1) 1778 Use the watershed algorithm(1) to “over-segment” the pixels in the 1779 ‘indexs’ dataset based on values in the ‘values’ dataset. 1780 Internally, each local extrema (maximum or minimum, based on 1781 ‘min0_max1’) and its surrounding pixels will be given a unique 1782 label. For demonstration, see Figures 8 and 9 of Akhlaghi and 1783 Ichikawa [2015] (http://arxiv.org/abs/1505.01664). If 1784 ‘topinds!=NULL’, it is assumed to point to an already allocated 1785 space to write the index of each clump’s local extrema, otherwise, 1786 it is ignored. 1787 1788 The ‘values’ dataset must have a 32-bit floating point type 1789 (‘GAL_TYPE_FLOAT32’, see *note Library data types::) and will only 1790 be read by this function. ‘indexs’ must contain the indices of the 1791 elements/pixels that will be over-segmented by this function and 1792 have a ‘GAL_TYPE_SIZE_T’ type, see the description of 1793 ‘gal_label_indexs’, above. The final labels will be written in the 1794 respective positions of ‘labels’, which must have a 1795 ‘GAL_TYPE_INT32’ type and be the same size as ‘values’. 1796 1797 When ‘indexs’ is already sorted, this function will ignore 1798 ‘min0_max1’. To judge if the dataset is sorted or not (by the 1799 values the indices correspond to in ‘values’, not the actual 1800 indices), this function will look into the bits of ‘indexs->flag’, 1801 for the respective bit flags, see *note Generic data container::. 1802 If ‘indexs’ is not already sorted, this function will sort it 1803 according to the values of the respective pixel in ‘values’. The 1804 increasing/decreasing order will be determined by ‘min0_max1’. 1805 Note that if this function is called on multiple threads _and_ 1806 ‘values’ points to a different array on each thread, this function 1807 will not return a reasonable result. In this case, please sort 1808 ‘indexs’ prior to calling this function (see 1809 ‘gal_qsort_index_multi_d’ in *note Qsort functions::). 1810 1811 When ‘indexs’ is decreasing (increasing), or ‘min0_max1’ is ‘1’ 1812 (‘0’), local minima (maxima), are considered rivers (watersheds) 1813 and given a label of ‘GAL_LABEL_RIVER’ (see above). 1814 1815 Note that rivers/watersheds will also be formed on the edges of the 1816 labeled regions or when the labeled pixels touch a blank pixel. 1817 Therefore this function will need to check for the presence of 1818 blank values. To be most efficient, it is thus recommended to use 1819 ‘gal_blank_present’ (with ‘updateflag=1’) prior to calling this 1820 function (see *note Library blank values::. Once the flag has been 1821 set, no other function (including this one) that needs special 1822 behavior for blank pixels will have to parse the dataset to see if 1823 it has blank values any more. 1824 1825 If you are sure your dataset doesn’t have blank values (by the 1826 design of your software), to avoid an extra parsing of the dataset 1827 and improve performance, you can set the two bits manually (see the 1828 description of ‘flags’ in *note Generic data container::): 1829 input->flag |= GAL_DATA_FLAG_BLANK_CH; /* Set bit to 1. */ 1830 input->flag &= ~GAL_DATA_FLAG_HASBLANK; /* Set bit to 0. */ 1831 1832 -- Function: 1833 void 1834 gal_label_clump_significance (gal_data_t *values, gal_data_t 1835 *std, gal_data_t *label, gal_data_t *indexs, struct 1836 gal_tile_two_layer_params *tl, size_t numclumps, size_t 1837 minarea, int variance, int keepsmall, gal_data_t *sig, 1838 gal_data_t *sigind) 1839 This function is usually called after ‘gal_label_watershed’, and is 1840 used as a measure to identify which over-segmented “clumps” are 1841 real and which are noise. 1842 1843 A measurement is done on each clump (using the ‘values’ and ‘std’ 1844 datasets, see below). To help in multi-threaded environments, the 1845 operation is only done on pixels which are indexed in ‘indexs’. It 1846 is expected for ‘indexs’ to be sorted by their values in ‘values’. 1847 If not sorted, the measurement may not be reliable. If sorted in a 1848 decreasing order, then clump building will start from their highest 1849 value and vice-versa. See the description of ‘gal_label_watershed’ 1850 for more on ‘indexs’. 1851 1852 Each “clump” (identified by a positive integer) is assumed to be 1853 surrounded by at least one river/watershed pixel (with a 1854 non-positive label). This function will parse the pixels 1855 identified in ‘indexs’ and make a measurement on each clump and 1856 over all the river/watershed pixels. The number of clumps 1857 (‘numclumps’) must be given as an input argument and any clump that 1858 is smaller than ‘minarea’ is ignored (because of scatter). If 1859 ‘variance’ is non-zero, then the ‘std’ dataset is interpreted as 1860 variance, not standard deviation. 1861 1862 The ‘values’ and ‘std’ datasets must have a ‘float’ (32-bit 1863 floating point) type. Also, ‘label’ and ‘indexs’ must respectively 1864 have ‘int32’ and ‘size_t’ types. ‘values’ and ‘label’ must have 1865 the same size, but ‘std’ can have three possible sizes: 1) a single 1866 element (which will be used for the whole dataset, 2) the same size 1867 as ‘values’ (so a different error can be assigned to every pixel), 1868 3) a single value for each tile, based on the ‘tl’ tessellation 1869 (see *note Tile grid::). In the last case, a tile/value will be 1870 associated to each clump based on its flux-weighted (only positive 1871 values) center. 1872 1873 The main output is an internally allocated, 1-dimensional array 1874 with one value per label. The array information (length, type, 1875 etc) will be written into the ‘sig’ generic data container. 1876 Therefore ‘sig->array’ must be ‘NULL’ when this function is called. 1877 After this function, the details of the array (number of elements, 1878 type and size, etc) will be written in to the various components of 1879 ‘sig’, see the definition of ‘gal_data_t’ in *note Generic data 1880 container::. Therefore ‘sig’ must already be allocated before 1881 calling this function. 1882 1883 Optionally (when ‘sigind!=NULL’, similar to ‘sig’) the clump labels 1884 of each measurement in ‘sig’ will be written in ‘sigind->array’. 1885 If ‘keepsmall’ zero, small clumps (where no measurement is made) 1886 will not be included in the output table. 1887 1888 This function is initially intended for a multi-threaded 1889 environment. In such cases, you will be writing arrays of clump 1890 measures from different regions in parallel into an array of 1891 ‘gal_data_t’s. You can simply allocate (and initialize), such an 1892 array with the ‘gal_data_array_calloc’ function in *note Arrays of 1893 datasets::. For example if the ‘gal_data_t’ array is called 1894 ‘array’, you can pass ‘&array[i]’ as ‘sig’. 1895 1896 Along with some other functions in ‘label.h’, this function was 1897 initially written for *note Segment::. The description of the 1898 parameter used to measure a clump’s significance is fully given in 1899 Akhlaghi [2019] (https://arxiv.org/abs/1909.11230). 1900 1901 -- Function: 1902 void 1903 gal_label_grow_indexs (gal_data_t *labels, gal_data_t *indexs, 1904 int withrivers, int connectivity) 1905 Grow the (positive) labels of ‘labels’ over the pixels in ‘indexs’ 1906 (see description of ‘gal_label_indexs’). The pixels (position in 1907 ‘indexs’, values in ‘labels’) that must be “grown” must have a 1908 value of ‘GAL_LABEL_INIT’ in ‘labels’ before calling this function. 1909 For a demonstration see Columns 2 and 3 of Figure 10 in Akhlaghi 1910 and Ichikawa [2015] (http://arxiv.org/abs/1505.01664). 1911 1912 In many aspects, this function is very similar to over-segmentation 1913 (watershed algorithm, ‘gal_label_watershed’). The big difference 1914 is that in over-segmentation local maximums (that aren’t touching 1915 any alreadylabeled pixel) get a separate label. However, here the 1916 final number of labels will not change. All pixels that aren’t 1917 directly touching a labeled pixel just get pushed back to the start 1918 of the loop, and the loop iterates until its size doesn’t change 1919 any more. This is because in a generic scenario some of the 1920 indexed pixels might not be reachable through other indexed pixels. 1921 1922 The next major difference with over-segmentation is that when there 1923 is only one label in growth region(s), it is not mandatory for 1924 ‘indexs’ to be sorted by values. If there are multiple labeled 1925 regions in growth region(s), then values are important and you can 1926 use ‘qsort’ with ‘gal_qsort_index_single_d’ to sort the indices by 1927 values in a separate array (see *note Qsort functions::). 1928 1929 This function looks for positive-valued neighbors of each pixel in 1930 ‘indexs’ and will label a pixel if it touches one. Therefore, it 1931 is very important that only pixels/labels that are intended for 1932 growth have positive values in ‘labels’ before calling this 1933 function. Any non-positive (zero or negative) value will be 1934 ignored as a label by this function. Thus, it is recommended that 1935 while filling in the ‘indexs’ array values, you initialize all the 1936 pixels that are in ‘indexs’ with ‘GAL_LABEL_INIT’, and set 1937 non-labeled pixels that you don’t want to grow to ‘0’. 1938 1939 This function will write into both the input datasets. After this 1940 function, some of the non-positive ‘labels’ pixels will have a new 1941 positivelabel and the number of useful elements in ‘indexs’ will 1942 have decreased. The index of those pixels that couldn’t be labeled 1943 will remain inside ‘indexs’. If ‘withrivers’ is non-zero, then 1944 pixels that are immediately touching more than one positive value 1945 will be given a ‘GAL_LABEL_RIVER’ label. 1946 1947 Note that the ‘indexs->array’ is not re-allocated to its new size 1948 at the end(2). But since ‘indexs->dsize[0]’ and ‘indexs->size’ 1949 have new values after this function is returned, the extra elements 1950 just won’t be used until they are ultimately freed by 1951 ‘gal_data_free’. 1952 1953 Connectivity is a value between ‘1’ (fewest number of neighbors) 1954 and the number of dimensions in the input (most number of 1955 neighbors). For example in a 2D dataset, a connectivity of ‘1’ and 1956 ‘2’ corresponds to 4-connected and 8-connected neighbors. 1957 1958 ---------- Footnotes ---------- 1959 1960 (1) The watershed algorithm was initially introduced by Vincent and 1961Soille (https://doi.org/10.1109/34.87344). It starts from the minima 1962and puts the pixels in, one by one, to grow them until the touch (create 1963a watershed). For more, also see the Wikipedia article: 1964<https://en.wikipedia.org/wiki/Watershed_%28image_processing%29>. 1965 1966 (2) Note that according to the GNU C Library, even a ‘realloc’ to a 1967smaller size can also cause a re-write of the whole array, which is not 1968a cheap operation. 1969 1970 1971File: gnuastro.info, Node: Convolution functions, Next: Interpolation, Prev: Labeled datasets, Up: Gnuastro library 1972 197311.3.25 Convolution functions (‘convolve.h’) 1974-------------------------------------------- 1975 1976Convolution is a very common operation during data analysis and is 1977thoroughly described as part of Gnuastro’s *note Convolve:: program 1978which is fully devoted to this job. Because of the complete 1979introduction that was presented there, we will directly skip onto the 1980currently available convolution functions in Gnuastro’s library. 1981 1982 As of this version, only spatial domain convolution is available in 1983Gnuastro’s libraries. We haven’t had the time to liberate the frequency 1984domain function convolution and de-convolution functions that are 1985available in the Convolve program(1). 1986 1987 -- Function: 1988 gal_data_t * 1989 gal_convolve_spatial (gal_data_t *tiles, gal_data_t *kernel, 1990 size_t numthreads, int edgecorrection, int convoverch) 1991 Convolve the given ‘tiles’ dataset (possibly a list of tiles, see 1992 *note List of gal_data_t:: and *note Tessellation library::) with 1993 ‘kernel’ on ‘numthreads’ threads. When ‘edgecorrection’ is 1994 non-zero, it will correct for the edge dimming effects as discussed 1995 in *note Edges in the spatial domain::. 1996 1997 ‘tiles’ can be a single/complete dataset, but in that case the 1998 speed will be very slow. Therefore, for larger images, it is 1999 recommended to give a list of tiles covering a dataset. To create 2000 a tessellation that fully covers an input image, you may use 2001 ‘gal_tile_full’, or ‘gal_tile_full_two_layers’ to also define 2002 channels over your input dataset. These functions are discussed in 2003 *note Tile grid::. You may then pass the list of tiles to this 2004 function. This is the recommended way to call this function 2005 because spatial domain convolution is slow and breaking the job 2006 into many small tiles and working on simultaneously on several 2007 threads can greatly speed up the processing. 2008 2009 If the tiles are defined within a channel (a larger tile), by 2010 default convolution will be done within the channel, so pixels on 2011 the edge of a channel will not be affected by their neighbors that 2012 are in another channel. See *note Tessellation:: for the necessity 2013 of channels in astronomical data analysis. This behavior may be 2014 disabled when ‘convoverch’ is non-zero. In this case, it will 2015 ignore channel borders (if they exist) and mix all pixels that 2016 cover the kernel within the dataset. 2017 2018 -- Function: 2019 void 2020 gal_convolve_spatial_correct_ch_edge (gal_data_t *tiles, 2021 gal_data_t *kernel, size_t numthreads, int edgecorrection, 2022 gal_data_t *tocorrect) 2023 Correct the edges of channels in an already convolved image when it 2024 was initially convolved with ‘gal_convolve_spatial’ and 2025 ‘convoverch==0’. In that case, strong boundaries might exist on 2026 the channel edges. So if you later need to remove those boundaries 2027 at later steps of your processing, you can call this function. It 2028 will only do convolution on the tiles that are near the edge and 2029 were effected by the channel borders. Other pixels in the image 2030 will not be touched. Hence, it is much faster. 2031 2032 ---------- Footnotes ---------- 2033 2034 (1) Hence any help would be greatly appreciated. 2035 2036 2037File: gnuastro.info, Node: Interpolation, Next: Git wrappers, Prev: Convolution functions, Up: Gnuastro library 2038 203911.3.26 Interpolation (‘interpolate.h’) 2040--------------------------------------- 2041 2042During data analysis, it happens that parts of the data cannot be given 2043a value, but one is necessary for the higher-level analysis. For 2044example a very bright star saturated part of your image and you need to 2045fill in the saturated pixels with some values. Another common usage 2046case are masked sky-lines in 1D spectra that similarly need to be 2047assigned a value for higher-level analysis. In other situations, you 2048might want a value in an arbitrary point: between the elements/pixels 2049where you have data. The functions described in this section are for 2050such operations. 2051 2052 The parametric interpolations discussed below are wrappers around the 2053interpolation functions of the GNU Scientific Library (or GSL, see *note 2054GNU Scientific Library::). To identify the different GSL interpolation 2055types, Gnuastro’s ‘gnuastro/interpolate.h’ header file contains macros 2056that are discussed below. The GSL wrappers provided here are not yet 2057complete because we are too busy. If you need them, please consider 2058helping us in adding them to Gnuastro’s library. Your would be very 2059welcome and appreciated. 2060 2061 -- Macro: GAL_INTERPOLATE_NEIGHBORS_METRIC_RADIAL 2062 -- Macro: GAL_INTERPOLATE_NEIGHBORS_METRIC_MANHATTAN 2063 -- Macro: GAL_INTERPOLATE_NEIGHBORS_METRIC_INVALID 2064 The metric used to find distance for nearest neighbor 2065 interpolation. A radial metric uses the simple Euclidean function 2066 to find the distance between two pixels. A manhattan metric will 2067 always be an integer and is like steps (but is also much faster to 2068 calculate than radial metric because it doesn’t need a square root 2069 calculation). 2070 2071 -- Macro: GAL_INTERPOLATE_NEIGHBORS_FUNC_MIN 2072 -- Macro: GAL_INTERPOLATE_NEIGHBORS_FUNC_MAX 2073 -- Macro: GAL_INTERPOLATE_NEIGHBORS_FUNC_MEDIAN 2074 -- Macro: GAL_INTERPOLATE_NEIGHBORS_FUNC_INVALID 2075 The various types of nearest-neighbor interpolation functions for 2076 ‘gal_interpolate_neighbors’. The names are descriptive for the 2077 operation they do, so we won’t go into much more detail here. The 2078 median operator will be one of the most used, but operators like 2079 the maximum are good to fill the center of saturated stars. 2080 2081 -- Function: 2082 gal_data_t * 2083 gal_interpolate_neighbors (gal_data_t *input, struct 2084 gal_tile_two_layer_params *tl, uint8_t metric, size_t 2085 numneighbors, size_t numthreads, int onlyblank, int 2086 aslinkedlist, int function) 2087 2088 Interpolate the values in the input dataset using a calculated 2089 statistics from the distribution of their ‘numneighbors’ closest 2090 neighbors. The desired statistics is determined from the ‘func’ 2091 argument, which takes any of the ‘GAL_INTERPOLATE_NEIGHBORS_FUNC_’ 2092 macros (see above). This function is non-parametric and thus 2093 agnostic to the input’s number of dimension or shape of the 2094 distribution. 2095 2096 Distance can be defined on different metrics that are identified 2097 through ‘metric’ (taking values determined by the 2098 ‘GAL_INTERPOLATE_NEIGHBORS_METRIC_’ macros described above). If 2099 ‘onlyblank’ is non-zero, then only blank elements will be 2100 interpolated and pixels that already have a value will be left 2101 untouched. This function is multi-threaded and will run on 2102 ‘numthreads’ threads (see ‘gal_threads_number’ in *note 2103 Multithreaded programming::). 2104 2105 ‘tl’ is Gnuastro’s tessellation structure used to define tiles over 2106 an image and is fully described in *note Tile grid::. When 2107 ‘tl!=NULL’, then it is assumed that the ‘input->array’ contains one 2108 value per tile and interpolation will respect certain tessellation 2109 properties, for example to not interpolate over channel borders. 2110 2111 If several datasets have the same set of blank values, you don’t 2112 need to call this function multiple times. When ‘aslinkedlist’ is 2113 non-zero, then ‘input’ will be seen as a *note List of 2114 gal_data_t::. In this case, the same neighbors will be used for 2115 all the datasets in the list. Of course, the values for each 2116 dataset will be different, so a different value will be written in 2117 the each dataset, but the neighbor checking that is the most CPU 2118 intensive part will only be done once. 2119 2120 This is a non-parametric and robust function for interpolation. 2121 The interpolated values are also always within the range of the 2122 non-blank values and strong outliers do not get created. However, 2123 this type of interpolation must be used with care when there are 2124 gradients. This is because it is non-parametric and if there 2125 aren’t enough neighbors, step-like features can be created. 2126 2127 -- Macro: GAL_INTERPOLATE_1D_INVALID 2128 This is just a place holder to manage errors. 2129 -- Macro: GAL_INTERPOLATE_1D_LINEAR 2130 [From GSL:] Linear interpolation. This interpolation method does 2131 not require any additional memory. 2132 -- Macro: GAL_INTERPOLATE_1D_POLYNOMIAL 2133 [From GSL:] Polynomial interpolation. This method should only be 2134 used for interpolating small numbers of points because polynomial 2135 interpolation introduces large oscillations, even for well-behaved 2136 datasets. The number of terms in the interpolating polynomial is 2137 equal to the number of points. 2138 -- Macro: GAL_INTERPOLATE_1D_CSPLINE 2139 [From GSL:] Cubic spline with natural boundary conditions. The 2140 resulting curve is piecewise cubic on each interval, with matching 2141 first and second derivatives at the supplied data-points. The 2142 second derivative is chosen to be zero at the first point and last 2143 point. 2144 -- Macro: GAL_INTERPOLATE_1D_CSPLINE_PERIODIC 2145 [From GSL:] Cubic spline with periodic boundary conditions. The 2146 resulting curve is piecewise cubic on each interval, with matching 2147 first and second derivatives at the supplied data-points. The 2148 derivatives at the first and last points are also matched. Note 2149 that the last point in the data must have the same y-value as the 2150 first point, otherwise the resulting periodic interpolation will 2151 have a discontinuity at the boundary. 2152 -- Macro: GAL_INTERPOLATE_1D_AKIMA 2153 [From GSL:] Non-rounded Akima spline with natural boundary 2154 conditions. This method uses the non-rounded corner algorithm of 2155 Wodicka. 2156 -- Macro: GAL_INTERPOLATE_1D_AKIMA_PERIODIC 2157 [From GSL:] Non-rounded Akima spline with periodic boundary 2158 conditions. This method uses the non-rounded corner algorithm of 2159 Wodicka. 2160 -- Macro: GAL_INTERPOLATE_1D_STEFFEN 2161 [From GSL:] Steffen’s method(1) guarantees the monotonicity of the 2162 interpolating function between the given data points. Therefore, 2163 minima and maxima can only occur exactly at the data points, and 2164 there can never be spurious oscillations between data points. The 2165 interpolated function is piecewise cubic in each interval. The 2166 resulting curve and its first derivative are guaranteed to be 2167 continuous, but the second derivative may be discontinuous. 2168 2169 -- Function: 2170 gsl_spline * 2171 gal_interpolate_1d_make_gsl_spline (gal_data_t *X, gal_data_t 2172 *Y, int type_1d) 2173 Allocate and initialize a GNU Scientific Library (GSL) 1D 2174 ‘gsl_spline’ structure using the non-blank elements of ‘Y’. 2175 ‘type_1d’ identifies the interpolation scheme and must be one of 2176 the ‘GAL_INTERPOLATE_1D_*’ macros defined above. 2177 2178 If ‘X==NULL’, the X-axis is assumed to be integers starting from 2179 zero (the index of each element in ‘Y’). Otherwise, the values in 2180 ‘X’ will be used to initialize the interpolation structure. Note 2181 that when given, ‘X’ must _not_ contain any blank elements and it 2182 must be sorted (in increasing order). 2183 2184 Each interpolation scheme needs a minimum number of elements to 2185 successfully operate. If the number of non-blank values in ‘Y’ is 2186 less than this number, this function will return a ‘NULL’ pointer. 2187 2188 To be as generic and modular as possible, GSL’s tools are 2189 low-level. Therefore before doing the interpolation, many steps 2190 are necessary (like preparing your dataset, then allocating and 2191 initializing ‘gsl_spline’). The metadata available in Gnuastro’s 2192 *note Generic data container:: make it easy to hide all those 2193 preparations within this function. 2194 2195 Once ‘gsl_spline’ has been initialized by this function, the 2196 interpolation can be evaluated for any X value within the non-blank 2197 range of the input using ‘gsl_spline_eval’ or ‘gsl_spline_eval_e’. 2198 2199 For example in the small program below, we read the first two 2200 columns of the table in ‘table.txt’ and feed them to this function 2201 to later estimate the values in the second column for three 2202 selected points. You can use *note BuildProgram:: to compile and 2203 run this function, see *note Library demo programs:: for more. 2204 2205 #include <stdio.h> 2206 #include <stdlib.h> 2207 #include <gnuastro/table.h> 2208 #include <gnuastro/interpolate.h> 2209 2210 int 2211 main(void) 2212 { 2213 size_t i; 2214 gal_data_t *X, *Y; 2215 gsl_spline *spline; 2216 gsl_interp_accel *acc; 2217 gal_list_str_t *cols=NULL; 2218 2219 /* Change the values based on your input table. */ 2220 double points[]={1.8, 2.5, 10.3}; 2221 2222 /* Read the first two columns from `tab.txt'. 2223 IMPORTANT: the list is first-in-first-out, so the output 2224 column order is the inverse of the input order. */ 2225 gal_list_str_add(&cols, "1", 0); 2226 gal_list_str_add(&cols, "2", 0); 2227 Y=gal_table_read("table.txt", NULL, cols, GAL_TABLE_SEARCH_NAME, 2228 0, -1, 1, NULL); 2229 X=Y->next; 2230 2231 /* Allocate the GSL interpolation accelerator and make the 2232 `gsl_spline' structure. */ 2233 acc=gsl_interp_accel_alloc(); 2234 spline=gal_interpolate_1d_make_gsl_spline(X, Y, 2235 GAL_INTERPOLATE_1D_STEFFEN); 2236 2237 /* Calculate the respective value for all the given points, 2238 if `spline' could be allocated. */ 2239 if(spline) 2240 for(i=0; i<(sizeof points)/(sizeof *points); ++i) 2241 printf("%f: %f\n", points[i], 2242 gsl_spline_eval(spline, points[i], acc)); 2243 2244 /* Clean up and return. */ 2245 gal_data_free(X); 2246 gal_data_free(Y); 2247 gsl_spline_free(spline); 2248 gsl_interp_accel_free(acc); 2249 gal_list_str_free(cols, 0); 2250 return EXIT_SUCCESS; 2251 } 2252 2253 -- Function: 2254 void 2255 gal_interpolate_1d_blank (gal_data_t *in, int type_1d) 2256 Fill the blank elements of ‘in’ using the rest of the elements and 2257 the given interpolation. The interpolation scheme can be set 2258 through ‘type_1d’, which accepts any of the ‘GAL_INTERPOLATE_1D_*’ 2259 macros above. The interpolation is internally done in 64-bit 2260 floating point type (‘double’). However the evaluated/interpolated 2261 values (originally blank) will be written (in ‘in’) with its 2262 original numeric datatype, using C’s standard type conversion. 2263 2264 By definition, interpolation is only defined “between” valid 2265 points. Therefore, if any number of elements on the start or end 2266 of the 1D array are blank, those elements will not be interpolated 2267 and will remain blank. To see if any blank (non-interpolated) 2268 elements remain, you can use ‘gal_blank_present’ on ‘in’ after this 2269 function is finished. 2270 2271 ---------- Footnotes ---------- 2272 2273 (1) <http://adsabs.harvard.edu/abs/1990A%26A...239..443S> 2274 2275 2276File: gnuastro.info, Node: Git wrappers, Next: Unit conversion library, Prev: Interpolation, Up: Gnuastro library 2277 227811.3.27 Git wrappers (‘git.h’) 2279------------------------------ 2280 2281Git is one of the most common tools for version control and it can often 2282be useful during development, for example see ‘COMMIT’ keyword in *note 2283Output FITS files::. At installation time, Gnuastro will also check for 2284the existence of libgit2, and store the value in the 2285‘GAL_CONFIG_HAVE_LIBGIT2’, see *note Configuration information:: and 2286*note Optional dependencies::. ‘gnuastro/git.h’ includes 2287‘gnuastro/config.h’ internally, so you won’t have to include both for 2288this macro. 2289 2290 -- Function: 2291 char * 2292 gal_git_describe ( ) 2293 When libgit2 is present and the program is called within a 2294 directory that is version controlled, this function will return a 2295 string containing the commit description (similar to Gnuastro’s 2296 unofficial version number, see *note Version numbering::). If 2297 there are uncommitted changes in the running directory, it will add 2298 a ‘‘-dirty’’ prefix to the description. When there is no tagged 2299 point in the previous commit, this function will return a uniquely 2300 abbreviated commit object as fallback. This function is used for 2301 generating the value of the ‘COMMIT’ keyword in *note Output FITS 2302 files::. The output string is similar to the output of the 2303 following command: 2304 2305 $ git describe --dirty --always 2306 2307 Space for the output string is allocated within this function, so 2308 after using the value you have to ‘free’ the output string. If 2309 libgit2 is not installed or the program calling this function is 2310 not within a version controlled directory, then the output will be 2311 the ‘NULL’ pointer. 2312 2313 2314File: gnuastro.info, Node: Unit conversion library, Next: Spectral lines library, Prev: Git wrappers, Up: Gnuastro library 2315 231611.3.28 Unit conversion library (‘units.h’) 2317------------------------------------------- 2318 2319Datasets can contain values in various formats or units. The functions 2320in this section are defined to facilitate the easy conversion between 2321them and are declared in ‘units.h’. If there are certain conversions 2322that are useful for your work, please get in touch. 2323 2324 -- Function: 2325 int 2326 gal_units_extract_decimal (char *convert, const char 2327 *delimiter, double *args, size_t n) 2328 Parse the input ‘convert’ string with a certain delimiter (for 2329 example ‘01:23:45’, where the delimiter is ‘":"’) as multiple 2330 numbers (for example 1,23,45) and write them as an array in the 2331 space that ‘args’ is pointing to. The expected number of values in 2332 the string is specified by the ‘n’ argument (3 in the example 2333 above). 2334 2335 If the function succeeds, it will return 1, otherwise it will 2336 return 0 and the values may not be fully written into ‘args’. If 2337 the number of values parsed in the string is different from ‘n’, 2338 this function will fail. 2339 2340 -- Function: 2341 double 2342 gal_units_ra_to_degree (char *convert) 2343 Convert the input Right Ascension (RA) string (in the format of 2344 hours, minutes and seconds either as ‘_h_m_s’ or ‘_:_:_’) to 2345 degrees (a single floating point number). 2346 2347 -- Function: 2348 double 2349 gal_units_dec_to_degree (char *convert) 2350 Convert the input Declination (Dec) string (in the format of 2351 degrees, arc-minutes and arc-seconds either as ‘_d_m_s’ or ‘_:_:_’) 2352 to degrees (a single floating point number). 2353 2354 -- Function: 2355 char * 2356 gal_units_degree_to_ra (double decimal, int usecolon) 2357 Convert the input Right Ascension (RA) degree (a single floating 2358 point number) to old/standard notation (in the format of hours, 2359 minutes and seconds of ‘_h_m_s’). If ‘usecolon!=0’, then the 2360 delimiters between the components will be colons: ‘_:_:_’. 2361 2362 -- Function: 2363 char * 2364 gal_units_degree_to_dec (double decimal, int usecolon) 2365 Convert the input Declination (Dec) degree (a single floating point 2366 number) to old/standard notation (in the format of degrees, 2367 arc-minutes and arc-seconds of ‘_d_m_s’). If ‘usecolon!=0’, then 2368 the delimiters between the components will be colons: ‘_:_:_’. 2369 2370 -- Function: 2371 double 2372 gal_units_counts_to_mag (double counts, double zeropoint) 2373 Convert counts to magnitudes through the given zero point. For 2374 more on the equation, see *note Brightness flux magnitude::. 2375 2376 -- Function: 2377 double 2378 gal_units_mag_to_counts (double mag, double zeropoint) 2379 Convert magnitudes to counts through the given zero point. For 2380 more on the equation, see *note Brightness flux magnitude::. 2381 2382 -- Function: 2383 double 2384 gal_units_counts_to_jy (double counts, double zeropoint_ab) 2385 Convert counts to Janskys through an AB magnitude-based zero point. 2386 For more on the equation, see *note Brightness flux magnitude::. 2387 2388 -- Function: 2389 double 2390 gal_units_au_to_pc (double au) 2391 Convert the input value (assumed to be in Astronomical Units) to 2392 Parsecs. For the conversion equation, see the description of 2393 ‘au-to-pc’ operator in *note Arithmetic operators::. 2394 2395 -- Function: 2396 double 2397 gal_units_pc_to_au (double pc) 2398 Convert the input value (assumed to be in Parsecs) to Astronomical 2399 Units (AUs). For the conversion equation, see the description of 2400 ‘au-to-pc’ operator in *note Arithmetic operators::. 2401 2402 -- Function: 2403 double 2404 gal_units_ly_to_pc (double ly) 2405 Convert the input value (assumed to be in Light-years) to Parsecs. 2406 For the conversion equation, see the description of ‘ly-to-pc’ 2407 operator in *note Arithmetic operators::. 2408 2409 -- Function: 2410 double 2411 gal_units_pc_to_ly (double pc) 2412 Convert the input value (assumed to be in Parsecs) to Light-years. 2413 For the conversion equation, see the description of ‘ly-to-pc’ 2414 operator in *note Arithmetic operators::. 2415 2416 -- Function: 2417 double 2418 gal_units_ly_to_au (double ly) 2419 Convert the input value (assumed to be in Light-years) to 2420 Astronomical Units. For the conversion equation, see the 2421 description of ‘ly-to-pc’ operator in *note Arithmetic operators::. 2422 2423 -- Function: 2424 double 2425 gal_units_au_to_ly (double au) 2426 Convert the input value (assumed to be in Astronomical Units) to 2427 Light-years. For the conversion equation, see the description of 2428 ‘ly-to-pc’ operator in *note Arithmetic operators::. 2429 2430 2431File: gnuastro.info, Node: Spectral lines library, Next: Cosmology library, Prev: Unit conversion library, Up: Gnuastro library 2432 243311.3.29 Spectral lines library (‘speclines.h’) 2434---------------------------------------------- 2435 2436Gnuastro’s library has the following macros and functions for dealing 2437with spectral lines. All these functions are declared in 2438‘gnuastro/spectra.h’. 2439 2440 -- Macro: GAL_SPECLINES_INVALID 2441 -- Macro: GAL_SPECLINES_SIIRED 2442 -- Macro: GAL_SPECLINES_SII 2443 -- Macro: GAL_SPECLINES_SIIBLUE 2444 -- Macro: GAL_SPECLINES_NIIRED 2445 -- Macro: GAL_SPECLINES_NII 2446 -- Macro: GAL_SPECLINES_HALPHA 2447 -- Macro: GAL_SPECLINES_NIIBLUE 2448 -- Macro: GAL_SPECLINES_OIIIRED 2449 -- Macro: GAL_SPECLINES_OIII 2450 -- Macro: GAL_SPECLINES_OIIIBLUE 2451 -- Macro: GAL_SPECLINES_HBETA 2452 -- Macro: GAL_SPECLINES_HEIIRED 2453 -- Macro: GAL_SPECLINES_HGAMMA 2454 -- Macro: GAL_SPECLINES_HDELTA 2455 -- Macro: GAL_SPECLINES_HEPSILON 2456 -- Macro: GAL_SPECLINES_NEIII 2457 -- Macro: GAL_SPECLINES_OIIRED 2458 -- Macro: GAL_SPECLINES_OII 2459 -- Macro: GAL_SPECLINES_OIIBLUE 2460 -- Macro: GAL_SPECLINES_BLIMIT 2461 -- Macro: GAL_SPECLINES_MGIIRED 2462 -- Macro: GAL_SPECLINES_MGII 2463 -- Macro: GAL_SPECLINES_MGIIBLUE 2464 -- Macro: GAL_SPECLINES_CIIIRED 2465 -- Macro: GAL_SPECLINES_CIII 2466 -- Macro: GAL_SPECLINES_CIIIBLUE 2467 -- Macro: GAL_SPECLINES_HEIIBLUE 2468 -- Macro: GAL_SPECLINES_LYALPHA 2469 -- Macro: GAL_SPECLINES_LYLIMIT 2470 -- Macro: GAL_SPECLINES_INVALID_MAX 2471 Internal values/identifiers for specific spectral lines as is clear 2472 from their names. Note the first and last one, they can be used 2473 when parsing the lines automatically: both don’t correspond to any 2474 line, but their integer values correspond to the two integers just 2475 before and after the first and last line identifier. 2476 2477 ‘GAL_SPECLINES_INVALID’ has a value of zero, and allows you to have 2478 a fixed integer which never corresponds to a line. 2479 ‘GAL_SPECLINES_INVALID_MAX’ is the total number of pre-defined 2480 lines, plus one. So you can parse all the known lines with a ‘for’ 2481 loop like this: 2482 for(i=1;i<GAL_SPECLINES_INVALID_MAX;++i) 2483 2484 -- Macro: GAL_SPECLINES_ANGSTROM_SIIRED 2485 -- Macro: GAL_SPECLINES_ANGSTROM_SII 2486 -- Macro: GAL_SPECLINES_ANGSTROM_SIIBLUE 2487 -- Macro: GAL_SPECLINES_ANGSTROM_NIIRED 2488 -- Macro: GAL_SPECLINES_ANGSTROM_NII 2489 -- Macro: GAL_SPECLINES_ANGSTROM_HALPHA 2490 -- Macro: GAL_SPECLINES_ANGSTROM_NIIBLUE 2491 -- Macro: GAL_SPECLINES_ANGSTROM_OIIIRED 2492 -- Macro: GAL_SPECLINES_ANGSTROM_OIII 2493 -- Macro: GAL_SPECLINES_ANGSTROM_OIIIBLUE 2494 -- Macro: GAL_SPECLINES_ANGSTROM_HBETA 2495 -- Macro: GAL_SPECLINES_ANGSTROM_HEIIRED 2496 -- Macro: GAL_SPECLINES_ANGSTROM_HGAMMA 2497 -- Macro: GAL_SPECLINES_ANGSTROM_HDELTA 2498 -- Macro: GAL_SPECLINES_ANGSTROM_HEPSILON 2499 -- Macro: GAL_SPECLINES_ANGSTROM_NEIII 2500 -- Macro: GAL_SPECLINES_ANGSTROM_OIIRED 2501 -- Macro: GAL_SPECLINES_ANGSTROM_OII 2502 -- Macro: GAL_SPECLINES_ANGSTROM_OIIBLUE 2503 -- Macro: GAL_SPECLINES_ANGSTROM_BLIMIT 2504 -- Macro: GAL_SPECLINES_ANGSTROM_MGIIRED 2505 -- Macro: GAL_SPECLINES_ANGSTROM_MGII 2506 -- Macro: GAL_SPECLINES_ANGSTROM_MGIIBLUE 2507 -- Macro: GAL_SPECLINES_ANGSTROM_CIIIRED 2508 -- Macro: GAL_SPECLINES_ANGSTROM_CIII 2509 -- Macro: GAL_SPECLINES_ANGSTROM_CIIIBLUE 2510 -- Macro: GAL_SPECLINES_ANGSTROM_HEIIBLUE 2511 -- Macro: GAL_SPECLINES_ANGSTROM_LYALPHA 2512 -- Macro: GAL_SPECLINES_ANGSTROM_LYLIMIT 2513 Wavelength (in Angstroms) of the named lines. 2514 2515 -- Macro: GAL_SPECLINES_NAME_SIIRED 2516 -- Macro: GAL_SPECLINES_NAME_SII 2517 -- Macro: GAL_SPECLINES_NAME_SIIBLUE 2518 -- Macro: GAL_SPECLINES_NAME_NIIRED 2519 -- Macro: GAL_SPECLINES_NAME_NII 2520 -- Macro: GAL_SPECLINES_NAME_HALPHA 2521 -- Macro: GAL_SPECLINES_NAME_NIIBLUE 2522 -- Macro: GAL_SPECLINES_NAME_OIIIRED 2523 -- Macro: GAL_SPECLINES_NAME_OIII 2524 -- Macro: GAL_SPECLINES_NAME_OIIIBLUE 2525 -- Macro: GAL_SPECLINES_NAME_HBETA 2526 -- Macro: GAL_SPECLINES_NAME_HEIIRED 2527 -- Macro: GAL_SPECLINES_NAME_HGAMMA 2528 -- Macro: GAL_SPECLINES_NAME_HDELTA 2529 -- Macro: GAL_SPECLINES_NAME_HEPSILON 2530 -- Macro: GAL_SPECLINES_NAME_NEIII 2531 -- Macro: GAL_SPECLINES_NAME_OIIRED 2532 -- Macro: GAL_SPECLINES_NAME_OII 2533 -- Macro: GAL_SPECLINES_NAME_OIIBLUE 2534 -- Macro: GAL_SPECLINES_NAME_BLIMIT 2535 -- Macro: GAL_SPECLINES_NAME_MGIIRED 2536 -- Macro: GAL_SPECLINES_NAME_MGII 2537 -- Macro: GAL_SPECLINES_NAME_MGIIBLUE 2538 -- Macro: GAL_SPECLINES_NAME_CIIIRED 2539 -- Macro: GAL_SPECLINES_NAME_CIII 2540 -- Macro: GAL_SPECLINES_NAME_CIIIBLUE 2541 -- Macro: GAL_SPECLINES_NAME_HEIIBLUE 2542 -- Macro: GAL_SPECLINES_NAME_LYALPHA 2543 -- Macro: GAL_SPECLINES_NAME_LYLIMIT 2544 Names (as literal stings without any space, all in small-caps) that 2545 can be used to refer to the lines in your program and converted to 2546 and from line identifiers using the functions below. 2547 2548 -- Function: 2549 char * 2550 gal_speclines_line_name (int linecode) 2551 Return the literal string of the given spectral line identifier 2552 Macro (for example ‘GAL_SPECLINES_HALPHA’ or 2553 ‘GAL_SPECLINES_LYLIMIT’). 2554 2555 -- Function: 2556 int 2557 gal_speclines_line_code (char *name) 2558 Return the spectral line identifier of the given standard name (for 2559 example ‘GAL_SPECLINES_NAME_HALPHA’ or 2560 ‘GAL_SPECLINES_NAME_LYLIMIT’). 2561 2562 -- Function: 2563 double 2564 gal_speclines_line_angstrom (int linecode) 2565 Return the wavelength (in Angstroms) of the given line. 2566 2567 -- Function: 2568 double 2569 gal_speclines_line_redshift (double obsline, double restline) 2570 Return the redshift where the observed wavelength (‘obsline’) was 2571 emitted from (if its restframe wavelength was ‘restline’). 2572 2573 -- Function: 2574 double 2575 gal_speclines_line_redshift_code (double obsline, int 2576 linecode) 2577 Return the redshift where the observed wavelength (‘obsline’) was 2578 emitted from (assuming its a specific spectra line, identified with 2579 ‘linecode’). 2580 2581 2582File: gnuastro.info, Node: Cosmology library, Next: SAO DS9 library, Prev: Spectral lines library, Up: Gnuastro library 2583 258411.3.30 Cosmology library (‘cosmology.h’) 2585----------------------------------------- 2586 2587This library does the main cosmological calculations that are commonly 2588necessary in extra-galactic astronomical studies. The main variable in 2589this context is the redshift ($z$). The cosmological input parameters 2590in the functions below are ‘H0’, ‘o_lambda_0’, ‘o_matter_0’, 2591‘o_radiation_0’ which respectively represent the current (at redshift 0) 2592expansion rate (Hubble constant in units of km/sec/Mpc), cosmological 2593constant ($\Lambda$), matter and radiation densities. 2594 2595 All these functions are declared in ‘gnuastro/cosmology.h’. For a 2596more extended introduction/discussion of the cosmological parameters, 2597please see *note CosmicCalculator::. 2598 2599 -- Function: 2600 double 2601 gal_cosmology_age (double z, double H0, double o_lambda_0, 2602 double o_matter_0, double o_radiation_0) 2603 Returns the age of the universe at redshift ‘z’ in units of Giga 2604 years. 2605 2606 -- Function: 2607 double 2608 gal_cosmology_proper_distance (double z, double H0, double 2609 o_lambda_0, double o_matter_0, double o_radiation_0) 2610 Returns the proper distance to an object at redshift ‘z’ in units 2611 of Mega parsecs. 2612 2613 -- Function: 2614 double 2615 gal_cosmology_comoving_volume (double z, double H0, double 2616 o_lambda_0, double o_matter_0, double o_radiation_0) 2617 Returns the comoving volume over 4pi stradian to ‘z’ in units of 2618 Mega parsecs cube. 2619 2620 -- Function: 2621 double 2622 gal_cosmology_critical_density (double z, double H0, double 2623 o_lambda_0, double o_matter_0, double o_radiation_0) 2624 Returns the critical density at redshift ‘z’ in units of $g/cm^3$. 2625 2626 -- Function: 2627 double 2628 gal_cosmology_angular_distance (double z, double H0, double 2629 o_lambda_0, double o_matter_0, double o_radiation_0) 2630 Return the angular diameter distance to an object at redshift ‘z’ 2631 in units of Mega parsecs. 2632 2633 -- Function: 2634 double 2635 gal_cosmology_luminosity_distance (double z, double H0, double 2636 o_lambda_0, double o_matter_0, double o_radiation_0) 2637 Return the luminosity diameter distance to an object at redshift 2638 ‘z’ in units of Mega parsecs. 2639 2640 -- Function: 2641 double 2642 gal_cosmology_distance_modulus (double z, double H0, double 2643 o_lambda_0, double o_matter_0, double o_radiation_0) 2644 Return the distance modulus at redshift ‘z’ (with no units). 2645 2646 -- Function: 2647 double 2648 gal_cosmology_to_absolute_mag (double z, double H0, double 2649 o_lambda_0, double o_matter_0, double o_radiation_0) 2650 Return the conversion from apparent to absolute magnitude for an 2651 object at redshift ‘z’. This value has to be added to the apparent 2652 magnitude to give the absolute magnitude of an object at redshift 2653 ‘z’. 2654 2655 -- Function: 2656 double 2657 gal_cosmology_velocity_from_z (double z) 2658 Return the velocity (in km/s) corresponding to the given redshift 2659 (‘z’). 2660 2661 -- Function: 2662 double 2663 gal_cosmology_z_from_velocity (double v) 2664 Return the redshift corresponding to the given velocity (‘v’ in 2665 km/s). 2666 2667 2668File: gnuastro.info, Node: SAO DS9 library, Prev: Cosmology library, Up: Gnuastro library 2669 267011.3.31 SAO DS9 library (‘ds9.h’) 2671--------------------------------- 2672 2673This library operates on the output files of SAO DS9(1). SAO DS9 is one 2674of the most commonly used FITS image and cube viewers today with an easy 2675to use graphic user interface (GUI), see *note SAO DS9::. But besides 2676merely opening FITS data, it can also produce certain kinds of files 2677that can be useful in common analysis. For example, on DS9’s GUI, it is 2678very easy to define a (possibly complex) polygon as a “region”. You can 2679then save that “region” into a file and using the functions below, feed 2680the polygon into Gnuastro’s programs (or your custom programs). 2681 2682 -- Macro: GAL_DS9_COORD_MODE_IMG 2683 -- Macro: GAL_DS9_COORD_MODE_WCS 2684 -- Macro: GAL_DS9_COORD_MODE_INVALID 2685 Macros to identify the coordinate mode of the DS9 file. Their 2686 names are sufficiently descriptive. The last one (‘INVALID’) is 2687 for sanity checks (for example, to know if the mode is already 2688 selected). 2689 2690 -- Function: 2691 gal_data_t * 2692 gal_ds9_reg_read_polygon (char *filename) 2693 Returns an allocated generic data container (‘gal_data_t’, with an 2694 array of ‘GAL_TYPE_FLOAT64’) containing the vertices of a polygon 2695 within the SAO DS9 region file given by ‘*filename’. Since SAO DS9 2696 region files are 2 dimensional, if there are $N$ vertices in the 2697 SAO DS9 region file, the returned dataset will have $2\times N$ 2698 elements (first two elements belonging to first vertice, and etc). 2699 2700 The mode to interpret the vertice coordinates is also read from the 2701 SAO DS9 region file and written into the ‘status’ attribute of the 2702 output ‘gal_data_t’. The coordinate mode can be one of the 2703 ‘GAL_DS9_COORD_MODE_*’ macros, mentioned above. 2704 2705 It is assumed that the file begins with ‘# Region file format: DS9’ 2706 and it has two more lines (at least): a line containing the mode of 2707 the coordinates (the line should only contain either ‘fk5’ or 2708 ‘image’), a line with the polygon vertices following this format: 2709 ‘polygon(V1X,V1Y,V2X,V2Y,...)’ where ‘V1X’ and ‘V1Y’ are the 2710 horizontal and vertical coordinates of the first vertice, and so 2711 on. 2712 2713 For example, here is a minimal acceptable SAO DS9 region file: 2714 2715 # Region file format: DS9 2716 fk5 2717 polygon(53.187414,-27.779152,53.159507,-27.759633,...) 2718 2719 ---------- Footnotes ---------- 2720 2721 (1) <https://sites.google.com/cfa.harvard.edu/saoimageds9> 2722 2723 2724File: gnuastro.info, Node: Library demo programs, Prev: Gnuastro library, Up: Library 2725 272611.4 Library demo programs 2727========================== 2728 2729In this final section of *note Library::, we give some example Gnuastro 2730programs to demonstrate various features in the library. All these 2731programs have been tested and once Gnuastro is installed you can compile 2732and run them with with Gnuastro’s *note BuildProgram:: program that will 2733take care of linking issues. If you don’t have any FITS file to 2734experiment on, you can use those that are generated by Gnuastro after 2735‘make check’ in the ‘tests/’ directory, see *note Quick start::. 2736 2737* Menu: 2738 2739* Library demo - reading a image:: Read a FITS image into memory. 2740* Library demo - inspecting neighbors:: Inspect the neighbors of a pixel. 2741* Library demo - multi-threaded operation:: Doing an operation on threads. 2742* Library demo - reading and writing table columns:: Simple Column I/O. 2743 2744 2745File: gnuastro.info, Node: Library demo - reading a image, Next: Library demo - inspecting neighbors, Prev: Library demo programs, Up: Library demo programs 2746 274711.4.1 Library demo - reading a FITS image 2748------------------------------------------ 2749 2750The following simple program demonstrates how to read a FITS image into 2751memory and use the ‘void *array’ pointer in of *note Generic data 2752container::. For easy linking/compilation of this program along with a 2753first run see *note BuildProgram::. Before running, also change the 2754‘filename’ and ‘hdu’ variable values to specify an existing FITS file 2755and/or extension/HDU. 2756 2757 This is just intended to demonstrate how to use the ‘array’ pointer 2758of ‘gal_data_t’. Hence it doesn’t do important sanity checks, for 2759example in real datasets you may also have blank pixels. In such cases, 2760this program will return a NaN value (see *note Blank pixels::). So for 2761general statistical information of a dataset, it is much better to use 2762Gnuastro’s *note Statistics:: program which can deal with blank pixels 2763any many other issues in a generic dataset. 2764 2765 #include <stdio.h> 2766 #include <stdlib.h> 2767 #include <gnuastro/fits.h> /* includes gnuastro's data.h and type.h */ 2768 #include <gnuastro/statistics.h> 2769 2770 int 2771 main(void) 2772 { 2773 size_t i; 2774 float *farray; 2775 double sum=0.0f; 2776 gal_data_t *image; 2777 char *filename="img.fits", *hdu="1"; 2778 2779 2780 /* Read `img.fits' (HDU: 1) as a float32 array. */ 2781 image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32, 2782 -1, 1); 2783 2784 2785 /* Use the allocated space as a single precision floating 2786 * point array (recall that `image->array' has `void *' 2787 * type, so it is not directly usable. */ 2788 farray=image->array; 2789 2790 2791 /* Calculate the sum of all the values. */ 2792 for(i=0; i<image->size; ++i) 2793 sum += farray[i]; 2794 2795 2796 /* Report the sum. */ 2797 printf("Sum of values in %s (hdu %s) is: %f\n", 2798 filename, hdu, sum); 2799 2800 2801 /* Clean up and return. */ 2802 gal_data_free(image); 2803 return EXIT_SUCCESS; 2804 } 2805 2806 2807File: gnuastro.info, Node: Library demo - inspecting neighbors, Next: Library demo - multi-threaded operation, Prev: Library demo - reading a image, Up: Library demo programs 2808 280911.4.2 Library demo - inspecting neighbors 2810------------------------------------------ 2811 2812The following simple program shows how you can inspect the neighbors of 2813a pixel using the ‘GAL_DIMENSION_NEIGHBOR_OP’ function-like macro that 2814was introduced in *note Dimensions::. For easy linking/compilation of 2815this program along with a first run see *note BuildProgram::. Before 2816running, also change the file name and HDU (first and second arguments 2817to ‘gal_fits_img_read_to_type’) to specify an existing FITS file and/or 2818extension/HDU. 2819 2820 #include <stdio.h> 2821 #include <stdlib.h> 2822 #include <gnuastro/fits.h> 2823 #include <gnuastro/dimension.h> 2824 2825 int 2826 main(void) 2827 { 2828 double sum; 2829 float *array; 2830 size_t i, num, *dinc; 2831 gal_data_t *input=gal_fits_img_read_to_type("input.fits", "1", 2832 GAL_TYPE_FLOAT32, -1, 1); 2833 2834 /* To avoid the `void *' pointer and have `dinc'. */ 2835 array=input->array; 2836 dinc=gal_dimension_increment(input->ndim, input->dsize); 2837 2838 /* Go over all the pixels. */ 2839 for(i=0;i<input->size;++i) 2840 { 2841 num=0; 2842 sum=0.0f; 2843 GAL_DIMENSION_NEIGHBOR_OP( i, input->ndim, input->dsize, 2844 input->ndim, dinc, 2845 {++num; sum+=array[nind];} ); 2846 printf("%zu: num: %zu, sum: %f\n", i, num, sum); 2847 } 2848 2849 /* Clean up and return. */ 2850 gal_data_free(input); 2851 return EXIT_SUCCESS; 2852 } 2853 2854 2855File: gnuastro.info, Node: Library demo - multi-threaded operation, Next: Library demo - reading and writing table columns, Prev: Library demo - inspecting neighbors, Up: Library demo programs 2856 285711.4.3 Library demo - multi-threaded operation 2858---------------------------------------------- 2859 2860The following simple program shows how to use Gnuastro to simplify 2861spinning off threads and distributing different jobs between the 2862threads. The relevant thread-related functions are defined in *note 2863Gnuastro's thread related functions::. For easy linking/compilation of 2864this program, along with a first run, see Gnuastro’s *note 2865BuildProgram::. Before running, also change the ‘filename’ and ‘hdu’ 2866variable values to specify an existing FITS file and/or extension/HDU. 2867 2868 This is a very simple program to open a FITS image, distribute its 2869pixels between different threads and print the value of each pixel and 2870the thread it was assigned to. The actual operation is very simple (and 2871would not usually be done with threads in a real-life program). It is 2872intentionally chosen to put more focus on the important steps in 2873spinning of threads and how the worker function (which is called by each 2874thread) can identify the job-IDs it should work on. 2875 2876 For example, instead of an array of pixels, you can define an array 2877of tiles or any other context-specific structures as separate targets. 2878The important thing is that each action should have its own unique ID 2879(counting from zero, as is done in an array in C). You can then follow 2880the process below and use each thread to work on all the targets that 2881are assigned to it. Recall that spinning-off threads is its self an 2882expensive process and we don’t want to spin-off one thread for each 2883target (see the description of ‘gal_threads_dist_in_threads’ in *note 2884Gnuastro's thread related functions::. 2885 2886 There are many (more complicated, real-world) examples of using 2887‘gal_threads_spin_off’ in Gnuastro’s actual source code, you can see 2888them by searching for the ‘gal_threads_spin_off’ function from the top 2889source (after unpacking the tarball) directory (for example with this 2890command): 2891 2892 $ grep -r gal_threads_spin_off ./ 2893 2894The code of this demonstration program is shown below. This program was 2895also built and run when you ran ‘make check’ during the building of 2896Gnuastro (‘tests/lib/multithread.c’), so it is already tested for your 2897system and you can safely use it as a guide. 2898 2899 #include <stdio.h> 2900 #include <stdlib.h> 2901 2902 #include <gnuastro/fits.h> 2903 #include <gnuastro/threads.h> 2904 2905 2906 /* This structure can keep all information you want to pass onto the 2907 * worker function on each thread. */ 2908 struct params 2909 { 2910 gal_data_t *image; /* Dataset to print values of. */ 2911 }; 2912 2913 2914 2915 2916 /* This is the main worker function which will be called by the 2917 * different threads. `gal_threads_params' is defined in 2918 * `gnuastro/threads.h' and contains the pointer to the parameter we 2919 * want. Note that the input argument and returned value of this 2920 * function always must have `void *' type. */ 2921 void * 2922 worker_on_thread(void *in_prm) 2923 { 2924 /* Low-level definitions to be done first. */ 2925 struct gal_threads_params *tprm=(struct gal_threads_params *)in_prm; 2926 struct params *p=(struct params *)tprm->params; 2927 2928 2929 /* Subsequent definitions. */ 2930 float *array=p->image->array; 2931 size_t i, index, *dsize=p->image->dsize; 2932 2933 2934 /* Go over all the actions (pixels in this case) that were assigned 2935 * to this thread. */ 2936 for(i=0; tprm->indexs[i] != GAL_BLANK_SIZE_T; ++i) 2937 { 2938 /* For easy reading. */ 2939 index = tprm->indexs[i]; 2940 2941 2942 /* Print the information. */ 2943 printf("(%zu, %zu) on thread %zu: %g\n", index%dsize[1]+1, 2944 index/dsize[1]+1, tprm->id, array[index]); 2945 } 2946 2947 2948 /* Wait for all the other threads to finish, then return. */ 2949 if(tprm->b) pthread_barrier_wait(tprm->b); 2950 return NULL; 2951 } 2952 2953 2954 2955 2956 /* High-level function (called by the operating system). */ 2957 int 2958 main(void) 2959 { 2960 struct params p; 2961 char *filename="input.fits", *hdu="1"; 2962 size_t numthreads=gal_threads_number(); 2963 2964 /* We are using * `-1' for `minmapsize' to ensure that the image is 2965 read into * memory and `1' for `quietmmap' (which can also be 2966 zero), see the "Memory management" section in the book. */ 2967 int quietmmap=1; 2968 size_t minmapsize=-1; 2969 2970 2971 /* Read the image into memory as a float32 data type. */ 2972 p.image=gal_fits_img_read_to_type(filename, hdu, GAL_TYPE_FLOAT32, 2973 minmapsize, quietmmap); 2974 2975 2976 /* Print some basic information before the actual contents: */ 2977 printf("Pixel values of %s (HDU: %s) on %zu threads.\n", filename, 2978 hdu, numthreads); 2979 printf("Used to check the compiled library's capability in opening " 2980 "a FITS file, and also spinning-off threads.\n"); 2981 2982 2983 /* A small sanity check: this is only intended for 2D arrays (to 2984 * print the coordinates of each pixel). */ 2985 if(p.image->ndim!=2) 2986 { 2987 fprintf(stderr, "only 2D images are supported."); 2988 exit(EXIT_FAILURE); 2989 } 2990 2991 2992 /* Spin-off the threads and do the processing on each thread. */ 2993 gal_threads_spin_off(worker_on_thread, &p, p.image->size, numthreads, 2994 minmapsize, quietmmap); 2995 2996 2997 /* Clean up and return. */ 2998 gal_data_free(p.image); 2999 return EXIT_SUCCESS; 3000 } 3001 3002 3003File: gnuastro.info, Node: Library demo - reading and writing table columns, Prev: Library demo - multi-threaded operation, Up: Library demo programs 3004 300511.4.4 Library demo - reading and writing table columns 3006------------------------------------------------------- 3007 3008Tables are some of the most common inputs to, and outputs of programs. 3009This section contains a small program for reading and writing tables 3010using the constructs described in *note Table input output::. For easy 3011linking/compilation of this program, along with a first run, see 3012Gnuastro’s *note BuildProgram::. Before running, also set the following 3013file and column names in the first two lines of ‘main’. The input and 3014output names may be ‘.txt’ and ‘.fits’ tables, ‘gal_table_read’ and 3015‘gal_table_write’ will be able to write to both formats. For plain text 3016tables see see *note Gnuastro text table format::. 3017 3018 This example program reads three columns from a table. The first two 3019columns are selected by their name (‘NAME1’ and ‘NAME2’) and the third 3020is selected by its number: column 10 (counting from 1). Gnuastro’s 3021column selection is discussed in *note Selecting table columns::. The 3022first and second columns can be any type, but this program will convert 3023them to ‘int32_t’ and ‘float’ for its internal usage respectively. 3024However, the third column must be double for this program. So if it 3025isn’t, the program will abort with an error. Having the columns in 3026memory, it will print them out along with their sum (just a simple 3027application, you can do what ever you want at this stage). Reading the 3028table finishes here. 3029 3030 The rest of the program is a demonstration of writing a table. While 3031parsing the rows, this program will change the first column (to be 3032counters) and multiply the second by 10 (so the output will be 3033different). Then it will define the order of the output columns by 3034setting the ‘next’ element (to create a *note List of gal_data_t::). 3035Before writing, this function will also set names for the columns (units 3036and comments can be defined in a similar manner). Writing the columns 3037to a file is then done through a simple call to ‘gal_table_write’. 3038 3039 The operations that are shown in this example program are not 3040necessary all the time. For example, in many cases, you know the 3041numerical data type of the column before writing your program (see *note 3042Numeric data types::), so type checking and copying to a specific type 3043won’t be necessary. 3044 3045 #include <stdio.h> 3046 #include <stdlib.h> 3047 3048 #include <gnuastro/table.h> 3049 3050 int 3051 main(void) 3052 { 3053 /* File names and column names (which may also be numbers). */ 3054 char *c1_name="NAME1", *c2_name="NAME2", *c3_name="10"; 3055 char *inname="input.fits", *hdu="1", *outname="out.fits"; 3056 3057 /* Internal parameters. */ 3058 float *array2; 3059 double *array3; 3060 int32_t *array1; 3061 size_t i, counter=0; 3062 gal_data_t *c1, *c2; 3063 gal_data_t tmp, *col, *columns; 3064 gal_list_str_t *column_ids=NULL; 3065 3066 /* Define the columns to read. */ 3067 gal_list_str_add(&column_ids, c1_name, 0); 3068 gal_list_str_add(&column_ids, c2_name, 0); 3069 gal_list_str_add(&column_ids, c3_name, 0); 3070 3071 /* The columns were added in reverse, so correct it. */ 3072 gal_list_str_reverse(&column_ids); 3073 3074 /* Read the desired columns. */ 3075 columns = gal_table_read(inname, hdu, column_ids, 3076 GAL_TABLE_SEARCH_NAME, 1, -1, 1, NULL); 3077 3078 /* Go over the columns, we'll assume that you don't know their type 3079 * a-priori, so we'll check */ 3080 counter=1; 3081 for(col=columns; col!=NULL; col=col->next) 3082 switch(counter++) 3083 { 3084 case 1: /* First column: we want it as int32_t. */ 3085 c1=gal_data_copy_to_new_type(col, GAL_TYPE_INT32); 3086 array1 = c1->array; 3087 break; 3088 3089 case 2: /* Second column: we want it as float. */ 3090 c2=gal_data_copy_to_new_type(col, GAL_TYPE_FLOAT32); 3091 array2 = c2->array; 3092 break; 3093 3094 case 3: /* Third column: it MUST be double. */ 3095 if(col->type!=GAL_TYPE_FLOAT64) 3096 { 3097 fprintf(stderr, "Column %s must be float64 type, it is " 3098 "%s", c3_name, gal_type_name(col->type, 1)); 3099 exit(EXIT_FAILURE); 3100 } 3101 array3 = col->array; 3102 break; 3103 } 3104 3105 /* As an example application we'll just print them out. In the 3106 * meantime (just for a simple demonstration), change the first 3107 * array value to the counter and multiply the second by 10. */ 3108 for(i=0;i<c1->size;++i) 3109 { 3110 printf("%zu: %d + %f + %f = %f\n", i+1, array1[i], array2[i], 3111 array3[i], array1[i]+array2[i]+array3[i]); 3112 array1[i] = i+1; 3113 array2[i] *= 10; 3114 } 3115 3116 /* Link the first two columns as a list. */ 3117 c1->next = c2; 3118 c2->next = NULL; 3119 3120 /* Set names for the columns and write them out. */ 3121 c1->name = "COUNTER"; 3122 c2->name = "VALUE"; 3123 gal_table_write(c1, NULL, NULL, GAL_TABLE_FORMAT_BFITS, outname, 3124 "MY-COLUMNS", 0); 3125 3126 /* The names weren't allocated, so to avoid cleaning-up problems, 3127 * we'll set them to NULL. */ 3128 c1->name = c2->name = NULL; 3129 3130 /* Clean up and return. */ 3131 gal_data_free(c1); 3132 gal_data_free(c2); 3133 gal_list_data_free(columns); 3134 gal_list_str_free(column_ids, 0); /* strings weren't allocated. */ 3135 return EXIT_SUCCESS; 3136 } 3137 3138 3139File: gnuastro.info, Node: Developing, Next: Gnuastro programs list, Prev: Library, Up: Top 3140 314112 Developing 3142************* 3143 3144The basic idea of GNU Astronomy Utilities is for an interested 3145astronomer to be able to easily understand the code of any of the 3146programs or libraries, be able to modify the code if s/he feels there is 3147an improvement and finally, to be able to add new programs or libraries 3148for their own benefit, and the larger community if they are willing to 3149share it. In short, we hope that at least from the software point of 3150view, the “obscurantist faith in the expert’s special skill and in his 3151personal knowledge and authority” can be broken, see *note Science and 3152its tools::. With this aim in mind, Gnuastro was designed to have a 3153very basic, simple, and easy to understand architecture for any 3154interested inquirer. 3155 3156 This chapter starts with very general design choices, in particular 3157*note Why C:: and *note Program design philosophy::. It will then get a 3158little more technical about the Gnuastro code and file/directory 3159structure in *note Coding conventions:: and *note Program source::. 3160*note The TEMPLATE program:: discusses a minimal (and working) template 3161to help in creating new programs or easier learning of a program’s 3162internal structure. Some other general issues about documentation, 3163building and debugging are then discussed. This chapter concludes with 3164how you can learn about the development and get involved in *note 3165Gnuastro project webpage::, *note Developing mailing lists:: and *note 3166Contributing to Gnuastro::. 3167 3168* Menu: 3169 3170* Why C:: Why Gnuastro is designed in C. 3171* Program design philosophy:: General ideas behind the package structure. 3172* Coding conventions:: Gnuastro coding conventions. 3173* Program source:: Conventions for the code. 3174* Documentation:: Documentation is an integral part of Gnuastro. 3175* Building and debugging:: Build and possibly debug during development. 3176* Test scripts:: Understanding the test scripts. 3177* Bash programmable completion:: Auto-completions for better user experience. 3178* Developer's checklist:: Checklist to finalize your changes. 3179* Gnuastro project webpage:: Central hub for Gnuastro activities. 3180* Developing mailing lists:: Stay up to date with Gnuastro’s development. 3181* Contributing to Gnuastro:: Share your changes with all users. 3182 3183 3184File: gnuastro.info, Node: Why C, Next: Program design philosophy, Prev: Developing, Up: Developing 3185 318612.1 Why C programming language? 3187================================ 3188 3189Currently the programming languages that are commonly used in scientific 3190applications are C++(1), Java(2); Python(3), and Julia(4) (which is a 3191newcomer but swiftly gaining ground). One of the main reasons behind 3192choosing these is their high-level abstractions. However, GNU Astronomy 3193Utilities is fully written in the C programming language(5). The 3194reasons can be summarized with simplicity, portability and 3195efficiency/speed. All four are very important in a scientific software 3196and we will discuss them below. 3197 3198 Simplicity can best be demonstrated in a comparison of the main books 3199of C++ and C. The “C programming language”(6) book, written by the 3200authors of C, is only 286 pages and covers a very good fraction of the 3201language, it has also remained unchanged from 1988. C is the main 3202programming language of nearly all operating systems and there is no 3203plan of any significant update. On the other hand, the most recent “C++ 3204programming language”(7) book, also written by its author, has 1366 3205pages and its fourth edition came out in 2013! As discussed in *note 3206Science and its tools::, it is very important for other scientists to be 3207able to readily read the code of a program at their will with minimum 3208requirements. 3209 3210 In C++ or Java, inheritance in the object oriented programming 3211paradigm and their internal functions make the code very easy to write 3212for a programmer who is deeply invested in those objects and understands 3213all their relations well. But it simultaneously makes reading the 3214program for a first time reader (a curious scientist who wants to know 3215only how a small step was done) extremely hard. Before understanding 3216the methods, the scientist has to invest a lot of time and energy in 3217understanding those objects and their relations. But in C, everything 3218is done with basic language types for example ‘int’s or ‘float’s and 3219their pointers to define arrays. So when an outside reader is only 3220interested in one part of the program, that part is all they have to 3221understand. 3222 3223 Recently it is also becoming common to write scientific software in 3224Python, or a combination of it with C or C++. Python is a high level 3225scripting language which doesn’t need compilation. It is very useful 3226when you want to do something on the go and don’t want to be halted by 3227the troubles of compiling, linking, memory checking, etc. When the 3228datasets are small and the job is temporary, this ability of Python is 3229great and is highly encouraged. A very good example might be plotting, 3230in which Python is undoubtedly one of the best. 3231 3232 But as the data sets increase in size and the processing becomes more 3233complicated, the speed of Python scripts significantly decrease. So 3234when the program doesn’t change too often and is widely used in a large 3235community, mostly on large data sets (like astronomical images), using 3236Python will waste a lot of valuable research-hours. It is possible to 3237wrap C or C++ functions with Python to fix the speed issue. But this 3238creates further complexity, because the interested scientist has to 3239master two programming languages and their connection (which is not 3240trivial). 3241 3242 Like C++, Python is object oriented, so as explained above, it needs 3243a high level of experience with that particular program to reasonably 3244understand its inner workings. To make things worse, since it is mainly 3245for on-the-go programming(8), it can undergo significant changes. One 3246recent example is how Python 2.x and Python 3.x are not compatible. 3247Lots of research teams that invested heavily in Python 2.x cannot 3248benefit from Python 3.x or future versions any more. Some converters 3249are available, but since they are automatic, lots of complications might 3250arise in the conversion(9). If a research project begins using Python 32513.x today, there is no telling how compatible their investments will be 3252when Python 4.x or 5.x will come out. 3253 3254 Java is also fully object-oriented, but uses a different paradigm: 3255its compilation generates a hardware-independent _bytecode_, and a _Java 3256Virtual Machine_ (JVM) is required for the actual execution of this 3257bytecode on a computer. Java also evolved with time, and tried to 3258remain backward compatible, but inevitably some evolutions required 3259discontinuities and replacements of a few Java components which were 3260first declared as becoming _deprecated_, and removed from later 3261versions. 3262 3263 This stems from the core principles of high-level languages like 3264Python or Java: that they evolve significantly on the scale of roughly 5 3265to 10 years. They are therefore useful when you want to solve a 3266short-term problem and you are ready to pay the high cost of keeping 3267your software up to date with all the changes in the language. This is 3268fine for private companies, but usually too expensive for scientific 3269projects that have limited funding for a fixed period. As a result, the 3270reproducibility of the result (ability to regenerate the result in the 3271future, which is a core principal of any scientific result) and 3272re-usability of all the investments that went into the science software 3273will be lost to future generations! Rebuilding all the dependencies of 3274a software in an obsolete language is not easy, or even not possible. 3275Future-proof code (as long as current operating systems will be used) is 3276therefore written in C. 3277 3278 The portability of C is best demonstrated by the fact that C++, Java 3279and Python are part of the C-family of programming languages which also 3280include Julia, Perl, and many other languages. C libraries can be 3281immediately included in C++, and it is easy to write wrappers for them 3282in all C-family programming languages. This will allow other scientists 3283to benefit from C libraries using any C-family language that they 3284prefer. As a result, Gnuastro’s library is already usable in C and C++, 3285and wrappers will be(10) added for higher-level languages like Python, 3286Julia and Java. 3287 3288 The final reason was speed. This is another very important aspect of 3289C which is not independent of simplicity (first reason discussed above). 3290The abstractions provided by the higher-level languages (which also 3291makes learning them harder for a newcomer) come at the cost of speed. 3292Since C is a low-level language(11) (closer to the hardware), it has a 3293direct access to the CPU(12), is generally considered as being faster in 3294its execution, and is much less complex for both the human reader _and_ 3295the computer. The benefits of simplicity for a human were discussed 3296above. Simplicity for the computer translates into more efficient 3297(faster) programs. This creates a much closer relation between the 3298scientist/programmer (or their program) and the actual data and 3299processing. The GNU coding standards(13) also encourage the use of C 3300over all other languages when generality of usage and “high speed” is 3301desired. 3302 3303 ---------- Footnotes ---------- 3304 3305 (1) <https://isocpp.org/> 3306 3307 (2) <https://en.wikipedia.org/wiki/Java_(programming_language)> 3308 3309 (3) <https://www.python.org/> 3310 3311 (4) <https://julialang.org/> 3312 3313 (5) <https://en.wikipedia.org/wiki/C_(programming_language)> 3314 3315 (6) Brian Kernighan, Dennis Ritchie. _The C programming language_. 3316Prentice Hall, Inc., Second edition, 1988. It is also commonly known as 3317K&R and is based on the ANSI C and ISO C90 standards. 3318 3319 (7) Bjarne Stroustrup. _The C++ programming language_. 3320Addison-Wesley Professional; 4 edition, 2013. 3321 3322 (8) Note that Python is good for fast programming, not fast programs. 3323 3324 (9) For example see Jenness (2017) (https://arxiv.org/abs/1712.00461) 3325which describes how LSST is managing the transition. 3326 3327 (10) <http://savannah.gnu.org/task/?13786> 3328 3329 (11) Low-level languages are those that directly operate the hardware 3330like assembly languages. So C is actually a high-level language, but it 3331can be considered one of the lowest-level languages among all high-level 3332languages. 3333 3334 (12) for instance the _long double_ numbers with at least 64-bit 3335mantissa are not accessible in Python or Java. 3336 3337 (13) <http://www.gnu.org/prep/standards/> 3338 3339 3340File: gnuastro.info, Node: Program design philosophy, Next: Coding conventions, Prev: Why C, Up: Developing 3341 334212.2 Program design philosophy 3343============================== 3344 3345The core processing functions of each program (and all libraries) are 3346written mostly with the basic ISO C90 standard. We do make lots of use 3347of the GNU additions to the C language in the GNU C library(1), but 3348these functions are mainly used in the user interface functions (reading 3349your inputs and preparing them prior to or after the analysis). The 3350actual algorithms, which most scientists would be more interested in, 3351are much more closer to ISO C90. For this reason, program source files 3352that deal with user interface issues and those doing the actual 3353processing are clearly separated, see *note Program source::. If 3354anything particular to the GNU C library is used in the processing 3355functions, it is explained in the comments in between the code. 3356 3357 All the Gnuastro programs provide very low level and modular 3358operations (modeled on GNU Coreutils). Almost all the basic 3359command-line programs like ‘ls’, ‘cp’ or ‘rm’ on GNU/Linux operating 3360systems are part of GNU Coreutils. This enables you to use shell 3361scripting languages (for example GNU Bash) to operate on a large number 3362of files or do very complex things through the creative combinations of 3363these tools that the authors had never dreamed of. We have put a few 3364simple examples in *note Tutorials::. 3365 3366 For example all the analysis output can be saved as ASCII tables 3367which can be fed into your favorite plotting program to inspect 3368visually. Python’s Matplotlib is very useful for fast plotting of the 3369tables to immediately check your results. If you want to include the 3370plots in a document, you can use the PGFplots package within LaTeX, no 3371attempt is made to include such operations in Gnuastro. In short, Bash 3372can act as a glue to connect the inputs and outputs of all these various 3373Gnuastro programs (and other programs) in any fashion. Of course, 3374Gnuastro’s programs are just front-ends to the main workhorse (*note 3375Gnuastro library::), allowing a user to create their own programs (for 3376example with *note BuildProgram::). So once the functions within 3377programs become mature enough, they will be moved within the libraries 3378for even more general applications. 3379 3380 The advantage of this architecture is that the programs become small 3381and transparent: the starting and finishing point of every program is 3382clearly demarcated. For nearly all operations on a modern computer 3383(fast file input-output) with a modest level of complexity, the 3384read/write speed is insignificant compared to the actual processing a 3385program does. Therefore the complexity which arises from sharing memory 3386in a large application is simply not worth the speed gain. Gnuastro’s 3387design is heavily influenced from Eric Raymond’s “The Art of Unix 3388Programming”(2) which beautifully describes the design philosophy and 3389practice which lead to the success of Unix-based operating systems(3). 3390 3391 ---------- Footnotes ---------- 3392 3393 (1) Gnuastro uses many GNU additions to the C library. However, 3394thanks to the GNU Portability library (Gnulib) which is included in the 3395Gnuastro tarball, users of non-GNU/Linux operating systems can also 3396benefit from all these features when using Gnuastro. 3397 3398 (2) Eric S. Raymond, 2004, _The Art of Unix Programming_, 3399Addison-Wesley Professional Computing Series. 3400 3401 (3) KISS principle: Keep It Simple, Stupid! 3402 3403 3404File: gnuastro.info, Node: Coding conventions, Next: Program source, Prev: Program design philosophy, Up: Developing 3405 340612.3 Coding conventions 3407======================= 3408 3409In Gnuastro, we try our best to follow the GNU coding standards. Added 3410to those, Gnuastro defines the following conventions. It is very 3411important for readability that the whole package follows the same 3412convention. 3413 3414 • The code must be easy to read by eye. So when the order of several 3415 lines within a function does not matter (for example when defining 3416 variables at the start of a function). You should put the lines in 3417 the order of increasing length and group the variables with similar 3418 types such that this half-pyramid of declarations becomes most 3419 visible. If the reader is interested, a simple search will show 3420 them the variable they are interested in. However, this visual aid 3421 greatly helps in general inspections of the code and help the 3422 reader get a grip of the function’s processing. 3423 3424 • A function that cannot be fully displayed (vertically) in your 3425 monitor is probably too long and may be more useful if it is broken 3426 up into multiple functions. 40 lines is usually a good reference. 3427 When the start and end of a function are clearly visible in one 3428 glance, the function is much more easier to understand. This is 3429 most important for low-level functions (which usually define a lot 3430 of variables). Low-level functions do most of the processing, they 3431 will also be the most interesting part of a program for an 3432 inquiring astronomer. This convention is less important for higher 3433 level functions that don’t define too many variables and whose only 3434 purpose is to run the lower-level functions in a specific order and 3435 with checks. 3436 3437 In general you can be very liberal in breaking up the functions 3438 into smaller parts, the GNU Compiler Collection (GCC) will 3439 automatically compile the functions as inline functions when the 3440 optimizations are turned on. So you don’t have to worry about 3441 decreasing the speed. By default Gnuastro will compile with the 3442 ‘-O3’ optimization flag. 3443 3444 • All Gnuastro hand-written text files (C source code, Texinfo 3445 documentation source, and version control commit messages) should 3446 not exceed *75* characters per line. Monitors today are certainly 3447 much wider, but with this limit, reading the functions becomes much 3448 more easier. Also for the developers, it allows multiple files (or 3449 multiple views of one file) to be displayed beside each other on 3450 wide monitors. 3451 3452 Emacs’s buffers are excellent for this capability, setting a buffer 3453 width of 80 with ‘<C-u 80 C-x 3>’ will allow you to view and work 3454 on several files or different parts of one file using the wide 3455 monitors common today. Emacs buffers can also be used as a shell 3456 prompt and compile the program (with <M-x compile>), and 80 3457 characters is the default width in most terminal emulators. If you 3458 use Emacs, Gnuastro sets the 75 character ‘fill-column’ variable 3459 automatically for you, see cartouche below. 3460 3461 For long comments you can use press <Alt-q> in Emacs to separate 3462 them into separate lines automatically. For long literal strings, 3463 you can use the fact that in C, two strings immediately after each 3464 other are concatenated, for example ‘"The first part, " "and the 3465 second part."’. Note the space character in the end of the first 3466 part. Since they are now separated, you can easily break a long 3467 literal string into several lines and adhere to the maximum 75 3468 character line length policy. 3469 3470 • The headers required by each source file (ending with ‘.c’) should 3471 be defined inside of it. All the headers a complete program needs 3472 should _not_ be stacked in another header to include in all source 3473 files (for example ‘main.h’). Although most ‘professional’ 3474 programmers choose this single header method, Gnuastro is primarily 3475 written for professional/inquisitive astronomers (who are generally 3476 amateur programmers). The list of header files included provides 3477 valuable general information and helps the reader. ‘main.h’ may 3478 only include the header file(s) that define types that the main 3479 program structure needs, see ‘main.h’ in *note Program source::. 3480 Those particular header files that are included in ‘main.h’ can of 3481 course be ignored (not included) in separate source files. 3482 3483 • The headers should be classified (by an empty line) into separate 3484 groups: 3485 3486 1. ‘#include <config.h>’: This must be the first code line (not 3487 commented or blank) in each source file _within Gnuastro_. It 3488 sets macros that the GNU Portability Library (Gnulib) will use 3489 for a unified environment (GNU C Library), even when the user 3490 is building on a system that doesn’t use the GNU C library. 3491 3492 2. The C library header files, for example ‘stdio.h’, ‘stdlib.h’, 3493 or ‘math.h’. 3494 3. Installed library header files, including Gnuastro’s installed 3495 headers (for example ‘cfitsio.h’ or ‘gsl/gsl_rng.h’, or 3496 ‘gnuastro/fits.h’). 3497 4. Gnuastro’s internal headers (that are not installed), for 3498 example ‘gnuastro-internal/options.h’. 3499 5. For programs, the ‘main.h’ file (which is needed by the next 3500 group of headers). 3501 6. That particular program’s header files, for example 3502 ‘mkprof.h’, or ‘noisechisel.h’. 3503 3504 As much as order does not matter when you include the header of 3505 each group, sort them by length, as described above. 3506 3507 • All function names, variables, etc should be in lower case. Macros 3508 and constant global ‘enum’s should be in upper case. 3509 3510 • For the naming of exported header files, functions, variables, 3511 macros, and library functions, we adopt similar conventions to 3512 those used by the GNU Scientific Library (GSL)(1). In particular, 3513 in order to avoid clashes with the names of functions and variables 3514 coming from other libraries the name-space ‘‘gal_’’ is prefixed to 3515 them. GAL stands for _G_NU _A_stronomy _L_ibrary. 3516 3517 • All installed header files should be in the ‘lib/gnuastro’ 3518 directory (under the top Gnuastro source directory). After 3519 installation, they will be put in the ‘$prefix/include/gnuastro’ 3520 directory (see *note Installation directory:: for ‘$prefix’). 3521 Therefore with this convention Gnuastro’s headers can be included 3522 in internal (to Gnuastro) and external (a library user) source 3523 files with the same line 3524 # include <gnuastro/headername.h> 3525 Note that the GSL convention for header file names is 3526 ‘gsl_specialname.h’, so your include directive for a GSL header 3527 must be something like ‘#include <gsl/gsl_specialname.h>’. 3528 Gnuastro doesn’t follow this GSL guideline because of the repeated 3529 ‘gsl’ in the include directive. It can be confusing and cause bugs 3530 for beginners. All Gnuastro (and GSL) headers must be located 3531 within a unique directory and will not be mixed with other headers. 3532 Therefore the ‘‘gsl_’’ prefix to the header file names is 3533 redundant(2). 3534 3535 • All installed functions and variables should also include the 3536 base-name of the file in which they are defined as prefix, using 3537 underscores to separate words(3). The same applies to exported 3538 macros, but in upper case. For example in Gnuastro’s top source 3539 directory, the prototype of function ‘gal_box_border_from_center’ 3540 is in ‘lib/gnuastro/box.h’, and the macro ‘GAL_POLYGON_MAX_CORNERS’ 3541 is defined in ‘lib/gnuastro/polygon.h’. 3542 3543 This is necessary to give any user (who is not familiar with the 3544 library structure) the ability to follow the code. This convention 3545 does make the function names longer (a little harder to write), but 3546 the extra documentation it provides plays an important role in 3547 Gnuastro and is worth the cost. 3548 3549 • There should be no trailing white space in a line. To do this 3550 automatically every time you save a file in Emacs, add the 3551 following line to your ‘~/.emacs’ file. 3552 (add-hook 'before-save-hook 'delete-trailing-whitespace) 3553 3554 • There should be no tabs in the indentation(4). 3555 3556 • Individual, contextually similar, functions in a source file are 3557 separated by 5 blank lines to be easily seen to be related in a 3558 group when parsing the source code by eye. In Emacs you can use 3559 <CTRL-u 5 CTRL-o>. 3560 3561 • One group of contextually similar functions in a source file is 3562 separated from another with 20 blank lines. In Emacs you can use 3563 <CTRL-u 20 CTRL-o>. Each group of functions has short descriptive 3564 title of the functions in that group. This title is surrounded by 3565 asterisks (<*>) to make it clearly distinguishable. Such 3566 contextual grouping and clear title are very important for easily 3567 understanding the code. 3568 3569 • Always read the comments before the patch of code under it. 3570 Similarly, try to add as many comments as you can regarding every 3571 patch of code. Effectively, we want someone to get a good feeling 3572 of the steps, without having to read the C code and only by reading 3573 the comments. This follows similar principles as Literate 3574 programming (https://en.wikipedia.org/wiki/Literate_programming). 3575 3576 The last two conventions are not common and might benefit from a 3577short discussion here. With a good experience in advanced text editor 3578operations, the last two are redundant for a professional developer. 3579However, recall that Gnuastro aspires to be friendly to unfamiliar, and 3580inexperienced (in programming) eyes. In other words, as discussed in 3581*note Science and its tools::, we want the code to appear welcoming to 3582someone who is completely new to coding (and text editors) and only has 3583a scientific curiosity. 3584 3585 Newcomers to coding and development, who are curious enough to 3586venture into the code, will probably not be using (or have any knowledge 3587of) advanced text editors. They will see the raw code in the web page 3588or on a simple text editor (like Gedit) as plain text. Trying to learn 3589and understand a file with dense functions that are all spaced with one 3590or two blank lines can be very taunting for a newcomer. But when they 3591scroll through the file and see clear titles and meaningful spaces for 3592similar functions, we are helping them find and focus on the part they 3593are most interested in sooner and easier. 3594 3595*GNU Emacs, the recommended text editor:* GNU Emacs is an extensible and 3596easily customizable text editor which many programmers rely on for 3597developing due to its countless features. Among them, it allows 3598specification of certain settings that are applied to a single file or 3599to all files in a directory and its sub-directories. In order to 3600harmonize code coming from different contributors, Gnuastro comes with a 3601‘.dir-locals.el’ file which automatically configures Emacs to satisfy 3602most of the coding conventions above when you are using it within 3603Gnuastro’s directories. Thus, Emacs users can readily start hacking 3604into Gnuastro. If you are new to developing, we strongly recommend this 3605editor. Emacs was the first project released by GNU and is still one of 3606its flagship projects. Some resources can be found at: 3607 3608Official manual 3609 At <https://www.gnu.org/software/emacs/manual/emacs.html>. This is 3610 a great and very complete manual which is being improved for over 3611 30 years and is the best starting point to learn it. It just 3612 requires a little patience and practice, but rest assured that you 3613 will be rewarded. If you install Emacs, you also have access to 3614 this manual on the command-line with the following command (see 3615 *note Info::). 3616 3617 $ info emacs 3618 3619A guided tour of emacs 3620 At <https://www.gnu.org/software/emacs/tour/>. A short visual tour 3621 of Emacs, officially maintained by the Emacs developers. 3622 3623Unofficial mini-manual 3624 At <https://tuhdo.github.io/emacs-tutor.html>. A shorter manual 3625 which contains nice animated images of using Emacs. 3626 3627 ---------- Footnotes ---------- 3628 3629 (1) <https://www.gnu.org/software/gsl/design/gsl-design.html#SEC15> 3630 3631 (2) For GSL, this prefix has an internal technical application: GSL’s 3632architecture mixes installed and not-installed headers in the same 3633directory. This prefix is used to identify their installation status. 3634Therefore this filename prefix in GSL a technical internal issue (for 3635developers, not users). 3636 3637 (3) The convention to use underscores to separate words, called 3638“snake case” (or “snake_case”). This is also recommended by the GNU 3639coding standards. 3640 3641 (4) If you use Emacs, Gnuastro’s ‘.dir-locals.el’ file will 3642automatically never use tabs for indentation. To make this a default in 3643all your Emacs sessions, you can add the following line to your 3644‘~/.emacs’ file: ‘(setq-default indent-tabs-mode nil)’ 3645 3646 3647File: gnuastro.info, Node: Program source, Next: Documentation, Prev: Coding conventions, Up: Developing 3648 364912.4 Program source 3650=================== 3651 3652Besides the fact that all the programs share some functions that were 3653explained in *note Library::, everything else about each program is 3654completely independent. Recall that Gnuastro is written for an active 3655astronomer/scientist (not a passive one who just uses a software). It 3656must thus be easily navigable. Hence there are fixed source files (that 3657contain fixed operations) that must be present in all programs, these 3658are discussed fully in *note Mandatory source code files::. To easily 3659understand the explanations in this section you can use *note The 3660TEMPLATE program:: which contains the bare minimum code for one working 3661program. This template can also be used to easily add new utilities: 3662just copy and paste the directory and change ‘TEMPLATE’ with your 3663program’s name. 3664 3665* Menu: 3666 3667* Mandatory source code files:: Description of files common to all programs. 3668* The TEMPLATE program:: Template for easy creation of a new program. 3669 3670 3671File: gnuastro.info, Node: Mandatory source code files, Next: The TEMPLATE program, Prev: Program source, Up: Program source 3672 367312.4.1 Mandatory source code files 3674---------------------------------- 3675 3676Some programs might need lots of source files and if there is no fixed 3677convention, navigating them can become very hard for a new inquirer into 3678the code. The following source files exist in every program’s source 3679directory (which is located in ‘bin/progname’). For small programs, 3680these files are enough. Larger programs will need more files and 3681developers are encouraged to define any number of new files. It is just 3682important that the following list of files exist and do what is 3683described here. When creating other source files, please choose 3684filenames that are a complete single word: don’t abbreviate 3685(abbreviations are cryptic). For a minimal program containing all these 3686files, see *note The TEMPLATE program::. 3687 3688‘main.c’ 3689 Each executable has a ‘main’ function, which is located in 3690 ‘main.c’. Therefore this file is the starting point when reading 3691 any program’s source code. No actual processing functions must be 3692 defined in this file, the function(s) in this file are only meant 3693 to connect the most high level steps of each program. Generally, 3694 ‘main’ will first call the top user interface function to read user 3695 input and make all the preparations. Then it will pass control to 3696 the top processing function for that program. The functions to do 3697 both these jobs must be defined in other source files. 3698 3699‘main.h’ 3700 All the major parameters which will be used in the program must be 3701 stored in a structure which is defined in ‘main.h’. The name of 3702 this structure is usually ‘prognameparams’, for example 3703 ‘cropparams’ or ‘noisechiselparams’. So ‘#include "main.h"’ will 3704 be a staple in all the source codes of the program. It is also 3705 regularly the first (and only) argument most of the program’s 3706 functions which greatly helps in readability. 3707 3708 Keeping all the major parameters of a program in this structure has 3709 the major benefit that most functions will only need one argument: 3710 a pointer to this structure. This will significantly facilitate 3711 the job of the programmer, the inquirer and the computer. All the 3712 programs in Gnuastro are designed to be low-level, small and 3713 independent parts, so this structure should not get too large. 3714 3715 The main root structure of all programs contains at least one 3716 instance of the ‘gal_options_common_params’ structure. This 3717 structure will keep the values to all common options in Gnuastro’s 3718 programs (see *note Common options::). This top root structure is 3719 conveniently called ‘p’ (short for parameters) by all the functions 3720 in the programs and the common options parameters within it are 3721 called ‘cp’. With this convention any reader can immediately 3722 understand where to look for the definition of one parameter. For 3723 example you know that ‘p->cp->output’ is in the common parameters 3724 while ‘p->threshold’ is in the program’s parameters. 3725 3726 With this basic root structure, source code of functions can 3727 potentially become full of structure de-reference operators (‘->’) 3728 which can make the code very unreadable. In order to avoid this, 3729 whenever a structure element is used more than a couple of times in 3730 a function, a variable of the same type and with the same name (so 3731 it can be searched) as the desired structure element should be 3732 defined with the value of the root structure inside of it in 3733 definition time. Here is an example. 3734 3735 char *hdu=p->cp.hdu; 3736 float threshold=p->threshold; 3737 3738‘args.h’ 3739 The options particular to each program are defined in this file. 3740 Each option is defined by a block of parameters in 3741 ‘program_options’. These blocks are all you should modify in this 3742 file, leave the bottom group of definitions untouched. These are 3743 fed directly into the GNU C library’s Argp facilities and it is 3744 recommended to have a look at that for better understand what is 3745 going on, although this is not required here. 3746 3747 Each element of the block defining an option is described under 3748 ‘argp_option’ in ‘bootstrapped/lib/argp.h’ (from Gnuastro’s top 3749 source file). Note that the last few elements of this structure 3750 are Gnuastro additions (not documented in the standard Argp 3751 manual). The values to these last elements are defined in 3752 ‘lib/gnuastro/type.h’ and ‘lib/gnuastro-internal/options.h’ (from 3753 Gnuastro’s top source directory). 3754 3755‘ui.h’ 3756 Besides declaring the exported functions of ‘ui.c’, this header 3757 also keeps the “key”s to every program-specific option. The first 3758 class of keys for the options that have a short-option version 3759 (single letter, see *note Options::). The character that is 3760 defined here is the option’s short option name. The list of 3761 available alphabet characters can be seen in the comments. Recall 3762 that some common options also take some characters, for those, see 3763 ‘lib/gnuastro-internal/options.h’. 3764 3765 The second group of options are those that don’t have a short 3766 option alternative. Only the first in this group needs a value 3767 (‘1000’), the rest will be given a value by C’s ‘enum’ definition, 3768 so the actual value is irrelevant and must never be used, always 3769 use the name. 3770 3771‘ui.c’ 3772 Everything related to reading the user input arguments and options, 3773 checking the configuration files and checking the consistency of 3774 the input parameters before the actual processing is run should be 3775 done in this file. Since most functions are the same, with only 3776 the internal checks and structure parameters differing. We 3777 recommend going through the ‘ui.c’ of *note The TEMPLATE program::, 3778 or several other programs for a better understanding. 3779 3780 The most high-level function in ‘ui.c’ is named 3781 ‘ui_read_check_inputs_setup’. It accepts the raw command-line 3782 inputs and a pointer to the root structure for that program (see 3783 the explanation for ‘main.h’). This is the function that ‘main’ 3784 calls. The basic idea of the functions in this file is that the 3785 processing functions should need a minimum number of such checks. 3786 With this convention an inquirer who only wants to understand only 3787 one part (mostly the processing part and not user input details and 3788 sanity checks) of the code can easily do so in the later files. It 3789 also makes all the errors related to input appear before the 3790 processing begins which is more convenient for the user. 3791 3792‘progname.c, progname.h’ 3793 The high-level processing functions in each program are in a file 3794 named ‘progname.c’, for example ‘crop.c’ or ‘noisechisel.c’. The 3795 function within these files which ‘main’ calls is also named after 3796 the program, for example 3797 3798 void 3799 crop(struct cropparams *p) 3800 3801 or 3802 3803 void 3804 noisechisel(struct noisechiselparams *p) 3805 3806 In this manner, if an inquirer is interested the processing steps, 3807 they can immediately come and check this file for the first 3808 processing step without having to go through ‘main.c’ and ‘ui.c’ 3809 first. In most situations, any failure in any step of the programs 3810 will result in an informative error message and an immediate abort 3811 in the program. So there is usually no need for return values. 3812 Under more complicated situations where a return value might be 3813 necessary, ‘void’ will be replaced with an ‘int’ in the examples 3814 above. This value must be directly returned by ‘main’, so it has 3815 to be an ‘int’. 3816 3817‘authors-cite.h’ 3818 This header file keeps the global variable for the program authors 3819 and its BibTeX record for citation. They are used in the outputs 3820 of the common options ‘--version’ and ‘--cite’, see *note Operating 3821 mode options::. 3822 3823‘progname-complete.bash’ 3824 This shell script is used for implementing auto-completion features 3825 when running Gnuastro’s programs within GNU Bash. For more on the 3826 concept of shell auto-completion and how it is managed in Gnuastro, 3827 see *note Bash programmable completion::. 3828 3829 These files assume a set of common shell functions that have the 3830 prefix ‘_gnuastro_autocomplete_’ in their name and are defined in 3831 ‘bin/complete.bash.in’ (of the source directory, and under version 3832 control) and ‘bin/complete.bash.built’ (built during the building 3833 of Gnuastro in the build directory). During Gnuastro’s build, all 3834 these Bash completion files are merged into one file that is 3835 installed and the user can ‘source’ them into their Bash startup 3836 file, for example see *note Quick start::. 3837 3838 3839File: gnuastro.info, Node: The TEMPLATE program, Prev: Mandatory source code files, Up: Program source 3840 384112.4.2 The TEMPLATE program 3842--------------------------- 3843 3844The extra creativity offered by libraries comes at a cost: you have to 3845actually write your ‘main’ function and get your hands dirty in managing 3846user inputs: are all the necessary parameters given a value? is the 3847input in the correct format? do the options and the inputs correspond? 3848and many other similar checks. So when an operation has well-defined 3849inputs and outputs and is commonly needed, it is much more worthwhile to 3850simply do use all the great features that Gnuastro has already defined 3851for such operations. 3852 3853 To make it easier to learn/apply the internal program infra-structure 3854discussed in *note Mandatory source code files::, in the *note Version 3855controlled source::, Gnuastro ships with a template program . This 3856template program is not available in the Gnuastro tarball so it doesn’t 3857confuse people using the tarball. The ‘bin/TEMPLATE’ directory in 3858Gnuastro’s Git repository contains the bare-minimum files necessary to 3859define a new program and all the basic/necessary files/functions are 3860pre-defined there. 3861 3862 Below you can see a list of initial steps to take for customizing 3863this template. We just assume that after cloning Gnuastro’s history, 3864you have already bootstrapped Gnuastro, if not, please see *note 3865Bootstrapping::. 3866 3867 1. Select a name for your new program (for example ‘myprog’). 3868 3869 2. Copy the ‘TEMPLATE’ directory to a directory with your program’s 3870 name: 3871 $ cp -R bin/TEMPLATE bin/myprog 3872 3873 3. As with all source files in Gnuastro, all the files in template 3874 also have a copyright notice at their top. Open all the files and 3875 correct these notices: 1) The first line contains a single-line 3876 description of the program. 2) In the second line only the name or 3877 your program needs to be fixed and 3) Add your name and email as a 3878 “Contributing author”. As your program grows, you will need to add 3879 new files, don’t forget to add this notice in those new files too, 3880 just put your name and email under “Original author” and correct 3881 the copyright years. 3882 3883 4. Open ‘configure.ac’ in the top Gnuastro source. This file manages 3884 the operations that are done when a user runs ‘./configure’. Going 3885 down the file, you will notice repetitive parts for each program. 3886 You will notice that the program names follow an alphabetic 3887 ordering in each part. There is also a commented line/patch for 3888 the ‘TEMPLATE’ program in each part. You can copy one line/patch 3889 (from the program above or below your desired name for example) and 3890 paste it in the proper place for your new program. Then correct 3891 the names of the copied program to your new program name. There 3892 are multiple places where this has to be done, so be patient and go 3893 down to the bottom of the file. Ultimately add 3894 ‘bin/myprog/Makefile’ to ‘AC_CONFIG_FILES’, only here the ordering 3895 depends on the length of the name (it isn’t alphabetical). 3896 3897 5. Open ‘Makefile.am’ in the top Gnuastro source. Similar to the 3898 previous step, add your new program similar to all the other 3899 programs. Here there are only two places: 1) at the top where we 3900 define the conditionals (three lines per program), and 2) 3901 immediately under it as part of the value for ‘SUBDIRS’. 3902 3903 6. Open ‘doc/Makefile.am’ and similar to ‘Makefile.am’ (above), add 3904 the proper entries for the man-page of your program to be created 3905 (here, the variable that keeps all the man-pages to be created is 3906 ‘dist_man_MANS’). Then scroll down and add a rule to build the 3907 man-page similar to the other existing rules (in alphabetical 3908 order). Don’t forget to add a short one-line description here, it 3909 will be displayed on top of the man-page. 3910 3911 7. Change ‘TEMPLATE.c’ and ‘TEMPLATE.h’ to ‘myprog.c’ and ‘myprog.h’ 3912 in the file names: 3913 3914 $ cd bin/myprog 3915 $ mv TEMPLATE.c myprog.c 3916 $ mv TEMPLATE.h myprog.h 3917 3918 8. Correct all occurrences of ‘TEMPLATE’ in the input files to 3919 ‘myprog’ (in short or long format). You can get a list of all 3920 occurrences with the following command. If you use Emacs, it will 3921 be able to parse the Grep output and open the proper file and line 3922 automatically. So this step can be very easy. 3923 3924 $ grep --color -nHi -e template * 3925 3926 9. Run the following commands to re-build the configuration and build 3927 system, and then to configure and build Gnuastro (which now 3928 includes your exciting new program). 3929 $ autoreconf -f 3930 $ ./configure 3931 $ make 3932 3933 10. You are done! You can now start customizing your new program to 3934 do your special processing. When its complete, just don’t forget 3935 to add checks also, so it can be tested at least once on a user’s 3936 system with ‘make check’, see *note Test scripts::. Finally, if 3937 you would like to share it with all Gnuastro users, inform us so we 3938 merge it into Gnuastro’s main history. 3939 3940 3941File: gnuastro.info, Node: Documentation, Next: Building and debugging, Prev: Program source, Up: Developing 3942 394312.5 Documentation 3944================== 3945 3946Documentation (this book) is an integral part of Gnuastro (see *note 3947Science and its tools::). Documentation is not considered a separate 3948project and must be written by its developers. Users can make 3949edits/corrections, but the initial writing must be by the developer. 3950So, no change is considered valid for implementation unless the 3951respective parts of the book have also been updated. The following 3952procedure can be a good suggestion to take when you have a new idea and 3953are about to start implementing it. 3954 3955 The steps below are not a requirement, the important thing is that 3956when you send your work to be included in Gnuastro, the book and the 3957code have to both be fully up-to-date and compatible, with the purpose 3958of the update very clearly explained. You can follow any strategy you 3959like, the following strategy was what we have found to be most useful 3960until now. 3961 3962 1. Edit the book and fully explain your desired change, such that your 3963 idea is completely embedded in the general context of the book with 3964 no sense of discontinuity for a first time reader. This will allow 3965 you to plan the idea much more accurately and in the general 3966 context of Gnuastro (a particular program or library). Later on, 3967 when you are coding, this general context will significantly help 3968 you as a road-map. 3969 3970 A very important part of this process is the program/library 3971 introduction. These first few paragraphs explain the purposes of 3972 the program or library and are fundamental to Gnuastro. Before 3973 actually starting to code, explain your idea’s purpose thoroughly 3974 in the start of the respective/new section you wish to work on. 3975 While actually writing its purpose for a new reader, you will 3976 probably get some valuable and interesting ideas that you hadn’t 3977 thought of before. This has occurred several times during the 3978 creation of Gnuastro. 3979 3980 If an introduction already exists, embed or blend your idea’s 3981 purpose with the existing introduction. We emphasize that doing 3982 this is equally useful for you (as the programmer) as it is useful 3983 for the user (reader). Recall that the purpose of a program is 3984 very important, see *note Program design philosophy::. 3985 3986 As you have already noticed for every program/library, it is very 3987 important that the basics of the science and technique be explained 3988 in separate subsections prior to the ‘Invoking Programname’ 3989 subsection. If you are writing a new program or your addition to 3990 an existing program involves a new concept, also include such 3991 subsections and explain the concepts so a person completely 3992 unfamiliar with the concepts can get a general initial 3993 understanding. You don’t have to go deep into the details, just 3994 enough to get an interested person (with absolutely no background) 3995 started with some good pointers/links to where they can continue 3996 studying if they are more interested. If you feel you can’t do 3997 that, then you have probably not understood the concept yourself. 3998 If you feel you don’t have the time, then think about yourself as 3999 the reader in one year: you will forget almost all the details, so 4000 now that you have done all the theoretical preparations, add a few 4001 more hours and document it. Therefore in one year, when you find a 4002 bug or want to add a new feature, you don’t have to prepare as 4003 much. Have in mind that your only limitation in length is the 4004 fatigue of the reader after reading a long text, nothing else. So 4005 as long as you keep it relevant/interesting for the reader, there 4006 is no page number limit/cost. 4007 4008 It might also help if you start discussing the usage of your idea 4009 in the ‘Invoking ProgramName’ subsection (explaining the options 4010 and arguments you have in mind) at this stage too. Actually 4011 starting to write it here will really help you later when you are 4012 coding. 4013 4014 2. After you have finished adding your initial intended plan to the 4015 book, then start coding your change or new program within the 4016 Gnuastro source files. While you are coding, you will notice that 4017 somethings should be different from what you wrote in the book 4018 (your initial plan). So correct them as you are actually coding, 4019 but don’t worry too much about missing a few things (see the next 4020 step). 4021 4022 3. After your work has been fully implemented, read the section 4023 documentation from the start and see if you didn’t miss any change 4024 in the coding and to see if the context is fairly continuous for a 4025 first time reader (who hasn’t seen the book or had known Gnuastro 4026 before you made your change). 4027 4028 4. If the change is notable, also update the ‘NEWS’ file. 4029 4030 4031File: gnuastro.info, Node: Building and debugging, Next: Test scripts, Prev: Documentation, Up: Developing 4032 403312.6 Building and debugging 4034=========================== 4035 4036To build the various programs and libraries in Gnuastro, the GNU build 4037system is used which defines the steps in *note Quick start::. It 4038consists of GNU Autoconf, GNU Automake and GNU Libtool which are 4039collectively known as GNU Autotools. They provide a very portable 4040system to check the hosts environment and compile Gnuastro based on 4041that. They also make installing everything in their standard places 4042very easy for the programmer. Most of the small caps files that you see 4043in the top source directory of the tarball are created by these three 4044tools (see *note Version controlled source::). To facilitate the 4045building and testing of your work during development, Gnuastro comes 4046with two useful scripts: 4047 4048‘developer-build’ 4049 This is more fully described in *note Configure and build in RAM::. 4050 During development, you will usually run this command only once (at 4051 the start of your work). 4052 4053‘tests/during-dev.sh’ 4054 This script is designed to be run each time you make a change and 4055 want to test your work (with some possible input and output). The 4056 script itself is heavily commented and thoroughly describes the 4057 best way to use it, so we won’t repeat it here. 4058 4059 As a short summary: you specify the build directory, an output 4060 directory (for the built program to be run in, and also contains 4061 the inputs), the program’s short name and the arguments and options 4062 that it should be run with. This script will then build Gnuastro, 4063 go to the output directory and run the built executable from there. 4064 One option for the output directory might be your desktop, so you 4065 can easily see the output files and delete them when you are 4066 finished. The main purpose of these scripts is to keep your source 4067 directory clean and facilitate your development. 4068 4069 By default all the programs are compiled with optimization flags for 4070increased speed. A side effect of optimization is that valuable 4071debugging information is lost. All the libraries are also linked as 4072shared libraries by default. Shared libraries further complicate the 4073debugging process and significantly slow down the compilation (the 4074‘make’ command). So during development it is recommended to configure 4075Gnuastro as follows: 4076 4077 $ ./configure --enable-debug 4078 4079In ‘developer-build’ you can ask for this behavior through the ‘--debug’ 4080option, see *note Separate build and source directories::. 4081 4082 In order to understand the building process, you can go through the 4083Autoconf, Automake and Libtool manuals, like all GNU manuals they 4084provide both a great tutorial and technical documentation. The “A small 4085Hello World” section in Automake’s manual (in chapter 2) can be a good 4086starting guide after you have read the separate introductions. 4087 4088 4089File: gnuastro.info, Node: Test scripts, Next: Bash programmable completion, Prev: Building and debugging, Up: Developing 4090 409112.7 Test scripts 4092================= 4093 4094As explained in *note Tests::, for every program some simple tests are 4095written to check the various independent features of the program. All 4096the tests are placed in the ‘tests/’ directory. The ‘tests/prepconf.sh’ 4097script is the first ‘test’ that will be run. It will copy all the 4098configuration files from the various directories to a ‘tests/.gnuastro’ 4099directory (which it will make) so the various tests can set the default 4100values. This script will also make sure the programs don’t go searching 4101for user and system wide configuration files to avoid the mixing of 4102values with different Gnuastro version on the system. 4103 4104 For each program, the tests are placed inside directories with the 4105program name. Each test is written as a shell script. The last line of 4106this script is the test which runs the program with certain parameters. 4107The return value of this script determines the fate of the test, see the 4108“Support for test suites” chapter of the Automake manual for a very nice 4109and complete explanation. In every script, two variables are defined at 4110first: ‘prog’ and ‘execname’. The first specifies the program name and 4111the second the location of the executable. 4112 4113 The most important thing to have in mind about all the test scripts 4114is that they are run from inside the ‘tests/’ directory in the “build 4115tree”. Which can be different from the directory they are stored in 4116(known as the “source tree”)(1). This distinction is made by GNU 4117Autoconf and Automake (which configure, build and install Gnuastro) so 4118that you can install the program even if you don’t have write access to 4119the directory keeping the source files. See the “Parallel build trees 4120(a.k.a VPATH builds)” in the Automake manual for a nice explanation. 4121 4122 Because of this, any necessary inputs that are distributed in the 4123tarball(2), for example the catalogs necessary for checks in 4124MakeProfiles and Crop, must be identified with the ‘$topsrc’ prefix 4125instead of ‘../’ (for the top source directory that is unpacked). This 4126‘$topsrc’ variable points to the source tree where the script can find 4127the source data (it is defined in ‘tests/Makefile.am’). The executables 4128and other test products were built in the build tree (where they are 4129being run), so they don’t need to be prefixed with that variable. This 4130is also true for images or files that were produced by other tests. 4131 4132 ---------- Footnotes ---------- 4133 4134 (1) The ‘developer-build’ script also uses this feature to keep the 4135source and build directories separate (see *note Separate build and 4136source directories::). 4137 4138 (2) In many cases, the inputs of a test are outputs of previous 4139tests, this doesn’t apply to this class of inputs. Because all outputs 4140of previous tests are in the “build tree”. 4141 4142 4143File: gnuastro.info, Node: Bash programmable completion, Next: Developer's checklist, Prev: Test scripts, Up: Developing 4144 414512.8 Bash programmable completion 4146================================= 4147 4148*Under development:* While work on TAB completion is ongoing, it is not 4149yet fully ready, please see the notice at the start of *note Shell TAB 4150completion::. 4151 4152 Gnuastro provides Programmable completion facilities in Bash. This 4153greatly helps users reach their desired result with minimal keystrokes, 4154and helps them spend less time on figuring out the option names and 4155values their acceptable values. Gnuastro’s completion script not only 4156completes the half-written commands, but also prints suggestions based 4157on previous arguments. 4158 4159 Imagine a scenario where we need to download three columns containing 4160the right ascension, declination, and parallax from the GAIA EDR3 4161dataset. We have to make sure how these columns are abbreviated or 4162spelled. So we can call the command below, and store the column names 4163in a file such as ‘gaia-edr3-columns.txt’. 4164 4165 $ astquery gaia --information > gaia-edr3-columns.txt 4166 4167Then we need to memorize or copy the column names of interest, and 4168specify an output fits file name such as ‘gaia.fits’: 4169 4170 $ astquery gaia --dataset=edr3 --output=gaia.fits \ 4171 --column=ra,dec,parallax 4172 4173However, this is much easier using the auto-completion feature: 4174 4175 $ astquery gaia --dataset=edr3 --output=gaia.fits --column=<[TAB]> 4176 4177After pressing <[TAB]>, a full list of gaia edr3 dataset column names 4178will be displayed. Typing the first key of the desired column and 4179pressing <[TAB]> again will limit the displayed list to only the 4180matching ones until the desired column is found. 4181 4182* Menu: 4183 4184* Bash TAB completion tutorial:: Fast tutorial to get you started on concepts. 4185* Implementing TAB completion in Gnuastro:: How Gnuastro uses Bash auto-completion features. 4186 4187 4188File: gnuastro.info, Node: Bash TAB completion tutorial, Next: Implementing TAB completion in Gnuastro, Prev: Bash programmable completion, Up: Bash programmable completion 4189 419012.8.1 Bash TAB completion tutorial 4191----------------------------------- 4192 4193When a user presses the <[TAB]> key while typing commands, Bash will 4194inspect the input to find a relevant “completion specification”, or 4195‘compspec’. If available, the ‘compspec’ will generate a list of 4196possible suggestions to complete the current word. A custom ‘compsec’ 4197can be generated for any command using bash completion builtins(1) and 4198the bash variables that start with the ‘COMP’ keyword(2). 4199 4200 First, let’s see a quick example of how you can make a completion 4201script in just one line of code. With the command below, we are asking 4202Bash to give us three suggestions for ‘echo’: ‘foo’, ‘bar’ and ‘bAr’. 4203Please run it in your terminal for the next steps. 4204 4205 $ complete -W "foo bar bAr" echo 4206 4207 The possible completion suggestions are fed into ‘complete’ using the 4208‘-W’ option followed by a list of space delimited words. Let’s see it 4209in action: 4210 4211 $ echo <[TAB][TAB]> 4212 bar bAr foo 4213 4214 Nicely done! Just note that the strings are sorted alphabetically, 4215not in the original order. Also, an arbitrary number of space 4216characters are printed between them (based on the number of suggestions 4217and terminal size and etc). Now, if you type ‘f’ and press <[TAB]>, 4218bash will automatically figure out that you wanted ‘foo’ and it be 4219completed right away: 4220 4221 $ myprogram f<[TAB]> 4222 $ myprogram foo 4223 4224However, nothing will happen if you type ‘b’ and press <[TAB]> only 4225once. This is because of the ambiguity: there isn’t enough information 4226to figure out which suggestion you want: ‘bar’ or ‘bAr’? So, if you 4227press <[TAB]> twice, it will print out all the options that start with 4228‘b’: 4229 4230 $ echo b<[TAB][TAB]> 4231 bar bAr 4232 $ echo ba<[TAB]> 4233 $ echo bar 4234 4235 Not bad for a simple program. But what if you need more control? By 4236passing the ‘-F’ option to ‘complete’ instead of ‘-W’, it will run a 4237function for generating the suggestions, instead of using a static 4238string. For example, let’s assume that the expected value after ‘foo’ 4239is the number of files in the current directory. Since the logic is 4240getting more complex, let’s write and save the commands below into a 4241shell script with an arbitrary name such as ‘completion-tutorial.sh’: 4242 4243 $ cat completion-tutorial.sh 4244 _echo(){ 4245 if [ "$3" == "foo" ]; then 4246 COMPREPLY=( $(ls | wc -l) ) 4247 else 4248 COMPREPLY=( $(compgen -W "foo bar bAr" -- "$2") ) 4249 fi 4250 } 4251 complete -F _echo echo 4252 4253We will look at it in detail soon. But for now, let’s ‘source’ the file 4254into your current terminal and check if it works as expected: 4255 4256 $ source completion-tutorial.sh 4257 $ echo <[TAB][TAB]> 4258 foo bar bAr 4259 $ echo foo <[TAB]> 4260 $ touch empty.txt 4261 $ echo foo <[TAB]> 4262 4263Success! As you see, this allows for setting up highly customized 4264completion scripts. Now let’s have a closer look at the 4265‘completion-tutorial.sh’ completion script from above. First, the ‘-F’ 4266option in front the ‘complete’ command indicates that we want shell to 4267execute the ‘_echo’ function whenever ‘echo’ is called. As a 4268convention, the function name should be the same as the program name, 4269but prefixed with an underscore (‘_’). 4270 4271 Within the ‘_echo’ function, we’re checking if ‘$3’ is equal to 4272‘foo’. In Bash’s auto-complete, ‘$3’ means the word before current 4273cursor position. In fact, these are the arguments that the ‘_echo’ 4274function is receiving: 4275 4276‘$1’ 4277 The name of the command, here it is ‘echo’. 4278 4279‘$2’ 4280 The current word being completed (empty unless we are in the middle 4281 of typing a word). 4282 4283‘$3’ 4284 The word before the word being completed. 4285 4286 To tell the completion script what to reply with, we use the 4287‘COMPREPLY’ array. This array holds all the suggestions that ‘complete’ 4288will show for the user in the end. In the example above, we simply give 4289it the string output of ‘ls | wc -l’. 4290 4291 Finally, we have the ‘compgen’ command. According to bash 4292programmable completion builtins manual, the command ‘compgen [OPTION] 4293[WORD]’ generates possible completion matches for ‘[WORD]’ according to 4294‘[OPTIONS]’. Using the ‘-W’ option asks ‘compgen’ to generate a list of 4295words from an input string. This is known as Word Splitting(3). 4296‘compgen’ will automatically use the ‘$IFS’ variable to split the string 4297into a list of words. You can check the default delimiters by calling: 4298 4299 $ printf %q "$IFS" 4300 4301The default value of ‘$IFS’ might be ‘ \t\n’. This means the SPACE, 4302TAB, and New-line characters. Finally, notice the ‘-- "$2"’ in this 4303command: 4304 4305 COMPREPLY=( $(compgen -W "foo bar bAr" -- "$2") ) 4306 4307Here, the ‘--’ instructs ‘compgen’ to only reply with a list of words 4308that match ‘$2’, i.e. the current word being completed. That is why 4309when you type the letter ‘b’, ‘complete’ will reply only with its 4310matches (‘bar’ and ‘bAr’), and will exclude ‘foo’. 4311 4312 Let’s get a little more realistic, and develop a very basic 4313completion script for one of Gnuastro’s programs. Since the ‘--help’ 4314option will list all the options available in Gnuastro’s programs, we 4315are going to use its output and create a very basic TAB completion for 4316it. Note that the actual TAB completion in Gnuastro is a little more 4317complex than this and fully described in *note Implementing TAB 4318completion in Gnuastro::. But this is a good exercise to get started. 4319 4320 We’ll use ‘asttable’ as the demo, and the goal is to suggest all 4321options that this program has to offer. You can print all of them (with 4322a lot of extra information) with this command: 4323 4324 $ asttable --help 4325 4326 Let’s write an ‘awk’ script that prints all of the long options. 4327When printing the option names we can safely ignore the short options 4328because if a user knows about the short options, s/he already knows 4329exactly what they want! Also, due to their single-character length, 4330they will be too cryptic without their descriptions. 4331 4332 One way to catch the long options is through ‘awk’ as shown below. 4333We only keep the lines that 1) starting with an empty space, 2) their 4334first no-white character is ‘-’ and that have the format of ‘--’ 4335followed by any number of numbers or characters. Within those lines, if 4336the first word ends in a comma (‘,’), the first word is the short 4337option, so we want the second word (which is the long option). 4338Otherwise, the first word is the long option. But for options that take 4339a value, this will also include the format of the value (for example 4340‘--column=STR’). So with a ‘sed’ command, we remove everything that is 4341after the equal sign, but keep the equal sign itself (to highlight to 4342the user that this option should have a value). 4343 4344 $ asttable --help \ 4345 | awk '/^ / && $1 ~ /^-/ && /--+[a-zA-Z0-9]*/ { \ 4346 if($1 ~ /,$/) name=$2; \ 4347 else name=$1; \ 4348 print name}' \ 4349 | sed -e's|=.*|=|' 4350 4351 If we wanted to show all the options to the user, we could simply 4352feed the values of the command above to ‘compgen’ and ‘COMPREPLY’ 4353subsequently. But, we need _smarter_ completions: we want to offer 4354suggestions based on the previous options that have already been typed 4355in. Just Beware! Sometimes the program might not be acting as you 4356expected. In that case, using debug messages can clear things up. You 4357can add a ‘echo’ command before the completion function ends, and check 4358all current variables. This can save a lot of headaches, since things 4359can get complex. 4360 4361 Take the option ‘--wcsfile=’ for example. This option accepts a FITS 4362file. Usually, the user is trying to feed a FITS file from the current 4363directory. So it would be nice if we could help them and print only a 4364list of FITS files sitting in the current directory – or whatever 4365directory they have typed-in so far. 4366 4367 But there’s a catch. When splitting the user’s input line, Bash will 4368consider ‘=’ as a separate word. To avoid getting caught in changing 4369the ‘IFS’ or ‘WORDBREAKS’ values, we will simply check for ‘=’ and act 4370accordingly. That is, if the previous word is a ‘=’, we’ll ignore it 4371and take the word before that as the previous word. Also, if the 4372current word is a ‘=’, ignore it completely. Taking all of that into 4373consideration, the code below might serve well: 4374 4375_asttable(){ 4376 if [ "$2" = "=" ]; then word="" 4377 else word="$2" 4378 fi 4379 4380 if [ "$3" = "=" ]; then prev="${COMP_WORDS[COMP_CWORD-2]}" 4381 else prev="${COMP_WORDS[COMP_CWORD-1]}" 4382 fi 4383 4384 case "$prev" in 4385 --wcsfile) 4386 COMPREPLY=( $(compgen -f -X "!*.[fF][iI][tT][sS]" -- "$word") ) 4387 ;; 4388 esac 4389} 4390complete -o nospace -F _asttable asttable 4391 4392To test the code above, write it into ‘asttable-tutorial.sh’, and load 4393it into your running terminal with this command: 4394 4395 $ source asttable-tutorial.sh 4396 4397 If you then go to a directory that has at least one FITS file (with a 4398‘.fits’ suffix, among other files), you can checkout the function by 4399typing the following command. You will see that only files ending in 4400‘.fits’ are shown, not any other file. 4401 4402 asttable --wcsfile=[TAB][TAB] 4403 4404 The code above first identifies the current and previous words. It 4405then checks if the previous word is equal to ‘--wcsfile’ and if so, 4406fills ‘COMPREPLY’ array with the necessary suggestions. We are using 4407‘case’ here (instead of ‘if’) because in a real scenario, we need to 4408check many more values and ‘case’ is far better suited for such cases 4409(cleaner and more efficient code). 4410 4411 The ‘-f’ option in ‘compgen’ indicates we’re looking for a file. The 4412‘-X’ option _filters out_ the filenames that match the next regular 4413expression pattern. Therefore we should start the regular expression 4414with ‘!’ if we want the files matching the regular expression. The ‘-- 4415"$word"’ component collects only filenames that match the current word 4416being typed. And last but not least, the ‘-o nospace’ option in the 4417‘complete’ command instructs the completion script to _not_ append a 4418white space after each suggestion. That is important because the long 4419format of an option, its value is more clear when it sticks to the 4420option name with a ‘=’ sign. 4421 4422 You have now written a very basic and working TAB completion script 4423that can easily be generalized to include more options (and be good for 4424a single/simple program). However, Gnuastro has many programs that 4425share many similar things and the options are not independent. Also, 4426complex situations do often come up: for example some people use a 4427‘.fit’ suffix for FITS files and others don’t even use a suffix at all! 4428So in practice, things need to get a little more complicated, but the 4429core concept is what you learnt in this section. We just modularize the 4430process (breaking logically independent steps into separate functions to 4431use in different situations). In *note Implementing TAB completion in 4432Gnuastro::, we will review the generalities of Gnuastro’s implementation 4433of Bash TAB completion. 4434 4435 ---------- Footnotes ---------- 4436 4437 (1) 4438<https://www.gnu.org/software/bash/manual/html_node/Programmable-Completion-Builtins.html> 4439 4440 (2) 4441<https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html> 4442 4443 (3) 4444<https://www.gnu.org/software/bash/manual/html_node/Word-Splitting.html> 4445 4446 4447File: gnuastro.info, Node: Implementing TAB completion in Gnuastro, Prev: Bash TAB completion tutorial, Up: Bash programmable completion 4448 444912.8.2 Implementing TAB completion in Gnuastro 4450---------------------------------------------- 4451 4452The basics of Bash auto-completion was reviewed in *note Bash TAB 4453completion tutorial::. Gnuastro is a very complex package of many 4454programs, that have many similar features, so implementing those 4455principles in an easy to maintain manner requires a modular solution. 4456As a result, Bash’s TAB completion is implemented as multiple files in 4457Gnuastro: 4458 4459‘bin/completion.bash.built’ (in build directory, automatically created) 4460 This file contains the values of all Gnuastro options or arguments 4461 that take fixed strings as values (not file names). For example 4462 the names of Arithmetic’s operators (see *note Arithmetic 4463 operators::), or spectral line names (like ‘--obsline’ in *note 4464 CosmicCalculator input options::). 4465 4466 This file is created automatically during the building of Gnuastro. 4467 The recipe to build it is available in Gnuastro’s top-level 4468 ‘Makefile.am’ (under the target ‘bin/completion.bash’). It parses 4469 the respective Gnuastro source file that contains the necessary 4470 user-specified strings. All the acceptable values values are then 4471 stored as shell variables (within a function). 4472 4473‘bin/completion.bash.in’ (in source directory, under version control) 4474 All the low-level completion functions that are common to all 4475 programs are stored here. It thus contains functions that will 4476 parse the command-line or files, or suggest the completion replies. 4477 4478‘PROGNAME-complete.bash’ (in source directory, under version control) 4479 All Gnuastro programs contain a ‘PROGNAME-complete.bash’ script 4480 within their source (for more on the fixed files of each program, 4481 see *note Program source::). This file contains the very 4482 high-level (program-specific) Bash programmable completion features 4483 that are almost always defined in Gnuastro-generic Bash completion 4484 file (‘bin/completion.bash.in’). 4485 4486 The top-level function that is called by Bash should be called 4487 ‘_gnuastro_autocomplete_PROGNAME’ and its last line should be the 4488 ‘complete’ command of Bash which calls this function. The contents 4489 of ‘_gnuastro_autocomplete_PROGNAME’ are almost identical for all 4490 the programs, it is just a very high-level function that either 4491 calls ‘_gnuastro_autocomplete_PROGNAME_arguments’ to manage 4492 suggestions for the program’s arguments or 4493 ‘_gnuastro_autocomplete_PROGNAME_option_value’ to manage 4494 suggestions for the program’s option values. 4495 4496The scripts above follow the following conventions. After reviewing the 4497list, please also look into the functions for examples of each point. 4498 • No global shell variables in any completion script: the contents of 4499 the files above are directly loaded into the user’s environment. 4500 So to keep the user’s environment clean and avoid annoyance to the 4501 users, everything should be defined as shell functions, and any 4502 variable within the functions should be set as ‘local’. 4503 4504 • All the function names should start with 4505 ‘‘_gnuastro_autocomplete_’’, again to avoid populating the user’s 4506 function name-space with possibly conflicting names. 4507 4508 • Outputs of functions should be written in the ‘local’ variables of 4509 the higher-level functions that called them. 4510 4511 4512File: gnuastro.info, Node: Developer's checklist, Next: Gnuastro project webpage, Prev: Bash programmable completion, Up: Developing 4513 451412.9 Developer’s checklist 4515========================== 4516 4517This is a checklist of things to do after applying your 4518changes/additions in Gnuastro: 4519 4520 1. If the change is non-trivial, write test(s) in the 4521 ‘tests/progname/’ directory to test the change(s)/addition(s) you 4522 have made. Then add their file names to ‘tests/Makefile.am’. 4523 4524 2. If your change involves a change in command-line behavior of a 4525 Gnuastro program or script (for example, adding a new option or 4526 argument), create or update the respective 4527 ‘bin/PROGNAME/completion.sh’ file described under the *note Bash 4528 programmable completion:: section. 4529 4530 3. Run ‘$ make check’ to make sure everything is working correctly. 4531 4532 4. Make sure the documentation (this book) is completely up to date 4533 with your changes, see *note Documentation::. 4534 4535 5. Commit the change to your issue branch (see *note Production 4536 workflow:: and *note Forking tutorial::). Afterwards, run 4537 Autoreconf to generate the appropriate version number: 4538 4539 $ autoreconf -f 4540 4541 6. Finally, to make sure everything will be built, installed and 4542 checked correctly run the following command (after re-configuring, 4543 and re-building). To greatly speed up the process, use multiple 4544 threads (8 in the example below, change it appropriately) 4545 4546 $ make distcheck -j8 4547 4548 This command will create a distribution file (ending with 4549 ‘.tar.gz’) and try to compile it in the most general cases, then it 4550 will run the tests on what it has built in its own 4551 mini-environment. If ‘$ make distcheck’ finishes successfully, 4552 then you are safe to send your changes to us to implement or for 4553 your own purposes. See *note Production workflow:: and *note 4554 Forking tutorial::. 4555 4556 4557File: gnuastro.info, Node: Gnuastro project webpage, Next: Developing mailing lists, Prev: Developer's checklist, Up: Developing 4558 455912.10 Gnuastro project webpage 4560============================== 4561 4562Gnuastro’s central management hub 4563(https://savannah.gnu.org/projects/gnuastro/)(1) is located on GNU 4564Savannah (https://savannah.gnu.org/)(2). Savannah is the central 4565software development management system for many GNU projects. Through 4566this central hub, you can view the list of activities that the 4567developers are engaged in, their activity on the version controlled 4568source, and other things. Each defined activity in the development 4569cycle is known as an ‘issue’ (or ‘item’). An issue can be a bug (see 4570*note Report a bug::), or a suggested feature (see *note Suggest new 4571feature::) or an enhancement or generally any _one_ job that is to be 4572done. In Savannah, issues are classified into three categories or 4573‘tracker’s: 4574 4575Support 4576 This tracker is a way that (possibly anonymous) users can get in 4577 touch with the Gnuastro developers. It is a complement to the 4578 bug-gnuastro mailing list (see *note Report a bug::). Anyone can 4579 post an issue to this tracker. The developers will not submit an 4580 issue to this list. They will only reassign the issues in this 4581 list to the other two trackers if they are valid(3). Ideally (when 4582 the developers have time to put on Gnuastro, please don’t forget 4583 that Gnuastro is a volunteer effort), there should be no open items 4584 in this tracker. 4585 4586Bugs 4587 This tracker contains all the known bugs in Gnuastro (problems with 4588 the existing tools). 4589 4590Tasks 4591 The items in this tracker contain the future plans (or new 4592 features/capabilities) that are to be added to Gnuastro. 4593 4594All the trackers can be browsed by a (possibly anonymous) visitor, but 4595to edit and comment on the Bugs and Tasks trackers, you have to be a 4596registered on Savannah. When posting an issue to a tracker, it is very 4597important to choose the ‘Category’ and ‘Item Group’ options accurately. 4598The first contains a list of all Gnuastro’s programs along with 4599‘Installation’, ‘New program’ and ‘Webpage’. The “Item Group” contains 4600the nature of the issue, for example if it is a ‘Crash’ in the software 4601(a bug), or a problem in the documentation (also a bug) or a feature 4602request or an enhancement. 4603 4604 The set of horizontal links on the top of the page (Starting with 4605‘Main’ and ‘Homepage’ and finishing with ‘News’) are the easiest way to 4606access these trackers (and other major aspects of the project) from any 4607part of the project web page. Hovering your mouse over them will open a 4608drop down menu that will link you to the different things you can do on 4609each tracker (for example, ‘Submit new’ or ‘Browse’). When you browse 4610each tracker, you can use the “Display Criteria” link above the list to 4611limit the displayed issues to what you are interested in. The 4612‘Category’ and ‘Group Item’ (explained above) are a good starting point. 4613 4614 Any new issue that is submitted to any of the trackers, or any 4615comments that are posted for an issue, is directly forwarded to the 4616gnuastro-devel mailing list 4617(<https://lists.gnu.org/mailman/listinfo/gnuastro-devel>, see *note 4618Developing mailing lists:: for more). This will allow anyone interested 4619to be up to date on the over-all development activity in Gnuastro and 4620will also provide an alternative (to Savannah) archiving for the 4621development discussions. Therefore, it is not recommended to directly 4622post an email to this mailing list, but do all the activities (for 4623example add new issues, or comment on existing ones) on Savannah. 4624 4625*Do I need to be a member in Savannah to contribute to Gnuastro?* No. 4626 4627 The full version controlled history of Gnuastro is available for 4628anonymous download or cloning. See *note Production workflow:: for a 4629description of Gnuastro’s Integration-Manager Workflow. In short, you 4630can either send in patches, or make your own fork. If you choose the 4631latter, you can push your changes to your own fork and inform us. We 4632will then pull your changes and merge them into the main project. 4633Please see *note Forking tutorial:: for a tutorial. 4634 4635 ---------- Footnotes ---------- 4636 4637 (1) <https://savannah.gnu.org/projects/gnuastro/> 4638 4639 (2) <https://savannah.gnu.org/> 4640 4641 (3) Some of the issues registered here might be due to a mistake on 4642the user’s side, not an actual bug in the program. 4643 4644 4645File: gnuastro.info, Node: Developing mailing lists, Next: Contributing to Gnuastro, Prev: Gnuastro project webpage, Up: Developing 4646 464712.11 Developing mailing lists 4648============================== 4649 4650To keep the developers and interested users up to date with the activity 4651and discussions within Gnuastro, there are two mailing lists which you 4652can subscribe to: 4653 4654‘gnuastro-devel@gnu.org’ 4655(at <https://lists.gnu.org/mailman/listinfo/gnuastro-devel>) 4656 4657 All the posts made in the support, bugs and tasks discussions of 4658 *note Gnuastro project webpage:: are also sent to this mailing 4659 address and archived. By subscribing to this list you can stay up 4660 to date with the discussions that are going on between the 4661 developers before, during and (possibly) after working on an issue. 4662 All discussions are either in the context of bugs or tasks which 4663 are done on Savannah and circulated to all interested people 4664 through this mailing list. Therefore it is not recommended to post 4665 anything directly to this mailing list. Any mail that is sent to 4666 it from Savannah to this list has a link under the title “Reply to 4667 this item at:”. That link will take you directly to the issue 4668 discussion page, where you can read the discussion history or join 4669 it. 4670 4671 While you are posting comments on the Savannah issues, be sure to 4672 update the meta-data. For example if the task/bug is not assigned 4673 to anyone and you would like to take it, change the “Assigned to” 4674 box, or if you want to report that it has been applied, change the 4675 status and so on. All these changes will also be circulated with 4676 the email very clearly. 4677 4678‘gnuastro-commits@gnu.org’ 4679(at <https://lists.gnu.org/mailman/listinfo/gnuastro-commits>) 4680 4681 This mailing list is defined to circulate all commits that are done 4682 in Gnuastro’s version controlled source, see *note Version 4683 controlled source::. If you have any ideas, or suggestions on the 4684 commits, please use the bug and task trackers on Savannah to 4685 followup the discussion, do not post to this list. All the commits 4686 that are made for an already defined issue or task will state the 4687 respective ID so you can find it easily. 4688 4689 4690File: gnuastro.info, Node: Contributing to Gnuastro, Prev: Developing mailing lists, Up: Developing 4691 469212.12 Contributing to Gnuastro 4693============================== 4694 4695You have this great idea or have found a good fix to a problem which you 4696would like to implement in Gnuastro. You have also become familiar with 4697the general design of Gnuastro in the previous sections of this chapter 4698(see *note Developing::) and want to start working on and sharing your 4699new addition/change with the whole community as part of the official 4700release. This is great and your contribution is most welcome. This 4701section and the next (see *note Developer's checklist::) are written in 4702the hope of making it as easy as possible for you to share your great 4703idea with the community. 4704 4705 In this section we discuss the final steps you have to take: legal 4706and technical. From the legal perspective, the copyright of any work 4707you do on Gnuastro has to be assigned to the Free Software Foundation 4708(FSF) and the GNU operating system, or you have to sign a disclaimer. 4709We do this to ensure that Gnuastro can remain free in the future, see 4710*note Copyright assignment::. From the technical point of view, in this 4711section we also discuss commit guidelines (*note Commit guidelines::) 4712and the general version control workflow of Gnuastro in *note Production 4713workflow::, along with a tutorial in *note Forking tutorial::. 4714 4715 Recall that before starting the work on your idea, be sure to 4716checkout the bugs and tasks trackers in *note Gnuastro project webpage:: 4717and announce your work there so you don’t end up spending time on 4718something others have already worked on, and also to attract similarly 4719interested developers to help you. 4720 4721* Menu: 4722 4723* Copyright assignment:: Copyright has to be assigned to the FSF. 4724* Commit guidelines:: Guidelines for commit messages. 4725* Production workflow:: Submitting your commits (work) for inclusion. 4726* Forking tutorial:: Tutorial on workflow steps with Git. 4727 4728 4729File: gnuastro.info, Node: Copyright assignment, Next: Commit guidelines, Prev: Contributing to Gnuastro, Up: Contributing to Gnuastro 4730 473112.12.1 Copyright assignment 4732---------------------------- 4733 4734Gnuastro’s copyright is owned by the Free Software Foundation (FSF) to 4735ensure that Gnuastro always remains free. The FSF has also provided a 4736Contributor FAQ (https://www.fsf.org/licensing/contributor-faq) to 4737further clarify the reasons, so we encourage you to read it. Professor 4738Eben Moglen, of the Columbia University Law School has given a nice 4739summary of the reasons for this at 4740<https://www.gnu.org/licenses/why-assign>. Below we are copying it 4741verbatim for self consistency (in case you are offline or reading in 4742print). 4743 4744 Under US copyright law, which is the law under which most free 4745 software programs have historically been first published, there are 4746 very substantial procedural advantages to registration of 4747 copyright. And despite the broad right of distribution conveyed by 4748 the GPL, enforcement of copyright is generally not possible for 4749 distributors: only the copyright holder or someone having 4750 assignment of the copyright can enforce the license. If there are 4751 multiple authors of a copyrighted work, successful enforcement 4752 depends on having the cooperation of all authors. 4753 4754 In order to make sure that all of our copyrights can meet the 4755 record keeping and other requirements of registration, and in order 4756 to be able to enforce the GPL most effectively, FSF requires that 4757 each author of code incorporated in FSF projects provide a 4758 copyright assignment, and, where appropriate, a disclaimer of any 4759 work-for-hire ownership claims by the programmer’s employer. That 4760 way we can be sure that all the code in FSF projects is free code, 4761 whose freedom we can most effectively protect, and therefore on 4762 which other developers can completely rely. 4763 4764 Please get in touch with the Gnuastro maintainer (currently Mohammad 4765Akhlaghi, mohammad -at- akhlaghi -dot- org) to follow the procedures. 4766It is possible to do this for each change (good for for a single 4767contribution), and also more generally for all the changes/additions you 4768do in the future within Gnuastro. So if you have already assigned the 4769copyright of your work on another GNU software to the FSF, it should be 4770done again for Gnuastro. The FSF has staff working on these legal 4771issues and the maintainer will get you in touch with them to do the 4772paperwork. The maintainer will just be informed in the end so your 4773contributions can be merged within the Gnuastro source code. 4774 4775 Gnuastro will gratefully acknowledge (see *note Acknowledgments::) 4776all the people who have assigned their copyright to the FSF and have 4777thus helped to guarantee the freedom and reliability of Gnuastro. The 4778Free Software Foundation will also acknowledge your copyright 4779contributions in the Free Software Supporter: 4780<https://www.fsf.org/free-software-supporter> which will circulate to a 4781very large community (225,910 people in July 2021). See the archives 4782for some examples and subscribe to receive interesting updates. The 4783very active code contributors (or developers) will also be recognized as 4784project members on the Gnuastro project web page (see *note Gnuastro 4785project webpage::) and can be given a ‘gnu.org’ email address. So your 4786very valuable contribution and copyright assignment will not be 4787forgotten and is highly appreciated by a very large community. If you 4788are reluctant to sign an assignment, a disclaimer is also acceptable. 4789 4790*Do I need a disclaimer from my university or employer?* It depends on 4791the contract with your university or employer. From the FSF’s 4792‘/gd/gnuorg/conditions.text’: “If you are employed to do programming, or 4793have made an agreement with your employer that says it owns programs you 4794write, we need a signed piece of paper from your employer disclaiming 4795rights to” Gnuastro. The FSF’s copyright clerk will kindly help you 4796decide, please consult the following email address: “assign -at- gnu 4797-dot- org”. 4798 4799 4800File: gnuastro.info, Node: Commit guidelines, Next: Production workflow, Prev: Copyright assignment, Up: Contributing to Gnuastro 4801 480212.12.2 Commit guidelines 4803------------------------- 4804 4805To be able to cleanly integrate your work with the other developers, 4806*never commit on the ‘master’ branch* (see *note Production workflow:: 4807for a complete discussion and *note Forking tutorial:: for a cookbook 4808example). In short, leave ‘master’ only for changes you fetch, or pull 4809from the official repository (see *note Synchronizing::). 4810 4811 In the Gnuastro commit messages, we strive to follow these standards. 4812Note that in the early phases of Gnuastro’s development, we are 4813experimenting and so if you notice earlier commits don’t satisfy some of 4814the guidelines below, it is because they predate that guideline. 4815 4816Commit title 4817 The commits have to start with one short descriptive title. The 4818 title is separated from the body with one blank line. Run ‘git 4819 log’ to see some of the most recent commit messages as an example. 4820 In general, the title should satisfy the following conditions: 4821 4822 • It is best for the title to be short, about 60 (or even 50) 4823 characters. Most emulated command-line terminals are about 80 4824 characters wide. However, we should also allow for the commit 4825 hashes which are printed in ‘git log --oneline’, and also 4826 branch names or the graph structure outputs of ‘git log’ which 4827 are also commonly used. 4828 4829 • The title should not finish with any full-stops or periods 4830 (‘<.>’). 4831 4832Commit body 4833 The body of the commit message is separated from the title by one 4834 empty line. Recall that anyone who has subscribed to 4835 ‘gnuastro-commits’ mailing list will get the commit in their email 4836 after it has been pushed to ‘master’. People will also read them 4837 when they synchronize with the main Gnuastro repository (see *note 4838 Synchronizing::). Finally, the commit messages will later be used 4839 to update the ‘NEWS’ file on each release. Therefore the commit 4840 message body plays a very important role in the development of 4841 Gnuastro, so please adhere to the following guidelines. 4842 4843 • The body should be very descriptive. Start the commit message 4844 body by explaining what changes your commit makes from a 4845 user’s perspective (added, changed, or removed options, or 4846 arguments to programs or libraries, or modified algorithms, or 4847 new installation step, etc). 4848 4849 • Try to explain the committed contents as best as you can. 4850 Recall that the readers of your commit message do not 4851 necessarily have your current background. After some time you 4852 will also forget the context, so this request is not just for 4853 others(1). Therefore be very descriptive and explain as much 4854 as possible: what the bug/task was, justify the way you fixed 4855 it and discuss other possible solutions that you might not 4856 have included. For the last item, it is best to discuss them 4857 thoroughly as comments in the appropriate section of the code, 4858 but only give a short summary in the commit message. Note 4859 that all added and removed source code lines will also be 4860 circulated in the ‘gnuastro-commits’ mailing list. 4861 4862 • Like all other Gnuastro’s text files, the lines in the commit 4863 body should not be longer than 75 characters, see *note Coding 4864 conventions::. This is to ensure that on standard terminal 4865 emulators (with 80 character width), the ‘git log’ output can 4866 be cleanly displayed (note that the commit message is indented 4867 in the output of ‘git log’). If you use Emacs, Gnuastro’s 4868 ‘.dir-locals.el’ file will ensure that your commits satisfy 4869 this condition (using <M-q>). 4870 4871 • When the commit is related to a task or a bug, please include 4872 the respective ID (in the format of ‘bug/task #ID’, note the 4873 space) in the commit message (from *note Gnuastro project 4874 webpage::) for interested people to be able to followup the 4875 discussion that took place there. If the commit fixes a bug 4876 or finishes a task, the recommended way is to add a line after 4877 the body with ‘‘This fixes bug #ID.’’, or ‘‘This finishes task 4878 #ID.’’. Don’t assume that the reader has internet access to 4879 check the bug’s full description when reading the commit 4880 message, so give a short introduction too. 4881 4882 Below you can see a good commit message example (don’t forget to read 4883it, it has tips for you). After reading this, please run ‘git log’ on 4884the ‘master’ branch and read some of the recent commits for more 4885realistic examples. 4886 4887 The first line should be the title of the commit 4888 4889 An empty line is necessary after the title so Git doesn't confuse 4890 lines. This top paragraph of the body of the commit usually describes 4891 the reason this commit was done. Therefore it usually starts with 4892 "Until now ...". It is very useful to explain the reason behind the 4893 change, things that aren't immediately obvious when looking into the 4894 code. You don't need to list the names of the files, or what lines 4895 have been changed, don't forget that the code changes are fully 4896 stored within Git :-). 4897 4898 In the second paragraph (or any later paragraph!) of the body, we 4899 describe the solution and why (not "how"!) the particular solution 4900 was implemented. So we usually start this part of the commit body 4901 with "With this commit ...". Again, you don't need to go into the 4902 details that can be seen from the 'git diff' command (like the 4903 file names that have been changed or the code that has been 4904 implemented). The important thing here is the things that aren't 4905 immediately obvious from looking into the code. 4906 4907 You can continue the explanation and it is encouraged to be very 4908 explicit about the "human factor" of the change as much as possible, 4909 not technical details. 4910 4911 ---------- Footnotes ---------- 4912 4913 (1) <http://catb.org/esr/writings/unix-koans/prodigy.html> 4914 4915 4916File: gnuastro.info, Node: Production workflow, Next: Forking tutorial, Prev: Commit guidelines, Up: Contributing to Gnuastro 4917 491812.12.3 Production workflow 4919--------------------------- 4920 4921Fortunately ‘Pro Git’ has done a wonderful job in explaining the 4922different workflows in Chapter 5(1) and in particular the 4923“Integration-Manager Workflow” explained there. The implementation of 4924this workflow is nicely explained in Section 5.2(2) under 4925“Forked-Public-Project”. We have also prepared a short tutorial in 4926*note Forking tutorial::. Anything on the master branch should always 4927be tested and ready to be built and used. As described in ‘Pro Git’, 4928there are two methods for you to contribute to Gnuastro in the 4929Integration-Manager Workflow: 4930 4931 1. You can send commit patches by email as fully explained in ‘Pro 4932 Git’. This is good for your first few contributions. Just note 4933 that raw patches (containing only the diff) do not have any 4934 meta-data (author name, date, etc). Therefore they will not allow 4935 us to fully acknowledge your contributions as an author in 4936 Gnuastro: in the ‘AUTHORS’ file and at the start of the PDF book. 4937 These author lists are created automatically from the version 4938 controlled source. 4939 4940 To receive full acknowledgment when submitting a patch, is thus 4941 advised to use Git’s ‘format-patch’ tool. See Pro Git’s Public 4942 project over email 4943 (https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project#Public-Project-over-Email) 4944 section for a nice explanation. If you would like to get more 4945 heavily involved in Gnuastro’s development, then you can try the 4946 next solution. 4947 4948 2. You can have your own forked copy of Gnuastro on any hosting site 4949 you like (GitHub, GitLab, BitBucket, etc) and inform us when your 4950 changes are ready so we merge them in Gnuastro. This is more 4951 suited for people who commonly contribute to the code (see *note 4952 Forking tutorial::). 4953 4954 In both cases, your commits (with your name and information) will be 4955preserved and your contributions will thus be fully recorded in the 4956history of Gnuastro and in the ‘AUTHORS’ file and this book (second page 4957in the PDF format) once they have been incorporated into the official 4958repository. Needless to say that in such cases, be sure to follow the 4959bug or task trackers (or subscribe to the ‘gnuastro-devel’ mailing list) 4960and contact us before hand so you don’t do something that someone else 4961is already working on. In that case, you can get in touch with them and 4962help the job go on faster, see *note Gnuastro project webpage::. This 4963workflow is currently mostly borrowed from the general recommendations 4964of Git(3) and GitHub. But since Gnuastro is currently under heavy 4965development, these might change and evolve to better suit our needs. 4966 4967 ---------- Footnotes ---------- 4968 4969 (1) 4970<http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows> 4971 4972 (2) 4973<http://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project> 4974 4975 (3) 4976<https://github.com/git/git/blob/master/Documentation/SubmittingPatches> 4977 4978 4979File: gnuastro.info, Node: Forking tutorial, Prev: Production workflow, Up: Contributing to Gnuastro 4980 498112.12.4 Forking tutorial 4982------------------------ 4983 4984This is a tutorial on the second suggested method (commonly known as 4985forking) that you can submit your modifications in Gnuastro (see *note 4986Production workflow::). 4987 4988 To start, please create an empty repository on your hosting service 4989web page (we recommend GitLab(1)). If this is your first hosted 4990repository on the web page, you also have to upload your public SSH 4991key(2) for the ‘git push’ command below to work. Here we’ll assume you 4992use the name ‘janedoe’ to refer to yourself everywhere and that you 4993choose ‘gnuastro-janedoe’ as the name of your Gnuastro fork. Any online 4994hosting service will give you an address (similar to the 4995‘‘git@gitlab.com:...’’ below) of the empty repository you have created 4996using their web page, use that address in the third line below. 4997 4998 $ git clone git://git.sv.gnu.org/gnuastro.git 4999 $ cd gnuastro 5000 $ git remote add janedoe git@gitlab.com:janedoe/gnuastro-janedoe.git 5001 $ git push janedoe master 5002 5003 The full Gnuastro history is now pushed onto your hosting service and 5004the ‘janedoe’ remote is now also following your ‘master’ branch. If you 5005run ‘git remote show REMOTENAME’ for the ‘origin’ and ‘janedoe’ remotes, 5006you will see their difference: the first has pull access and the second 5007doesn’t. This nicely summarizes the main idea behind this workflow: you 5008push to your remote repository, we pull from it and merge it into 5009‘master’, then you finalize it by pulling from the main repository. 5010 5011 To test (compile) your changes during your work, you will need to 5012bootstrap the version controlled source, see *note Bootstrapping:: for a 5013full description. The cloning process above is only necessary for your 5014first time setup, you don’t need to repeat it. However, please repeat 5015the steps below for each independent issue you intend to work on. 5016 5017 Let’s assume you have found a bug in ‘lib/statistics.c’’s median 5018calculating function. Before actually doing anything, please announce 5019it (see *note Report a bug::) so everyone knows you are working on it or 5020to find out others aren’t already working on it. With the commands 5021below, you make a branch, checkout to it, correct the bug, check if it 5022is indeed fixed, add it to the staging area, commit it to the new branch 5023and push it to your hosting service. But before all of them, make sure 5024that you are on the ‘master’ branch and that your ‘master’ branch is up 5025to date with the main Gnuastro repository with the first two commands. 5026 5027 $ git checkout master 5028 $ git pull 5029 $ git checkout -b bug-median-stats # Choose a descriptive name 5030 $ emacs lib/statistics.c 5031 $ # do your checks here 5032 $ git add lib/statistics.c 5033 $ git commit 5034 $ git push janedoe bug-median-stats 5035 5036 Your new branch is now on your hosted repository. Through the 5037respective tacker on Savannah (see *note Gnuastro project webpage::) you 5038can then let the other developers know that your ‘bug-median-stats’ 5039branch is ready. They will pull your work, test it themselves and if it 5040is ready to be merged into the main Gnuastro history, they will merge it 5041into the ‘master’ branch. After that is done, you can simply checkout 5042your local ‘master’ branch and pull all the changes from the main 5043repository. After the pull you can run ‘‘git log’’ as shown below, to 5044see how ‘bug-median-stats’ is merged with master. To finalize, you can 5045push all the changes to your hosted repository and delete the branch: 5046 5047 $ git checkout master 5048 $ git pull 5049 $ git log --oneline --graph --decorate --all 5050 $ git push janedoe master 5051 $ git branch -d bug-median-stats # delete local branch 5052 $ git push janedoe --delete bug-median-stats # delete remote branch 5053 5054 Just as a reminder, always keep your work on each issue in a separate 5055local and remote branch so work can progress on them independently. 5056After you make your announcement, other people might contribute to the 5057branch before merging it in to ‘master’, so this is very important. As 5058a final reminder: before starting each issue branch from ‘master’, be 5059sure to run ‘git pull’ in ‘master’ as shown above. This will enable you 5060to start your branch (work) from the most recent commit and thus 5061simplify the final merging of your work. 5062 5063 ---------- Footnotes ---------- 5064 5065 (1) See <https://www.gnu.org/software/repo-criteria-evaluation.html> 5066for an evaluation of the major existing repositories. Gnuastro uses GNU 5067Savannah (which also has the highest ranking in the evaluation), but for 5068starters, GitLab may be easier. 5069 5070 (2) For example see this explanation provided by GitLab: 5071<http://docs.gitlab.com/ce/ssh/README.html>. 5072 5073 5074File: gnuastro.info, Node: Gnuastro programs list, Next: Other useful software, Prev: Developing, Up: Top 5075 5076Appendix A Gnuastro programs list 5077********************************* 5078 5079GNU Astronomy Utilities 0.16, contains the following programs. They are 5080sorted in alphabetical order and a short description is provided for 5081each program. The description starts with the executable names in 5082‘thisfont’ followed by a pointer to the respective section in 5083parenthesis. Throughout this book, they are ordered based on their 5084context, please see the top-level contents for contextual ordering 5085(based on what they do). 5086 5087Arithmetic 5088 (‘astarithmetic’, see *note Arithmetic::) For arithmetic operations 5089 on multiple (theoretically unlimited) number of datasets (images). 5090 It has a large and growing set of arithmetic, mathematical, and 5091 even statistical operators (for example ‘+’, ‘-’, ‘*’, ‘/’, ‘sqrt’, 5092 ‘log’, ‘min’, ‘average’, ‘median’). 5093 5094BuildProgram 5095 (‘astbuildprog’, see *note BuildProgram::) Compile, link and run 5096 programs that depend on the Gnuastro library (see *note Gnuastro 5097 library::). This program will automatically link with the 5098 libraries that Gnuastro depends on, so there is no need to 5099 explicitly mention them every time you are compiling a Gnuastro 5100 library dependent program. 5101 5102ConvertType 5103 (‘astconvertt’, see *note ConvertType::) Convert astronomical data 5104 files (FITS or IMH) to and from several other standard image and 5105 data formats, for example TXT, JPEG, EPS or PDF. 5106 5107Convolve 5108 (‘astconvolve’, see *note Convolve::) Convolve (blur or smooth) 5109 data with a given kernel in spatial and frequency domain on 5110 multiple threads. Convolve can also do de-convolution to find the 5111 appropriate kernel to PSF-match two images. 5112 5113CosmicCalculator 5114 (‘astcosmiccal’, see *note CosmicCalculator::) Do cosmological 5115 calculations, for example the luminosity distance, distance 5116 modulus, comoving volume and many more. 5117 5118Crop 5119 (‘astcrop’, see *note Crop::) Crop region(s) from an image and 5120 stitch several images if necessary. Inputs can be in pixel 5121 coordinates or world coordinates. 5122 5123Fits 5124 (‘astfits’, see *note Fits::) View and manipulate FITS file 5125 extensions and header keywords. 5126 5127MakeCatalog 5128 (‘astmkcatalog’, see *note MakeCatalog::) Make catalog of labeled 5129 image (output of NoiseChisel). The catalogs are highly 5130 customizable and adding new calculations/columns is very 5131 straightforward. 5132 5133MakeNoise 5134 (‘astmknoise’, see *note MakeNoise::) Make (add) noise to an image, 5135 with a large set of random number generators and any seed. 5136 5137MakeProfiles 5138 (‘astmkprof’, see *note MakeProfiles::) Make mock 2D profiles in an 5139 image. The central regions of radial profiles are made with a 5140 configurable 2D Monte Carlo integration. It can also build the 5141 profiles on an over-sampled image. 5142 5143Match 5144 (‘astmatch’, see *note Match::) Given two input catalogs, find the 5145 rows that match with each other within a given aperture (may be an 5146 ellipse). 5147 5148NoiseChisel 5149 (‘astnoisechisel’, see *note NoiseChisel::) Detect signal in noise. 5150 It uses a technique to detect very faint and diffuse, irregularly 5151 shaped signal in noise (galaxies in the sky), using thresholds that 5152 are below the Sky value, see arXiv:1505.01664 5153 (http://arxiv.org/abs/1505.01664). 5154 5155Query 5156 (‘astquery’, see *note Query::) High-level interface to query 5157 pre-defined remote, or external databases, and directly download 5158 the required sub-tables on the command-line. 5159 5160Segment 5161 (‘astsegment’, see *note Segment::) Segment detected regions based 5162 on the structure of signal and the input dataset’s noise 5163 properties. 5164 5165Statistics 5166 (‘aststatistics’, see *note Statistics::) Statistical calculations 5167 on the input dataset (column in a table, image or datacube). 5168 5169Table 5170 (‘asttable’, *note Table::) Convert FITS binary and ASCII tables 5171 into other such tables, print them on the command-line, save them 5172 in a plain text file, or get the FITS table information. 5173 5174Warp 5175 (‘astwarp’, see *note Warp::) Warp image to new pixel grid. Any 5176 projective transformation or Homography can be applied to the input 5177 images. 5178 5179 The programs listed above are designed to be highly modular and 5180generic. Hence, they are naturally for lower-level operations. In 5181Gnuastro, higher-level operations (combining multiple programs, or 5182running a program in a special way), are done with installed Bash 5183scripts (all prefixed with ‘astscript-’). They can be run just like a 5184program and behave very similarly (with minor differences, see *note 5185Installed scripts::). 5186 5187‘astscript-ds9-region’ 5188 (See *note SAO DS9 region files from table::) Given a table (either 5189 as a file or from standard input), create an SAO DS9 region file 5190 from the requested positional columns (WCS or image coordinates). 5191 5192‘astscript-radial-profile’ 5193 (See *note Generate radial profile::) Calculate the radial profile 5194 of an object within an image. The object can be at any location in 5195 the image, using various measures (median, sigma-clipped mean and 5196 etc), and the radial distance can also be measured on any general 5197 ellipse. 5198 5199‘astscript-sort-by-night’ 5200 (See *note Sort FITS files by night::) Given a list of FITS files, 5201 and a HDU and keyword name (for a date), this script separates the 5202 files in the same night (possibly over two calendar days). 5203 5204 5205File: gnuastro.info, Node: Other useful software, Next: GNU Free Doc. License, Prev: Gnuastro programs list, Up: Top 5206 5207Appendix B Other useful software 5208******************************** 5209 5210In this appendix the installation of programs and libraries that are not 5211direct Gnuastro dependencies are discussed. However they can be useful 5212for working with Gnuastro. 5213 5214* Menu: 5215 5216* SAO DS9:: Viewing FITS images. 5217* PGPLOT:: Plotting directly in C 5218 5219 5220File: gnuastro.info, Node: SAO DS9, Next: PGPLOT, Prev: Other useful software, Up: Other useful software 5221 5222B.1 SAO DS9 5223=========== 5224 5225SAO DS9(1) is not a requirement of Gnuastro, it is a FITS image viewer. 5226So to check your inputs and outputs, it is one of the best options. 5227Like the other packages, it might already be available in your 5228distribution’s repositories. It is already pre-compiled in the download 5229section of its web page. Once you download it you can unpack and 5230install (move it to a system recognized directory) with the following 5231commands (‘x.x.x’ is the version number): 5232 5233 $ tar xf ds9.linux64.x.x.x.tar.gz 5234 $ sudo mv ds9 /usr/local/bin 5235 5236 Once you run it, there might be a complaint about the Xss library, 5237which you can find in your distribution package management system. You 5238might also get an ‘XPA’ related error. In this case, you have to add 5239the following line to your ‘~/.bashrc’ and ‘~/.profile’ file (you will 5240have to log out and back in again for the latter): 5241 5242 export XPA_METHOD=local 5243 5244* Menu: 5245 5246* Viewing multiextension FITS images:: Configure SAO DS9 for multiextension images. 5247 5248 ---------- Footnotes ---------- 5249 5250 (1) <http://ds9.si.edu/> 5251 5252 5253File: gnuastro.info, Node: Viewing multiextension FITS images, Prev: SAO DS9, Up: SAO DS9 5254 5255B.1.1 Viewing multiextension FITS images 5256---------------------------------------- 5257 5258The FITS definition allows for multiple extensions inside one FITS file, 5259each extension can have a completely independent dataset inside of it. 5260If you just double click on a multi-extension FITS file or run ‘$ds9 5261foo.fits’, SAO DS9 will only show you the first extension. If you have 5262a multi-extension file containing 2D images, one way to load and switch 5263between the each 2D extension is to take the following steps in the SAO 5264DS9 window: “File”→”Open Other”→”Open Multi Ext Cube” and then choose 5265the Multi extension FITS file in your computer’s file structure. 5266 5267 The method above is a little tedious to do every time you want view a 5268multi-extension FITS file. A different series of steps is also 5269necessary if you the extensions are 3D data cubes. Fortunately SAO DS9 5270also provides command-line options that you can use to specify a 5271particular behavior. One of those options is ‘-mecube’ which opens a 5272FITS image as a multi-extension data cube (treating each 2D extension as 5273a slice in a 3D cube). This allows you to flip through the extensions 5274easily while keeping all the settings similar. 5275 5276 Try running ‘$ds9 -mecube foo.fits’ to see the effect (for example on 5277the output of *note NoiseChisel::). If the file has multiple 5278extensions, a small window will also be opened along with the main ds9 5279window. This small window allows you to slide through the image 5280extensions of ‘foo.fits’. If ‘foo.fits’ only consists of one extension, 5281then SAO DS9 will open as usual. Just to avoid confusion, note that SAO 5282DS9 does not follow the GNU style of separating long and short options 5283as explained in *note Arguments and options::. In the GNU style, this 5284‘long’ (multi-character) option should have been called like ‘--mecube’, 5285but SAO DS9 follows its own conventions. 5286 5287 Recall the ‘-mecube’ opens each 2D input extension as a slice in 3D. 5288Therefore, when you want to inspect a multi-extension FITS file 5289containing a 3D dataset, the ‘-mecube’ option is no good any more (it 5290only opens the first slice of the 3D cube in each extension). In that 5291case, we have to use SAO DS9’s ‘-multiframe’ option to open each 5292extension as a separate frame. Since the input is a 3D dataset, we get 5293the same small window as the 2D case above for scrolling through the 3D 5294slices. We then have to also ask ds9 to match the frames and lock the 5295slices, so for example zooming in one, will also zoom the others. 5296 5297 We can use a script to automatize this process and make work much 5298easier (and save a lot of time) when opening any generic 2D or 3D 5299dataset. After taking the following steps, when you click on a FITS 5300file in your graphic user interface, ds9 will open in the respective 2D 5301or 3D mode when double clicking a FITS file on the graphic user 5302interface, and an executable will also be available to open ds9 5303similarly on the command-line. Note that the following solution assumes 5304you already have Gnuastro installed (and in particular the *note Fits:: 5305program). 5306 5307 Let’s assume that you want to store this script in ‘BINDIR’ (that is 5308in your ‘PATH’ environment variable, see *note Installation 5309directory::). [Tip: a good place would be ‘~/.local/bin’, just don’t 5310forget to make sure it is in your ‘PATH’]. Using your favorite text 5311editor, put the following script into a file called 5312‘BINDIR/ds9-multi-ext’. You can change the size of the opened ds9 5313window by changing the ‘1800x3000’ part of the script below. 5314 5315 #! /bin/bash 5316 5317 # To allow generic usage, if no input file is given (the `if' below is 5318 # true), then just open an empty ds9. 5319 if [ "x$1" == "x" ]; then 5320 ds9 5321 else 5322 # Make sure we are dealing with a FITS file. We are using shell 5323 # redirection here to make sure that nothing is printed in the 5324 # terminal (to standard output when we have a FITS file, or to 5325 # standard error when we don't). Since we've used redirection, 5326 # we'll also have to echo the return value of `astfits'. 5327 check=$(astfits $1 -h0 > /dev/null 2>&1; echo $?) 5328 5329 # If the file was a FITS file, then `check' will be 0. 5330 if [ "$check" == "0" ]; then 5331 5332 # Read the number of dimensions. 5333 n0=$(astfits $1 -h0 | awk '$1=="NAXIS"{print $3}') 5334 5335 # Find the number of dimensions. 5336 if [ "$n0" == "0" ]; then 5337 ndim=$(astfits $1 -h1 | awk '$1=="NAXIS"{print $3}') 5338 else 5339 ndim=$n0 5340 fi; 5341 5342 # Open DS9 based on the number of dimension. 5343 if [ "$ndim" = "2" ]; then 5344 # 2D multi-extension file: use the "Cube" window to 5345 # flip/slide through the extensions. 5346 ds9 -zscale -geometry 1800x3000 -mecube $1 \ 5347 -zoom to fit -wcs degrees 5348 else 5349 # 3D multi-extension file: The "Cube" window will slide 5350 # between the slices of a single extension. To flip 5351 # through the extensions (not the slices), press the top 5352 # row "frame" button and from the last four buttons of the 5353 # bottom row ("first", "previous", "next" and "last") can 5354 # be used to switch through the extensions (while keeping 5355 # the same slice). 5356 ds9 -zscale -geometry 1800x3000 -wcs degrees \ 5357 -multiframe $1 -match frame image \ 5358 -lock slice image -lock frame image -single \ 5359 -zoom to fit 5360 fi 5361 else 5362 if [ -f $1 ]; then 5363 echo "'$1' isn't a FITS file." 5364 else 5365 echo "'$1' doesn't exist." 5366 fi 5367 fi 5368 fi 5369 5370 As described above (also in the comments of the script), if you have 5371opened a multi-extension 2D dataset (image), the “Cube” window can be 5372used to slide/flip through each extension. But when the input is a 3D 5373data cube, the “Cube” window will slide/flip through the slices in each 5374extension (a separate 3D dataset). To flip through the extensions 5375(while keeping the slice fixed), click the “frame” button on the top row 5376of buttons, then use the last four buttons of the bottom row ("first", 5377"previous", "next" and "last") to change between the extensions. 5378 5379 To run this script, you have to activate its executable flag with 5380this command: 5381 5382 $ chmod +x BINDIR/ds9-multi-ext 5383 5384 If ‘BINDIR’ is within your system’s ‘PATH’ environment variable (see 5385*note Installation directory::), you can now open ds9 conditionally 5386using the script above with this command: 5387 5388 $ ds9-multi-ext foo.fits 5389 5390 For the graphic user interface, we’ll assume you are using GNOME (the 5391most popular graphic user interface for GNU/Linux systems), version 3. 5392For GNOME 2, see below. You can customize GNOME to open specific files 5393with ‘.desktop’ files. For each user, they are stored in 5394‘~/.local/share/applications/’. In case you don’t have the directory, 5395make it your self (with ‘mkdir’). Using your favorite text editor, you 5396can now create ‘~/.local/share/applications/saods9.desktop’ with the 5397following contents. Just don’t forget to correct ‘BINDIR’. If you 5398would also like to have ds9’s logo/icon in GNOME, download it, uncomment 5399the ‘Icon’ line, and write its address in the value. 5400 5401 [Desktop Entry] 5402 Type=Application 5403 Version=1.0 5404 Name=SAO DS9 5405 Comment=View FITS images 5406 Terminal=false 5407 Categories=Graphics;RasterGraphics;2DGraphics;3DGraphics 5408 #Icon=/PATH/TO/DS9/ICON/ds9.png 5409 Exec=BINDIR/ds9-multi-ext %f 5410 5411 The steps above will add SAO DS9 as one of your applications. To 5412make it default, take the following steps (just once is enough). Right 5413click on a FITS file and select Open with other application→View all 5414applications→SAO DS9. 5415 5416 In case you are using GNOME 2 you can take the following steps: right 5417click on a FITS file and choose Properties→Open With→Add button. A list 5418of applications will show up, ds9 might already be present in the list, 5419but don’t choose it because it will run with no options. Below the list 5420is an option “Use a custom command”. Click on it and write the 5421following command: ‘BINDIR/ds9-multi-ext’ in the box and click “Add”. 5422Then finally choose the command you just added as the default and click 5423the “Close” button. 5424 5425 5426File: gnuastro.info, Node: PGPLOT, Prev: SAO DS9, Up: Other useful software 5427 5428B.2 PGPLOT 5429========== 5430 5431PGPLOT is a package for making plots in C. It is not directly needed by 5432Gnuastro, but can be used by WCSLIB, see *note WCSLIB::. As explained 5433in *note WCSLIB::, you can install WCSLIB without it too. It is very 5434old (the most recent version was released early 2001!), but remains one 5435of the main packages for plotting directly in C. WCSLIB uses this 5436package to make plots if you want it to make plots. If you are 5437interested you can also use it for your own purposes. 5438 5439 If you want your plotting codes in between your C program, PGPLOT is 5440currently one of your best options. The recommended alternative to this 5441method is to get the raw data for the plots in text files and input them 5442into any of the various more modern and capable plotting tools 5443separately, for example the Matplotlib library in Python or PGFplots in 5444LaTeX. This will also significantly help code readability. Let’s get 5445back to PGPLOT for the sake of WCSLIB. Installing it is a little tricky 5446(mainly because it is so old!). 5447 5448 You can download the most recent version from the FTP link in its web 5449page(1). You can unpack it with the ‘tar xf’ command. Let’s assume the 5450directory you have unpacked it to is ‘PGPLOT’, most probably it is: 5451‘/home/username/Downloads/pgplot/’. open the ‘drivers.list’ file: 5452 $ gedit drivers.list 5453Remove the ‘!’ for the following lines and save the file in the end: 5454 PSDRIV 1 /PS 5455 PSDRIV 2 /VPS 5456 PSDRIV 3 /CPS 5457 PSDRIV 4 /VCPS 5458 XWDRIV 1 /XWINDOW 5459 XWDRIV 2 /XSERVE 5460Don’t choose GIF or VGIF, there is a problem in their codes. 5461 5462 Open the ‘PGPLOT/sys_linux/g77_gcc.conf’ file: 5463 $ gedit PGPLOT/sys_linux/g77_gcc.conf 5464change the line saying: ‘FCOMPL="g77"’ to ‘FCOMPL="gfortran"’, and save 5465it. This is a very important step during the compilation of the code if 5466you are in GNU/Linux. You now have to create a folder in ‘/usr/local’, 5467don’t forget to replace ‘PGPLOT’ with your unpacked address: 5468 $ su 5469 # mkdir /usr/local/pgplot 5470 # cd /usr/local/pgplot 5471 # cp PGPLOT/drivers.list ./ 5472 To make the Makefile, type the following command: 5473 # PGPLOT/makemake PGPLOT linux g77_gcc 5474It should finish by saying: ‘Determining object file dependencies’. You 5475have done the hard part! The rest is easy: run these three commands in 5476order: 5477 # make 5478 # make clean 5479 # make cpg 5480 5481 Finally you have to place the position of this directory you just 5482made into the ‘LD_LIBRARY_PATH’ environment variable and define the 5483environment variable ‘PGPLOT_DIR’. To do that, you have to edit your 5484‘.bashrc’ file: 5485 $ cd ~ 5486 $ gedit .bashrc 5487Copy these lines into the text editor and save it: 5488 PGPLOT_DIR="/usr/local/pgplot/"; export PGPLOT_DIR 5489 LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/pgplot/ 5490 export LD_LIBRARY_PATH 5491You need to log out and log back in again so these definitions take 5492effect. After you logged back in, you want to see the result of all 5493this labor, right? Tim Pearson has done that for you, create a 5494temporary folder in your home directory and copy all the demonstration 5495files in it: 5496 $ cd ~ 5497 $ mkdir temp 5498 $ cd temp 5499 $ cp /usr/local/pgplot/pgdemo* ./ 5500 $ ls 5501 You will see a lot of pgdemoXX files, where XX is a number. In order 5502to execute them type the following command and drink your coffee while 5503looking at all the beautiful plots! You are now ready to create your 5504own. 5505 $ ./pgdemoXX 5506 5507 ---------- Footnotes ---------- 5508 5509 (1) <http://www.astro.caltech.edu/~tjp/pgplot/> 5510 5511 5512File: gnuastro.info, Node: GNU Free Doc. License, Next: GNU General Public License, Prev: Other useful software, Up: Top 5513 5514Appendix C GNU Free Doc. License 5515******************************** 5516 5517 Version 1.3, 3 November 2008 5518 5519 Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. 5520 <https://fsf.org/> 5521 5522 Everyone is permitted to copy and distribute verbatim copies 5523 of this license document, but changing it is not allowed. 5524 5525 0. PREAMBLE 5526 5527 The purpose of this License is to make a manual, textbook, or other 5528 functional and useful document “free” in the sense of freedom: to 5529 assure everyone the effective freedom to copy and redistribute it, 5530 with or without modifying it, either commercially or 5531 noncommercially. Secondarily, this License preserves for the 5532 author and publisher a way to get credit for their work, while not 5533 being considered responsible for modifications made by others. 5534 5535 This License is a kind of “copyleft”, which means that derivative 5536 works of the document must themselves be free in the same sense. 5537 It complements the GNU General Public License, which is a copyleft 5538 license designed for free software. 5539 5540 We have designed this License in order to use it for manuals for 5541 free software, because free software needs free documentation: a 5542 free program should come with manuals providing the same freedoms 5543 that the software does. But this License is not limited to 5544 software manuals; it can be used for any textual work, regardless 5545 of subject matter or whether it is published as a printed book. We 5546 recommend this License principally for works whose purpose is 5547 instruction or reference. 5548 5549 1. APPLICABILITY AND DEFINITIONS 5550 5551 This License applies to any manual or other work, in any medium, 5552 that contains a notice placed by the copyright holder saying it can 5553 be distributed under the terms of this License. Such a notice 5554 grants a world-wide, royalty-free license, unlimited in duration, 5555 to use that work under the conditions stated herein. The 5556 “Document”, below, refers to any such manual or work. Any member 5557 of the public is a licensee, and is addressed as “you”. You accept 5558 the license if you copy, modify or distribute the work in a way 5559 requiring permission under copyright law. 5560 5561 A “Modified Version” of the Document means any work containing the 5562 Document or a portion of it, either copied verbatim, or with 5563 modifications and/or translated into another language. 5564 5565 A “Secondary Section” is a named appendix or a front-matter section 5566 of the Document that deals exclusively with the relationship of the 5567 publishers or authors of the Document to the Document’s overall 5568 subject (or to related matters) and contains nothing that could 5569 fall directly within that overall subject. (Thus, if the Document 5570 is in part a textbook of mathematics, a Secondary Section may not 5571 explain any mathematics.) The relationship could be a matter of 5572 historical connection with the subject or with related matters, or 5573 of legal, commercial, philosophical, ethical or political position 5574 regarding them. 5575 5576 The “Invariant Sections” are certain Secondary Sections whose 5577 titles are designated, as being those of Invariant Sections, in the 5578 notice that says that the Document is released under this License. 5579 If a section does not fit the above definition of Secondary then it 5580 is not allowed to be designated as Invariant. The Document may 5581 contain zero Invariant Sections. If the Document does not identify 5582 any Invariant Sections then there are none. 5583 5584 The “Cover Texts” are certain short passages of text that are 5585 listed, as Front-Cover Texts or Back-Cover Texts, in the notice 5586 that says that the Document is released under this License. A 5587 Front-Cover Text may be at most 5 words, and a Back-Cover Text may 5588 be at most 25 words. 5589 5590 A “Transparent” copy of the Document means a machine-readable copy, 5591 represented in a format whose specification is available to the 5592 general public, that is suitable for revising the document 5593 straightforwardly with generic text editors or (for images composed 5594 of pixels) generic paint programs or (for drawings) some widely 5595 available drawing editor, and that is suitable for input to text 5596 formatters or for automatic translation to a variety of formats 5597 suitable for input to text formatters. A copy made in an otherwise 5598 Transparent file format whose markup, or absence of markup, has 5599 been arranged to thwart or discourage subsequent modification by 5600 readers is not Transparent. An image format is not Transparent if 5601 used for any substantial amount of text. A copy that is not 5602 “Transparent” is called “Opaque”. 5603 5604 Examples of suitable formats for Transparent copies include plain 5605 ASCII without markup, Texinfo input format, LaTeX input format, 5606 SGML or XML using a publicly available DTD, and standard-conforming 5607 simple HTML, PostScript or PDF designed for human modification. 5608 Examples of transparent image formats include PNG, XCF and JPG. 5609 Opaque formats include proprietary formats that can be read and 5610 edited only by proprietary word processors, SGML or XML for which 5611 the DTD and/or processing tools are not generally available, and 5612 the machine-generated HTML, PostScript or PDF produced by some word 5613 processors for output purposes only. 5614 5615 The “Title Page” means, for a printed book, the title page itself, 5616 plus such following pages as are needed to hold, legibly, the 5617 material this License requires to appear in the title page. For 5618 works in formats which do not have any title page as such, “Title 5619 Page” means the text near the most prominent appearance of the 5620 work’s title, preceding the beginning of the body of the text. 5621 5622 The “publisher” means any person or entity that distributes copies 5623 of the Document to the public. 5624 5625 A section “Entitled XYZ” means a named subunit of the Document 5626 whose title either is precisely XYZ or contains XYZ in parentheses 5627 following text that translates XYZ in another language. (Here XYZ 5628 stands for a specific section name mentioned below, such as 5629 “Acknowledgements”, “Dedications”, “Endorsements”, or “History”.) 5630 To “Preserve the Title” of such a section when you modify the 5631 Document means that it remains a section “Entitled XYZ” according 5632 to this definition. 5633 5634 The Document may include Warranty Disclaimers next to the notice 5635 which states that this License applies to the Document. These 5636 Warranty Disclaimers are considered to be included by reference in 5637 this License, but only as regards disclaiming warranties: any other 5638 implication that these Warranty Disclaimers may have is void and 5639 has no effect on the meaning of this License. 5640 5641 2. VERBATIM COPYING 5642 5643 You may copy and distribute the Document in any medium, either 5644 commercially or noncommercially, provided that this License, the 5645 copyright notices, and the license notice saying this License 5646 applies to the Document are reproduced in all copies, and that you 5647 add no other conditions whatsoever to those of this License. You 5648 may not use technical measures to obstruct or control the reading 5649 or further copying of the copies you make or distribute. However, 5650 you may accept compensation in exchange for copies. If you 5651 distribute a large enough number of copies you must also follow the 5652 conditions in section 3. 5653 5654 You may also lend copies, under the same conditions stated above, 5655 and you may publicly display copies. 5656 5657 3. COPYING IN QUANTITY 5658 5659 If you publish printed copies (or copies in media that commonly 5660 have printed covers) of the Document, numbering more than 100, and 5661 the Document’s license notice requires Cover Texts, you must 5662 enclose the copies in covers that carry, clearly and legibly, all 5663 these Cover Texts: Front-Cover Texts on the front cover, and 5664 Back-Cover Texts on the back cover. Both covers must also clearly 5665 and legibly identify you as the publisher of these copies. The 5666 front cover must present the full title with all words of the title 5667 equally prominent and visible. You may add other material on the 5668 covers in addition. Copying with changes limited to the covers, as 5669 long as they preserve the title of the Document and satisfy these 5670 conditions, can be treated as verbatim copying in other respects. 5671 5672 If the required texts for either cover are too voluminous to fit 5673 legibly, you should put the first ones listed (as many as fit 5674 reasonably) on the actual cover, and continue the rest onto 5675 adjacent pages. 5676 5677 If you publish or distribute Opaque copies of the Document 5678 numbering more than 100, you must either include a machine-readable 5679 Transparent copy along with each Opaque copy, or state in or with 5680 each Opaque copy a computer-network location from which the general 5681 network-using public has access to download using public-standard 5682 network protocols a complete Transparent copy of the Document, free 5683 of added material. If you use the latter option, you must take 5684 reasonably prudent steps, when you begin distribution of Opaque 5685 copies in quantity, to ensure that this Transparent copy will 5686 remain thus accessible at the stated location until at least one 5687 year after the last time you distribute an Opaque copy (directly or 5688 through your agents or retailers) of that edition to the public. 5689 5690 It is requested, but not required, that you contact the authors of 5691 the Document well before redistributing any large number of copies, 5692 to give them a chance to provide you with an updated version of the 5693 Document. 5694 5695 4. MODIFICATIONS 5696 5697 You may copy and distribute a Modified Version of the Document 5698 under the conditions of sections 2 and 3 above, provided that you 5699 release the Modified Version under precisely this License, with the 5700 Modified Version filling the role of the Document, thus licensing 5701 distribution and modification of the Modified Version to whoever 5702 possesses a copy of it. In addition, you must do these things in 5703 the Modified Version: 5704 5705 A. Use in the Title Page (and on the covers, if any) a title 5706 distinct from that of the Document, and from those of previous 5707 versions (which should, if there were any, be listed in the 5708 History section of the Document). You may use the same title 5709 as a previous version if the original publisher of that 5710 version gives permission. 5711 5712 B. List on the Title Page, as authors, one or more persons or 5713 entities responsible for authorship of the modifications in 5714 the Modified Version, together with at least five of the 5715 principal authors of the Document (all of its principal 5716 authors, if it has fewer than five), unless they release you 5717 from this requirement. 5718 5719 C. State on the Title page the name of the publisher of the 5720 Modified Version, as the publisher. 5721 5722 D. Preserve all the copyright notices of the Document. 5723 5724 E. Add an appropriate copyright notice for your modifications 5725 adjacent to the other copyright notices. 5726 5727 F. Include, immediately after the copyright notices, a license 5728 notice giving the public permission to use the Modified 5729 Version under the terms of this License, in the form shown in 5730 the Addendum below. 5731 5732 G. Preserve in that license notice the full lists of Invariant 5733 Sections and required Cover Texts given in the Document’s 5734 license notice. 5735 5736 H. Include an unaltered copy of this License. 5737 5738 I. Preserve the section Entitled “History”, Preserve its Title, 5739 and add to it an item stating at least the title, year, new 5740 authors, and publisher of the Modified Version as given on the 5741 Title Page. If there is no section Entitled “History” in the 5742 Document, create one stating the title, year, authors, and 5743 publisher of the Document as given on its Title Page, then add 5744 an item describing the Modified Version as stated in the 5745 previous sentence. 5746 5747 J. Preserve the network location, if any, given in the Document 5748 for public access to a Transparent copy of the Document, and 5749 likewise the network locations given in the Document for 5750 previous versions it was based on. These may be placed in the 5751 “History” section. You may omit a network location for a work 5752 that was published at least four years before the Document 5753 itself, or if the original publisher of the version it refers 5754 to gives permission. 5755 5756 K. For any section Entitled “Acknowledgements” or “Dedications”, 5757 Preserve the Title of the section, and preserve in the section 5758 all the substance and tone of each of the contributor 5759 acknowledgements and/or dedications given therein. 5760 5761 L. Preserve all the Invariant Sections of the Document, unaltered 5762 in their text and in their titles. Section numbers or the 5763 equivalent are not considered part of the section titles. 5764 5765 M. Delete any section Entitled “Endorsements”. Such a section 5766 may not be included in the Modified Version. 5767 5768 N. Do not retitle any existing section to be Entitled 5769 “Endorsements” or to conflict in title with any Invariant 5770 Section. 5771 5772 O. Preserve any Warranty Disclaimers. 5773 5774 If the Modified Version includes new front-matter sections or 5775 appendices that qualify as Secondary Sections and contain no 5776 material copied from the Document, you may at your option designate 5777 some or all of these sections as invariant. To do this, add their 5778 titles to the list of Invariant Sections in the Modified Version’s 5779 license notice. These titles must be distinct from any other 5780 section titles. 5781 5782 You may add a section Entitled “Endorsements”, provided it contains 5783 nothing but endorsements of your Modified Version by various 5784 parties—for example, statements of peer review or that the text has 5785 been approved by an organization as the authoritative definition of 5786 a standard. 5787 5788 You may add a passage of up to five words as a Front-Cover Text, 5789 and a passage of up to 25 words as a Back-Cover Text, to the end of 5790 the list of Cover Texts in the Modified Version. Only one passage 5791 of Front-Cover Text and one of Back-Cover Text may be added by (or 5792 through arrangements made by) any one entity. If the Document 5793 already includes a cover text for the same cover, previously added 5794 by you or by arrangement made by the same entity you are acting on 5795 behalf of, you may not add another; but you may replace the old 5796 one, on explicit permission from the previous publisher that added 5797 the old one. 5798 5799 The author(s) and publisher(s) of the Document do not by this 5800 License give permission to use their names for publicity for or to 5801 assert or imply endorsement of any Modified Version. 5802 5803 5. COMBINING DOCUMENTS 5804 5805 You may combine the Document with other documents released under 5806 this License, under the terms defined in section 4 above for 5807 modified versions, provided that you include in the combination all 5808 of the Invariant Sections of all of the original documents, 5809 unmodified, and list them all as Invariant Sections of your 5810 combined work in its license notice, and that you preserve all 5811 their Warranty Disclaimers. 5812 5813 The combined work need only contain one copy of this License, and 5814 multiple identical Invariant Sections may be replaced with a single 5815 copy. If there are multiple Invariant Sections with the same name 5816 but different contents, make the title of each such section unique 5817 by adding at the end of it, in parentheses, the name of the 5818 original author or publisher of that section if known, or else a 5819 unique number. Make the same adjustment to the section titles in 5820 the list of Invariant Sections in the license notice of the 5821 combined work. 5822 5823 In the combination, you must combine any sections Entitled 5824 “History” in the various original documents, forming one section 5825 Entitled “History”; likewise combine any sections Entitled 5826 “Acknowledgements”, and any sections Entitled “Dedications”. You 5827 must delete all sections Entitled “Endorsements.” 5828 5829 6. COLLECTIONS OF DOCUMENTS 5830 5831 You may make a collection consisting of the Document and other 5832 documents released under this License, and replace the individual 5833 copies of this License in the various documents with a single copy 5834 that is included in the collection, provided that you follow the 5835 rules of this License for verbatim copying of each of the documents 5836 in all other respects. 5837 5838 You may extract a single document from such a collection, and 5839 distribute it individually under this License, provided you insert 5840 a copy of this License into the extracted document, and follow this 5841 License in all other respects regarding verbatim copying of that 5842 document. 5843 5844 7. AGGREGATION WITH INDEPENDENT WORKS 5845 5846 A compilation of the Document or its derivatives with other 5847 separate and independent documents or works, in or on a volume of a 5848 storage or distribution medium, is called an “aggregate” if the 5849 copyright resulting from the compilation is not used to limit the 5850 legal rights of the compilation’s users beyond what the individual 5851 works permit. When the Document is included in an aggregate, this 5852 License does not apply to the other works in the aggregate which 5853 are not themselves derivative works of the Document. 5854 5855 If the Cover Text requirement of section 3 is applicable to these 5856 copies of the Document, then if the Document is less than one half 5857 of the entire aggregate, the Document’s Cover Texts may be placed 5858 on covers that bracket the Document within the aggregate, or the 5859 electronic equivalent of covers if the Document is in electronic 5860 form. Otherwise they must appear on printed covers that bracket 5861 the whole aggregate. 5862 5863 8. TRANSLATION 5864 5865 Translation is considered a kind of modification, so you may 5866 distribute translations of the Document under the terms of section 5867 4. Replacing Invariant Sections with translations requires special 5868 permission from their copyright holders, but you may include 5869 translations of some or all Invariant Sections in addition to the 5870 original versions of these Invariant Sections. You may include a 5871 translation of this License, and all the license notices in the 5872 Document, and any Warranty Disclaimers, provided that you also 5873 include the original English version of this License and the 5874 original versions of those notices and disclaimers. In case of a 5875 disagreement between the translation and the original version of 5876 this License or a notice or disclaimer, the original version will 5877 prevail. 5878 5879 If a section in the Document is Entitled “Acknowledgements”, 5880 “Dedications”, or “History”, the requirement (section 4) to 5881 Preserve its Title (section 1) will typically require changing the 5882 actual title. 5883 5884 9. TERMINATION 5885 5886 You may not copy, modify, sublicense, or distribute the Document 5887 except as expressly provided under this License. Any attempt 5888 otherwise to copy, modify, sublicense, or distribute it is void, 5889 and will automatically terminate your rights under this License. 5890 5891 However, if you cease all violation of this License, then your 5892 license from a particular copyright holder is reinstated (a) 5893 provisionally, unless and until the copyright holder explicitly and 5894 finally terminates your license, and (b) permanently, if the 5895 copyright holder fails to notify you of the violation by some 5896 reasonable means prior to 60 days after the cessation. 5897 5898 Moreover, your license from a particular copyright holder is 5899 reinstated permanently if the copyright holder notifies you of the 5900 violation by some reasonable means, this is the first time you have 5901 received notice of violation of this License (for any work) from 5902 that copyright holder, and you cure the violation prior to 30 days 5903 after your receipt of the notice. 5904 5905 Termination of your rights under this section does not terminate 5906 the licenses of parties who have received copies or rights from you 5907 under this License. If your rights have been terminated and not 5908 permanently reinstated, receipt of a copy of some or all of the 5909 same material does not give you any rights to use it. 5910 5911 10. FUTURE REVISIONS OF THIS LICENSE 5912 5913 The Free Software Foundation may publish new, revised versions of 5914 the GNU Free Documentation License from time to time. Such new 5915 versions will be similar in spirit to the present version, but may 5916 differ in detail to address new problems or concerns. See 5917 <https://www.gnu.org/licenses/>. 5918 5919 Each version of the License is given a distinguishing version 5920 number. If the Document specifies that a particular numbered 5921 version of this License “or any later version” applies to it, you 5922 have the option of following the terms and conditions either of 5923 that specified version or of any later version that has been 5924 published (not as a draft) by the Free Software Foundation. If the 5925 Document does not specify a version number of this License, you may 5926 choose any version ever published (not as a draft) by the Free 5927 Software Foundation. If the Document specifies that a proxy can 5928 decide which future versions of this License can be used, that 5929 proxy’s public statement of acceptance of a version permanently 5930 authorizes you to choose that version for the Document. 5931 5932 11. RELICENSING 5933 5934 “Massive Multiauthor Collaboration Site” (or “MMC Site”) means any 5935 World Wide Web server that publishes copyrightable works and also 5936 provides prominent facilities for anybody to edit those works. A 5937 public wiki that anybody can edit is an example of such a server. 5938 A “Massive Multiauthor Collaboration” (or “MMC”) contained in the 5939 site means any set of copyrightable works thus published on the MMC 5940 site. 5941 5942 “CC-BY-SA” means the Creative Commons Attribution-Share Alike 3.0 5943 license published by Creative Commons Corporation, a not-for-profit 5944 corporation with a principal place of business in San Francisco, 5945 California, as well as future copyleft versions of that license 5946 published by that same organization. 5947 5948 “Incorporate” means to publish or republish a Document, in whole or 5949 in part, as part of another Document. 5950 5951 An MMC is “eligible for relicensing” if it is licensed under this 5952 License, and if all works that were first published under this 5953 License somewhere other than this MMC, and subsequently 5954 incorporated in whole or in part into the MMC, (1) had no cover 5955 texts or invariant sections, and (2) were thus incorporated prior 5956 to November 1, 2008. 5957 5958 The operator of an MMC Site may republish an MMC contained in the 5959 site under CC-BY-SA on the same site at any time before August 1, 5960 2009, provided the MMC is eligible for relicensing. 5961 5962ADDENDUM: How to use this License for your documents 5963==================================================== 5964 5965To use this License in a document you have written, include a copy of 5966the License in the document and put the following copyright and license 5967notices just after the title page: 5968 5969 Copyright (C) YEAR YOUR NAME. 5970 Permission is granted to copy, distribute and/or modify this document 5971 under the terms of the GNU Free Documentation License, Version 1.3 5972 or any later version published by the Free Software Foundation; 5973 with no Invariant Sections, no Front-Cover Texts, and no Back-Cover 5974 Texts. A copy of the license is included in the section entitled ``GNU 5975 Free Documentation License''. 5976 5977 If you have Invariant Sections, Front-Cover Texts and Back-Cover 5978Texts, replace the “with...Texts.” line with this: 5979 5980 with the Invariant Sections being LIST THEIR TITLES, with 5981 the Front-Cover Texts being LIST, and with the Back-Cover Texts 5982 being LIST. 5983 5984 If you have Invariant Sections without Cover Texts, or some other 5985combination of the three, merge those two alternatives to suit the 5986situation. 5987 5988 If your document contains nontrivial examples of program code, we 5989recommend releasing these examples in parallel under your choice of free 5990software license, such as the GNU General Public License, to permit 5991their use in free software. 5992 5993 5994File: gnuastro.info, Node: GNU General Public License, Next: Index, Prev: GNU Free Doc. License, Up: Top 5995 5996Appendix D GNU Gen. Pub. License v3 5997*********************************** 5998 5999 Version 3, 29 June 2007 6000 6001 Copyright © 2007 Free Software Foundation, Inc. <https://fsf.org/> 6002 6003 Everyone is permitted to copy and distribute verbatim copies of this 6004 license document, but changing it is not allowed. 6005 6006Preamble 6007======== 6008 6009The GNU General Public License is a free, copyleft license for software 6010and other kinds of works. 6011 6012 The licenses for most software and other practical works are designed 6013to take away your freedom to share and change the works. By contrast, 6014the GNU General Public License is intended to guarantee your freedom to 6015share and change all versions of a program—to make sure it remains free 6016software for all its users. We, the Free Software Foundation, use the 6017GNU General Public License for most of our software; it applies also to 6018any other work released this way by its authors. You can apply it to 6019your programs, too. 6020 6021 When we speak of free software, we are referring to freedom, not 6022price. Our General Public Licenses are designed to make sure that you 6023have the freedom to distribute copies of free software (and charge for 6024them if you wish), that you receive source code or can get it if you 6025want it, that you can change the software or use pieces of it in new 6026free programs, and that you know you can do these things. 6027 6028 To protect your rights, we need to prevent others from denying you 6029these rights or asking you to surrender the rights. Therefore, you have 6030certain responsibilities if you distribute copies of the software, or if 6031you modify it: responsibilities to respect the freedom of others. 6032 6033 For example, if you distribute copies of such a program, whether 6034gratis or for a fee, you must pass on to the recipients the same 6035freedoms that you received. You must make sure that they, too, receive 6036or can get the source code. And you must show them these terms so they 6037know their rights. 6038 6039 Developers that use the GNU GPL protect your rights with two steps: 6040(1) assert copyright on the software, and (2) offer you this License 6041giving you legal permission to copy, distribute and/or modify it. 6042 6043 For the developers’ and authors’ protection, the GPL clearly explains 6044that there is no warranty for this free software. For both users’ and 6045authors’ sake, the GPL requires that modified versions be marked as 6046changed, so that their problems will not be attributed erroneously to 6047authors of previous versions. 6048 6049 Some devices are designed to deny users access to install or run 6050modified versions of the software inside them, although the manufacturer 6051can do so. This is fundamentally incompatible with the aim of 6052protecting users’ freedom to change the software. The systematic 6053pattern of such abuse occurs in the area of products for individuals to 6054use, which is precisely where it is most unacceptable. Therefore, we 6055have designed this version of the GPL to prohibit the practice for those 6056products. If such problems arise substantially in other domains, we 6057stand ready to extend this provision to those domains in future versions 6058of the GPL, as needed to protect the freedom of users. 6059 6060 Finally, every program is threatened constantly by software patents. 6061States should not allow patents to restrict development and use of 6062software on general-purpose computers, but in those that do, we wish to 6063avoid the special danger that patents applied to a free program could 6064make it effectively proprietary. To prevent this, the GPL assures that 6065patents cannot be used to render the program non-free. 6066 6067 The precise terms and conditions for copying, distribution and 6068modification follow. 6069 6070TERMS AND CONDITIONS 6071==================== 6072 6073 0. Definitions. 6074 6075 “This License” refers to version 3 of the GNU General Public 6076 License. 6077 6078 “Copyright” also means copyright-like laws that apply to other 6079 kinds of works, such as semiconductor masks. 6080 6081 “The Program” refers to any copyrightable work licensed under this 6082 License. Each licensee is addressed as “you”. “Licensees” and 6083 “recipients” may be individuals or organizations. 6084 6085 To “modify” a work means to copy from or adapt all or part of the 6086 work in a fashion requiring copyright permission, other than the 6087 making of an exact copy. The resulting work is called a “modified 6088 version” of the earlier work or a work “based on” the earlier work. 6089 6090 A “covered work” means either the unmodified Program or a work 6091 based on the Program. 6092 6093 To “propagate” a work means to do anything with it that, without 6094 permission, would make you directly or secondarily liable for 6095 infringement under applicable copyright law, except executing it on 6096 a computer or modifying a private copy. Propagation includes 6097 copying, distribution (with or without modification), making 6098 available to the public, and in some countries other activities as 6099 well. 6100 6101 To “convey” a work means any kind of propagation that enables other 6102 parties to make or receive copies. Mere interaction with a user 6103 through a computer network, with no transfer of a copy, is not 6104 conveying. 6105 6106 An interactive user interface displays “Appropriate Legal Notices” 6107 to the extent that it includes a convenient and prominently visible 6108 feature that (1) displays an appropriate copyright notice, and (2) 6109 tells the user that there is no warranty for the work (except to 6110 the extent that warranties are provided), that licensees may convey 6111 the work under this License, and how to view a copy of this 6112 License. If the interface presents a list of user commands or 6113 options, such as a menu, a prominent item in the list meets this 6114 criterion. 6115 6116 1. Source Code. 6117 6118 The “source code” for a work means the preferred form of the work 6119 for making modifications to it. “Object code” means any non-source 6120 form of a work. 6121 6122 A “Standard Interface” means an interface that either is an 6123 official standard defined by a recognized standards body, or, in 6124 the case of interfaces specified for a particular programming 6125 language, one that is widely used among developers working in that 6126 language. 6127 6128 The “System Libraries” of an executable work include anything, 6129 other than the work as a whole, that (a) is included in the normal 6130 form of packaging a Major Component, but which is not part of that 6131 Major Component, and (b) serves only to enable use of the work with 6132 that Major Component, or to implement a Standard Interface for 6133 which an implementation is available to the public in source code 6134 form. A “Major Component”, in this context, means a major 6135 essential component (kernel, window system, and so on) of the 6136 specific operating system (if any) on which the executable work 6137 runs, or a compiler used to produce the work, or an object code 6138 interpreter used to run it. 6139 6140 The “Corresponding Source” for a work in object code form means all 6141 the source code needed to generate, install, and (for an executable 6142 work) run the object code and to modify the work, including scripts 6143 to control those activities. However, it does not include the 6144 work’s System Libraries, or general-purpose tools or generally 6145 available free programs which are used unmodified in performing 6146 those activities but which are not part of the work. For example, 6147 Corresponding Source includes interface definition files associated 6148 with source files for the work, and the source code for shared 6149 libraries and dynamically linked subprograms that the work is 6150 specifically designed to require, such as by intimate data 6151 communication or control flow between those subprograms and other 6152 parts of the work. 6153 6154 The Corresponding Source need not include anything that users can 6155 regenerate automatically from other parts of the Corresponding 6156 Source. 6157 6158 The Corresponding Source for a work in source code form is that 6159 same work. 6160 6161 2. Basic Permissions. 6162 6163 All rights granted under this License are granted for the term of 6164 copyright on the Program, and are irrevocable provided the stated 6165 conditions are met. This License explicitly affirms your unlimited 6166 permission to run the unmodified Program. The output from running 6167 a covered work is covered by this License only if the output, given 6168 its content, constitutes a covered work. This License acknowledges 6169 your rights of fair use or other equivalent, as provided by 6170 copyright law. 6171 6172 You may make, run and propagate covered works that you do not 6173 convey, without conditions so long as your license otherwise 6174 remains in force. You may convey covered works to others for the 6175 sole purpose of having them make modifications exclusively for you, 6176 or provide you with facilities for running those works, provided 6177 that you comply with the terms of this License in conveying all 6178 material for which you do not control copyright. Those thus making 6179 or running the covered works for you must do so exclusively on your 6180 behalf, under your direction and control, on terms that prohibit 6181 them from making any copies of your copyrighted material outside 6182 their relationship with you. 6183 6184 Conveying under any other circumstances is permitted solely under 6185 the conditions stated below. Sublicensing is not allowed; section 6186 10 makes it unnecessary. 6187 6188 3. Protecting Users’ Legal Rights From Anti-Circumvention Law. 6189 6190 No covered work shall be deemed part of an effective technological 6191 measure under any applicable law fulfilling obligations under 6192 article 11 of the WIPO copyright treaty adopted on 20 December 6193 1996, or similar laws prohibiting or restricting circumvention of 6194 such measures. 6195 6196 When you convey a covered work, you waive any legal power to forbid 6197 circumvention of technological measures to the extent such 6198 circumvention is effected by exercising rights under this License 6199 with respect to the covered work, and you disclaim any intention to 6200 limit operation or modification of the work as a means of 6201 enforcing, against the work’s users, your or third parties’ legal 6202 rights to forbid circumvention of technological measures. 6203 6204 4. Conveying Verbatim Copies. 6205 6206 You may convey verbatim copies of the Program’s source code as you 6207 receive it, in any medium, provided that you conspicuously and 6208 appropriately publish on each copy an appropriate copyright notice; 6209 keep intact all notices stating that this License and any 6210 non-permissive terms added in accord with section 7 apply to the 6211 code; keep intact all notices of the absence of any warranty; and 6212 give all recipients a copy of this License along with the Program. 6213 6214 You may charge any price or no price for each copy that you convey, 6215 and you may offer support or warranty protection for a fee. 6216 6217 5. Conveying Modified Source Versions. 6218 6219 You may convey a work based on the Program, or the modifications to 6220 produce it from the Program, in the form of source code under the 6221 terms of section 4, provided that you also meet all of these 6222 conditions: 6223 6224 a. The work must carry prominent notices stating that you 6225 modified it, and giving a relevant date. 6226 6227 b. The work must carry prominent notices stating that it is 6228 released under this License and any conditions added under 6229 section 7. This requirement modifies the requirement in 6230 section 4 to “keep intact all notices”. 6231 6232 c. You must license the entire work, as a whole, under this 6233 License to anyone who comes into possession of a copy. This 6234 License will therefore apply, along with any applicable 6235 section 7 additional terms, to the whole of the work, and all 6236 its parts, regardless of how they are packaged. This License 6237 gives no permission to license the work in any other way, but 6238 it does not invalidate such permission if you have separately 6239 received it. 6240 6241 d. If the work has interactive user interfaces, each must display 6242 Appropriate Legal Notices; however, if the Program has 6243 interactive interfaces that do not display Appropriate Legal 6244 Notices, your work need not make them do so. 6245 6246 A compilation of a covered work with other separate and independent 6247 works, which are not by their nature extensions of the covered 6248 work, and which are not combined with it such as to form a larger 6249 program, in or on a volume of a storage or distribution medium, is 6250 called an “aggregate” if the compilation and its resulting 6251 copyright are not used to limit the access or legal rights of the 6252 compilation’s users beyond what the individual works permit. 6253 Inclusion of a covered work in an aggregate does not cause this 6254 License to apply to the other parts of the aggregate. 6255 6256 6. Conveying Non-Source Forms. 6257 6258 You may convey a covered work in object code form under the terms 6259 of sections 4 and 5, provided that you also convey the 6260 machine-readable Corresponding Source under the terms of this 6261 License, in one of these ways: 6262 6263 a. Convey the object code in, or embodied in, a physical product 6264 (including a physical distribution medium), accompanied by the 6265 Corresponding Source fixed on a durable physical medium 6266 customarily used for software interchange. 6267 6268 b. Convey the object code in, or embodied in, a physical product 6269 (including a physical distribution medium), accompanied by a 6270 written offer, valid for at least three years and valid for as 6271 long as you offer spare parts or customer support for that 6272 product model, to give anyone who possesses the object code 6273 either (1) a copy of the Corresponding Source for all the 6274 software in the product that is covered by this License, on a 6275 durable physical medium customarily used for software 6276 interchange, for a price no more than your reasonable cost of 6277 physically performing this conveying of source, or (2) access 6278 to copy the Corresponding Source from a network server at no 6279 charge. 6280 6281 c. Convey individual copies of the object code with a copy of the 6282 written offer to provide the Corresponding Source. This 6283 alternative is allowed only occasionally and noncommercially, 6284 and only if you received the object code with such an offer, 6285 in accord with subsection 6b. 6286 6287 d. Convey the object code by offering access from a designated 6288 place (gratis or for a charge), and offer equivalent access to 6289 the Corresponding Source in the same way through the same 6290 place at no further charge. You need not require recipients 6291 to copy the Corresponding Source along with the object code. 6292 If the place to copy the object code is a network server, the 6293 Corresponding Source may be on a different server (operated by 6294 you or a third party) that supports equivalent copying 6295 facilities, provided you maintain clear directions next to the 6296 object code saying where to find the Corresponding Source. 6297 Regardless of what server hosts the Corresponding Source, you 6298 remain obligated to ensure that it is available for as long as 6299 needed to satisfy these requirements. 6300 6301 e. Convey the object code using peer-to-peer transmission, 6302 provided you inform other peers where the object code and 6303 Corresponding Source of the work are being offered to the 6304 general public at no charge under subsection 6d. 6305 6306 A separable portion of the object code, whose source code is 6307 excluded from the Corresponding Source as a System Library, need 6308 not be included in conveying the object code work. 6309 6310 A “User Product” is either (1) a “consumer product”, which means 6311 any tangible personal property which is normally used for personal, 6312 family, or household purposes, or (2) anything designed or sold for 6313 incorporation into a dwelling. In determining whether a product is 6314 a consumer product, doubtful cases shall be resolved in favor of 6315 coverage. For a particular product received by a particular user, 6316 “normally used” refers to a typical or common use of that class of 6317 product, regardless of the status of the particular user or of the 6318 way in which the particular user actually uses, or expects or is 6319 expected to use, the product. A product is a consumer product 6320 regardless of whether the product has substantial commercial, 6321 industrial or non-consumer uses, unless such uses represent the 6322 only significant mode of use of the product. 6323 6324 “Installation Information” for a User Product means any methods, 6325 procedures, authorization keys, or other information required to 6326 install and execute modified versions of a covered work in that 6327 User Product from a modified version of its Corresponding Source. 6328 The information must suffice to ensure that the continued 6329 functioning of the modified object code is in no case prevented or 6330 interfered with solely because modification has been made. 6331 6332 If you convey an object code work under this section in, or with, 6333 or specifically for use in, a User Product, and the conveying 6334 occurs as part of a transaction in which the right of possession 6335 and use of the User Product is transferred to the recipient in 6336 perpetuity or for a fixed term (regardless of how the transaction 6337 is characterized), the Corresponding Source conveyed under this 6338 section must be accompanied by the Installation Information. But 6339 this requirement does not apply if neither you nor any third party 6340 retains the ability to install modified object code on the User 6341 Product (for example, the work has been installed in ROM). 6342 6343 The requirement to provide Installation Information does not 6344 include a requirement to continue to provide support service, 6345 warranty, or updates for a work that has been modified or installed 6346 by the recipient, or for the User Product in which it has been 6347 modified or installed. Access to a network may be denied when the 6348 modification itself materially and adversely affects the operation 6349 of the network or violates the rules and protocols for 6350 communication across the network. 6351 6352 Corresponding Source conveyed, and Installation Information 6353 provided, in accord with this section must be in a format that is 6354 publicly documented (and with an implementation available to the 6355 public in source code form), and must require no special password 6356 or key for unpacking, reading or copying. 6357 6358 7. Additional Terms. 6359 6360 “Additional permissions” are terms that supplement the terms of 6361 this License by making exceptions from one or more of its 6362 conditions. Additional permissions that are applicable to the 6363 entire Program shall be treated as though they were included in 6364 this License, to the extent that they are valid under applicable 6365 law. If additional permissions apply only to part of the Program, 6366 that part may be used separately under those permissions, but the 6367 entire Program remains governed by this License without regard to 6368 the additional permissions. 6369 6370 When you convey a copy of a covered work, you may at your option 6371 remove any additional permissions from that copy, or from any part 6372 of it. (Additional permissions may be written to require their own 6373 removal in certain cases when you modify the work.) You may place 6374 additional permissions on material, added by you to a covered work, 6375 for which you have or can give appropriate copyright permission. 6376 6377 Notwithstanding any other provision of this License, for material 6378 you add to a covered work, you may (if authorized by the copyright 6379 holders of that material) supplement the terms of this License with 6380 terms: 6381 6382 a. Disclaiming warranty or limiting liability differently from 6383 the terms of sections 15 and 16 of this License; or 6384 6385 b. Requiring preservation of specified reasonable legal notices 6386 or author attributions in that material or in the Appropriate 6387 Legal Notices displayed by works containing it; or 6388 6389 c. Prohibiting misrepresentation of the origin of that material, 6390 or requiring that modified versions of such material be marked 6391 in reasonable ways as different from the original version; or 6392 6393 d. Limiting the use for publicity purposes of names of licensors 6394 or authors of the material; or 6395 6396 e. Declining to grant rights under trademark law for use of some 6397 trade names, trademarks, or service marks; or 6398 6399 f. Requiring indemnification of licensors and authors of that 6400 material by anyone who conveys the material (or modified 6401 versions of it) with contractual assumptions of liability to 6402 the recipient, for any liability that these contractual 6403 assumptions directly impose on those licensors and authors. 6404 6405 All other non-permissive additional terms are considered “further 6406 restrictions” within the meaning of section 10. If the Program as 6407 you received it, or any part of it, contains a notice stating that 6408 it is governed by this License along with a term that is a further 6409 restriction, you may remove that term. If a license document 6410 contains a further restriction but permits relicensing or conveying 6411 under this License, you may add to a covered work material governed 6412 by the terms of that license document, provided that the further 6413 restriction does not survive such relicensing or conveying. 6414 6415 If you add terms to a covered work in accord with this section, you 6416 must place, in the relevant source files, a statement of the 6417 additional terms that apply to those files, or a notice indicating 6418 where to find the applicable terms. 6419 6420 Additional terms, permissive or non-permissive, may be stated in 6421 the form of a separately written license, or stated as exceptions; 6422 the above requirements apply either way. 6423 6424 8. Termination. 6425 6426 You may not propagate or modify a covered work except as expressly 6427 provided under this License. Any attempt otherwise to propagate or 6428 modify it is void, and will automatically terminate your rights 6429 under this License (including any patent licenses granted under the 6430 third paragraph of section 11). 6431 6432 However, if you cease all violation of this License, then your 6433 license from a particular copyright holder is reinstated (a) 6434 provisionally, unless and until the copyright holder explicitly and 6435 finally terminates your license, and (b) permanently, if the 6436 copyright holder fails to notify you of the violation by some 6437 reasonable means prior to 60 days after the cessation. 6438 6439 Moreover, your license from a particular copyright holder is 6440 reinstated permanently if the copyright holder notifies you of the 6441 violation by some reasonable means, this is the first time you have 6442 received notice of violation of this License (for any work) from 6443 that copyright holder, and you cure the violation prior to 30 days 6444 after your receipt of the notice. 6445 6446 Termination of your rights under this section does not terminate 6447 the licenses of parties who have received copies or rights from you 6448 under this License. If your rights have been terminated and not 6449 permanently reinstated, you do not qualify to receive new licenses 6450 for the same material under section 10. 6451 6452 9. Acceptance Not Required for Having Copies. 6453 6454 You are not required to accept this License in order to receive or 6455 run a copy of the Program. Ancillary propagation of a covered work 6456 occurring solely as a consequence of using peer-to-peer 6457 transmission to receive a copy likewise does not require 6458 acceptance. However, nothing other than this License grants you 6459 permission to propagate or modify any covered work. These actions 6460 infringe copyright if you do not accept this License. Therefore, 6461 by modifying or propagating a covered work, you indicate your 6462 acceptance of this License to do so. 6463 6464 10. Automatic Licensing of Downstream Recipients. 6465 6466 Each time you convey a covered work, the recipient automatically 6467 receives a license from the original licensors, to run, modify and 6468 propagate that work, subject to this License. You are not 6469 responsible for enforcing compliance by third parties with this 6470 License. 6471 6472 An “entity transaction” is a transaction transferring control of an 6473 organization, or substantially all assets of one, or subdividing an 6474 organization, or merging organizations. If propagation of a 6475 covered work results from an entity transaction, each party to that 6476 transaction who receives a copy of the work also receives whatever 6477 licenses to the work the party’s predecessor in interest had or 6478 could give under the previous paragraph, plus a right to possession 6479 of the Corresponding Source of the work from the predecessor in 6480 interest, if the predecessor has it or can get it with reasonable 6481 efforts. 6482 6483 You may not impose any further restrictions on the exercise of the 6484 rights granted or affirmed under this License. For example, you 6485 may not impose a license fee, royalty, or other charge for exercise 6486 of rights granted under this License, and you may not initiate 6487 litigation (including a cross-claim or counterclaim in a lawsuit) 6488 alleging that any patent claim is infringed by making, using, 6489 selling, offering for sale, or importing the Program or any portion 6490 of it. 6491 6492 11. Patents. 6493 6494 A “contributor” is a copyright holder who authorizes use under this 6495 License of the Program or a work on which the Program is based. 6496 The work thus licensed is called the contributor’s “contributor 6497 version”. 6498 6499 A contributor’s “essential patent claims” are all patent claims 6500 owned or controlled by the contributor, whether already acquired or 6501 hereafter acquired, that would be infringed by some manner, 6502 permitted by this License, of making, using, or selling its 6503 contributor version, but do not include claims that would be 6504 infringed only as a consequence of further modification of the 6505 contributor version. For purposes of this definition, “control” 6506 includes the right to grant patent sublicenses in a manner 6507 consistent with the requirements of this License. 6508 6509 Each contributor grants you a non-exclusive, worldwide, 6510 royalty-free patent license under the contributor’s essential 6511 patent claims, to make, use, sell, offer for sale, import and 6512 otherwise run, modify and propagate the contents of its contributor 6513 version. 6514 6515 In the following three paragraphs, a “patent license” is any 6516 express agreement or commitment, however denominated, not to 6517 enforce a patent (such as an express permission to practice a 6518 patent or covenant not to sue for patent infringement). To “grant” 6519 such a patent license to a party means to make such an agreement or 6520 commitment not to enforce a patent against the party. 6521 6522 If you convey a covered work, knowingly relying on a patent 6523 license, and the Corresponding Source of the work is not available 6524 for anyone to copy, free of charge and under the terms of this 6525 License, through a publicly available network server or other 6526 readily accessible means, then you must either (1) cause the 6527 Corresponding Source to be so available, or (2) arrange to deprive 6528 yourself of the benefit of the patent license for this particular 6529 work, or (3) arrange, in a manner consistent with the requirements 6530 of this License, to extend the patent license to downstream 6531 recipients. “Knowingly relying” means you have actual knowledge 6532 that, but for the patent license, your conveying the covered work 6533 in a country, or your recipient’s use of the covered work in a 6534 country, would infringe one or more identifiable patents in that 6535 country that you have reason to believe are valid. 6536 6537 If, pursuant to or in connection with a single transaction or 6538 arrangement, you convey, or propagate by procuring conveyance of, a 6539 covered work, and grant a patent license to some of the parties 6540 receiving the covered work authorizing them to use, propagate, 6541 modify or convey a specific copy of the covered work, then the 6542 patent license you grant is automatically extended to all 6543 recipients of the covered work and works based on it. 6544 6545 A patent license is “discriminatory” if it does not include within 6546 the scope of its coverage, prohibits the exercise of, or is 6547 conditioned on the non-exercise of one or more of the rights that 6548 are specifically granted under this License. You may not convey a 6549 covered work if you are a party to an arrangement with a third 6550 party that is in the business of distributing software, under which 6551 you make payment to the third party based on the extent of your 6552 activity of conveying the work, and under which the third party 6553 grants, to any of the parties who would receive the covered work 6554 from you, a discriminatory patent license (a) in connection with 6555 copies of the covered work conveyed by you (or copies made from 6556 those copies), or (b) primarily for and in connection with specific 6557 products or compilations that contain the covered work, unless you 6558 entered into that arrangement, or that patent license was granted, 6559 prior to 28 March 2007. 6560 6561 Nothing in this License shall be construed as excluding or limiting 6562 any implied license or other defenses to infringement that may 6563 otherwise be available to you under applicable patent law. 6564 6565 12. No Surrender of Others’ Freedom. 6566 6567 If conditions are imposed on you (whether by court order, agreement 6568 or otherwise) that contradict the conditions of this License, they 6569 do not excuse you from the conditions of this License. If you 6570 cannot convey a covered work so as to satisfy simultaneously your 6571 obligations under this License and any other pertinent obligations, 6572 then as a consequence you may not convey it at all. For example, 6573 if you agree to terms that obligate you to collect a royalty for 6574 further conveying from those to whom you convey the Program, the 6575 only way you could satisfy both those terms and this License would 6576 be to refrain entirely from conveying the Program. 6577 6578 13. Use with the GNU Affero General Public License. 6579 6580 Notwithstanding any other provision of this License, you have 6581 permission to link or combine any covered work with a work licensed 6582 under version 3 of the GNU Affero General Public License into a 6583 single combined work, and to convey the resulting work. The terms 6584 of this License will continue to apply to the part which is the 6585 covered work, but the special requirements of the GNU Affero 6586 General Public License, section 13, concerning interaction through 6587 a network will apply to the combination as such. 6588 6589 14. Revised Versions of this License. 6590 6591 The Free Software Foundation may publish revised and/or new 6592 versions of the GNU General Public License from time to time. Such 6593 new versions will be similar in spirit to the present version, but 6594 may differ in detail to address new problems or concerns. 6595 6596 Each version is given a distinguishing version number. If the 6597 Program specifies that a certain numbered version of the GNU 6598 General Public License “or any later version” applies to it, you 6599 have the option of following the terms and conditions either of 6600 that numbered version or of any later version published by the Free 6601 Software Foundation. If the Program does not specify a version 6602 number of the GNU General Public License, you may choose any 6603 version ever published by the Free Software Foundation. 6604 6605 If the Program specifies that a proxy can decide which future 6606 versions of the GNU General Public License can be used, that 6607 proxy’s public statement of acceptance of a version permanently 6608 authorizes you to choose that version for the Program. 6609 6610 Later license versions may give you additional or different 6611 permissions. However, no additional obligations are imposed on any 6612 author or copyright holder as a result of your choosing to follow a 6613 later version. 6614 6615 15. Disclaimer of Warranty. 6616 6617 THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 6618 APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE 6619 COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” 6620 WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, 6621 INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 6622 MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE 6623 RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. 6624 SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL 6625 NECESSARY SERVICING, REPAIR OR CORRECTION. 6626 6627 16. Limitation of Liability. 6628 6629 IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN 6630 WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES 6631 AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR 6632 DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR 6633 CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE 6634 THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA 6635 BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 6636 PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 6637 PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF 6638 THE POSSIBILITY OF SUCH DAMAGES. 6639 6640 17. Interpretation of Sections 15 and 16. 6641 6642 If the disclaimer of warranty and limitation of liability provided 6643 above cannot be given local legal effect according to their terms, 6644 reviewing courts shall apply local law that most closely 6645 approximates an absolute waiver of all civil liability in 6646 connection with the Program, unless a warranty or assumption of 6647 liability accompanies a copy of the Program in return for a fee. 6648 6649END OF TERMS AND CONDITIONS 6650=========================== 6651 6652How to Apply These Terms to Your New Programs 6653============================================= 6654 6655If you develop a new program, and you want it to be of the greatest 6656possible use to the public, the best way to achieve this is to make it 6657free software which everyone can redistribute and change under these 6658terms. 6659 6660 To do so, attach the following notices to the program. It is safest 6661to attach them to the start of each source file to most effectively 6662state the exclusion of warranty; and each file should have at least the 6663“copyright” line and a pointer to where the full notice is found. 6664 6665 ONE LINE TO GIVE THE PROGRAM'S NAME AND A BRIEF IDEA OF WHAT IT DOES. 6666 Copyright (C) YEAR NAME OF AUTHOR 6667 6668 This program is free software: you can redistribute it and/or modify 6669 it under the terms of the GNU General Public License as published by 6670 the Free Software Foundation, either version 3 of the License, or (at 6671 your option) any later version. 6672 6673 This program is distributed in the hope that it will be useful, but 6674 WITHOUT ANY WARRANTY; without even the implied warranty of 6675 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU 6676 General Public License for more details. 6677 6678 You should have received a copy of the GNU General Public License 6679 along with this program. If not, see <https://www.gnu.org/licenses/>. 6680 6681 Also add information on how to contact you by electronic and paper 6682mail. 6683 6684 If the program does terminal interaction, make it output a short 6685notice like this when it starts in an interactive mode: 6686 6687 PROGRAM Copyright (C) YEAR NAME OF AUTHOR 6688 This program comes with ABSOLUTELY NO WARRANTY; for details type ‘show w’. 6689 This is free software, and you are welcome to redistribute it 6690 under certain conditions; type ‘show c’ for details. 6691 6692 The hypothetical commands ‘show w’ and ‘show c’ should show the 6693appropriate parts of the General Public License. Of course, your 6694program’s commands might be different; for a GUI interface, you would 6695use an “about box”. 6696 6697 You should also get your employer (if you work as a programmer) or 6698school, if any, to sign a “copyright disclaimer” for the program, if 6699necessary. For more information on this, and how to apply and follow 6700the GNU GPL, see <https://www.gnu.org/licenses/>. 6701 6702 The GNU General Public License does not permit incorporating your 6703program into proprietary programs. If your program is a subroutine 6704library, you may consider it more useful to permit linking proprietary 6705applications with the library. If this is what you want to do, use the 6706GNU Lesser General Public License instead of this License. But first, 6707please read <https://www.gnu.org/licenses/why-not-lgpl.html>. 6708 6709
