1rvm (1.11) stable; urgency=low 2 3 * Added command delete-command-path (also for experimental use of clustering 4 filesystems, where a threaded rm command is much faster than /bin/rm) 5 6 -- Michael Peek <peek@catus.nimbios.org> Tue, 11 Nov 2014 14:52:04 -0500 7 8rvm (1.10) stable; urgency=low 9 10 * When deleting an old archive directory, rename it to <timestamp>.deleting 11 * Added an extra check to rm_file() and rm_dir(): If we can't delete the 12 file/directory because it is gone, then don't sweat it. Only throw an 13 exception if the file/directory can't be deleted AND it still exists. 14 * Added option to rsync-connection-type: ssh-local 15 - [configure.in] Test for the default ssh binary to use, added 16 LOCAL_SSH to config.h 17 - [rvm.1.in] Add documentation for ssh-local 18 - [rconfig.h] 19 - Add m_ssh_local_path to class configuration_manager 20 - Add connection_ssh_local to job::rsync_connection_type 21 - Add prototype for global_parser::parse_ssh_local_path() 22 - [rconfig.cc] 23 - Modified job::check 24 "rsync-connection-type references hostname" check 25 - Modified job_parser::parse_rsync_connection_type 26 added parser for "ssh-local" 27 - Added function global_parser::parse_ssh_local_path() 28 (This is to add support for experimental use of clustering filesystems.) 29 * Bugfix: The program was still trying to execute the given action even 30 though configuration failed. Depending on the failure, this could result 31 in a segmentation fault. 32 * Added option ssh-options and <ssh-options>...</ssh-options> configuration 33 * Bugfix: rm_recursive could not remove some files due to permission 34 problems (for files with --- perms). Added code to check to see if the 35 file/directory is writable, and if it's not, attempt to chmod it to 333 so 36 that it is writable/executable (for dirs), then immediately remove it. 37 38 -- Michael Peek <peek@catus.nimbios.org> Mon, 03 Feb 2014 08:47:53 -0500 39 40rvm (1.09) stable; urgency=low 41 42 * Bugfix: estring.cc[101] estring::T_fraction_to_strings() contained a bug 43 whereby some fractional values were rendered into strings incorrectly. 44 45 -- Michael Peek <peek@nimbios.org> Tue, 07 Jan 2014 15:40:14 -0500 46 47rvm (1.08) stable; urgency=low 48 49 * Added configure option: --enable-assume-stl-memory-exception 50 * Added rvm.spec? 51 * Added various changes to compile under OSX: 52 53 ERRORS: 54 1) ld: duplicate symbol bool is_signed_char<char>(char const&) in string.o and error.o 55 Move bool is_signed_char(const char& a_arg) body from types.h to types.cc 56 57 2) ld: duplicate symbol bool is_unsigned_char<unsigned char>(unsigned char const&) in estring.o and error.o 58 Move bool is_unsigned_char(const unsigned char& a_arg) body from types.h to types.cc 59 60 3) ld: duplucate symbol bool is_signed_short<short>(short const&) in estring.o and error.o 61 Move bool is_signed_short(const short& a_arg) body from types.h to types.cc 62 63 4) ld: duplicate symbol bool is_unsigned_short<short>(unsigned short const&) in estring.o and error.o 64 Move bool is_unsigned_short(const unsigned short& a_arg) body from types.h to types.cc 65 66 5) ld: duplicate symbol bool is_signed_int<int>(int const&) in estring.o and error.o 67 Move bool is_signed_int(const int& a_arg) body from types.h to types.cc 68 69 6) ld: duplicate symbol bool is_unsigned_int<unsigned int>(unsigned int const&) in estring.o and error.o 70 Move bool is_unsigned_int(const unsigned int& a_arg) body from types.h to types.cc 71 72 7) ld: duplicate symbol bool is_signed_long<long>(long const&) in estring.o and error.o 73 Move bool is_signed_long(const long& a_arg) body from types.h to types.cc 74 75 8) ld: duplicate symbol bool is_unsigned_long<unsigned long>(unsigned long const&) in estring.o and error.o 76 Move bool is_unsigned_long(const unsigned long& a_arg) body from types.h to types.cc 77 78 9) ld: duplicate symbol bool is_signed_long_long<long long>(long long const&) in estring.o and error.o 79 Move bool is_signed_long_long(const long long& a_arg) body from types.h to types.cc 80 81 10) ld: duplicate symbol bool is_unsigned_long_long<unsigned long long>(unsigned long long const&) in estring.o and error.o 82 Move bool is_unsigned_long_long(const unsigned long long& a_arg) body from types.h to types.cc 83 84 11) ld: duplicate symbol bool is_float<float>(float const&) in estring.o and error.o 85 Move bool is_float(const float& a_arg) body from types.h to types.cc 86 87 12) ld: duplicate symbol bool is_double<double>(const double&) in estring.o and error.o 88 Move bool is_double(const double& a_arg) body from types.h to types.cc 89 90 13) ld: duplicate symbol bool is_bool<bool>(bool const&) in estring.o and error.o 91 Move bool is_bool(const bool& a_arg) body from types.h to types.cc 92 93 14) ld: duplicate symbol char const* type_name<unsigned char>(unsigned char const&) in estring.o and error.o 94 Moved the following function bodies from types.h to types.cc: 95 const char * type_name(const unsigned char & a_arg) 96 const char * type_name(const char & a_arg) 97 const char * type_name(const unsigned short & a_arg) 98 const char * type_name(const short & a_arg) 99 const char * type_name(const unsigned int & a_arg) 100 const char * type_name(const int & a_arg) 101 const char * type_name(const unsigned long & a_arg) 102 const char * type_name(const long & a_arg) 103 const char * type_name(const unsigned long long & a_arg) 104 const char * type_name(const long long & a_arg) 105 const char * type_name(const float & a_arg) 106 const char * type_name(const double & a_arg) 107 const char * type_name(const bool & a_arg) 108 109 15) ld: duplicate symbol float const max_limit<float>() in vaulter.o and timer.o 110 Moved the following function bodies from rmath.h to rmath.cc: 111 const float max_limit<float>() 112 const float min_limit<float>() 113 const double max_limit<double>() 114 const double min_limit<double>() 115 116 Extra: Had to create a types.cc and rmath.cc, which includes adding 117 types.cc and rmath.cc to Makefile.am *_SOURCES= sections. 118 119 16) Failed test-rvm-002: 120 Could not rename file: "./test-rvm.dir/vault-1/2006-06-16.112632" to "./test-rvm.dir/vault-1/2006-06-16.112632.incomplete" 121 Assertion failed: (!thrown), function test_rvm, file test-rvm-002.cc, line 239. 122 123 This test sets up 2006-06-16.112632/ with no write permisions, and the 124 test fails when rename() cannot modify the directory name. 125 126 A) Added more verbosity to rename_file() to specifically throw an error 127 stating that it has no write permissions to the from directory. 128 129 B) Modified archive_manager::archive() to check and see if the 130 permissions on the directory allow modification. If not, throw an 131 error. Why? Because it enforces uniform behavior across all tested 132 operating systems. This leads me to... 133 134 C) Modify test-rvm-002 to expect a thrown error. 135 136 -- Michael Peek <peek@nimbios.org> Wed, 30 Nov 2011 13:23:52 -0500 137 138rvm (1.07) stable; urgency=low 139 140 * Bugfix: Many bugs in exec.cc's execute class: 141 - execute::wait() called the wrong function, ::wait(...), which did not 142 specify which child pid to wait for, therefore causing problems when 143 multiple child processes were running concurrently. 144 - Now using waitpid(...). 145 - New tests added to test-exec.cc that would detect this bug. 146 - execute::child_running(), execute::child_exited() used improper logic. 147 - Rewritten 148 - execute::child_exited_normally() is only intended to check exit code, 149 not signal status, although it is sometimes used improperly elsewhere 150 apparently under the assumption that it does. 151 - Created new member function execute::child_exited_success() to replace 152 instances of improper use. execute::child_exited_success() returns 153 true if both the exit status is 0 and no signals were received by the 154 child. 155 156 -- Michael Peek <peek@nimbios.org> Thu, 30 Sep 2010 09:25:19 -0400 157 158rvm (1.06) stable; urgency=low 159 160 * Bugfix: Allowing directory structures to be created by subprocesses also 161 allows for a race condition whereby mk_dir exits with an error. Directory 162 structures are now created by the parent process instead. 163 * Add: configure option --enable-assume-stl-memory-exception 164 This option tells configure to assume that the STL will in fact throw an 165 exception on memory allocation failure and to skip the test. On modern 166 hardware the test can take quite a long time and can bring the machine to 167 a crawl during the testing process. (In fact, on OSX this test never 168 seems to finish -- the machine simply never runs out of memory? 169 Apparently Steve Jobs has done for the memory chip what Dr. Who did for 170 tiny blue police boxes...) 171 172 -- Michael Peek <peek@nimbios.org> Mon, 27 Sep 2010 23:32:44 -0400 173 174rvm (1.05) stable; urgency=low 175 176 * Update: Updates to code to compile w/ gcc 4.3.2 177 * Bug: In test-rmath, absolute value of a signed char -128 == -128 178 * Bugfix: Depending on the situation, test_cwd() in test-fs.cc will always 179 fail, even though it's not a bug. This test has been removed. 180 * Bugfix: job_archiver::mf_process_rsync_io() had flawed rsync-timeout 181 logic. 182 * Bugfix: When rsync-timout event occurs, job was treated as though it had 183 succeeded b/c rsync didn't report an error code or a signal when it was 184 killed. Now rsync-timeout sets a flag that is checked, in addition to 185 signal and exit code returned by rsync, when evalulating whether or not 186 the job was successful. 187 188 -- Michael Peek <peek@home.nimbios.org> Fri, 12 Jun 2009 10:00:21 -0400 189 190rvm (1.04b) stable; urgency=low 191 192 * Bugfix: job-path processing was ignoring rsync_behavior::ok setting. 193 * Added new rsync exit codes to man page and exit-code strings. 194 195 -- Michael S. Peek <peek@tortoise.tiem.utk.edu> Thu, 28 Aug 2008 09:40:43 -0400 196 197rvm (1.04a) stable; urgency=low 198 199 * Bugfix: Report generation should take rsync-behavior into account in 200 addition to rsync exit code and signal generated when deciding whether or 201 not a job path was OK or ERROR. 202 203 Bugfix needs testing before promoting to 1.05 204 205 -- Michael S. Peek <peek@tortoise.tiem.utk.edu> Wed, 27 Aug 2008 12:34:38 -0400 206 207rvm (1.04) stable; urgency=low 208 209 * Bugfix: rconfig's job::generate_rsync_options_vector() has a completely 210 new parser. 211 212 -- Michael S. Peek <peek@tortoise.tiem.utk.edu> Wed, 2 Jul 2008 13:05:57 -0400 213 214rvm (1.03) stable; urgency=low 215 216 * Bugfix: Applied patch submitted by Ezra Peisach to fix a bug in 217 rsync-options parsing that would result in an empty argument string being 218 passed to rsync by mistake. 219 220 -- Michael S. Peek <peek@tortoise.tiem.utk.edu> Tue, 1 Jul 2008 12:01:59 -0400 221 222rvm (1.02) stable; urgency=low 223 224 * Feature: Added rsync-retry-delay to specify how long to wait (in minutes) 225 before rerunning rsync after a failed attempt. In the event that rsync 226 failed because of an open and changing file that it could not read the 227 last time, this will give the user or OS time to finish and close that 228 file. (A better solution would be to watch the file and back it up as 229 soon as it's free, but that is outside the scope of rvm.) 230 231 * Feature: Added rsync-multi-hardlink option for people with newer versions 232 of rsync that can take advantage of using multiple --link-dest=<path> 233 command line options to rsync. 234 235 * Feature: Added rsync-multi-hardlink-max option to specify a maximum number 236 of --link-dest=<path> options to pass to rsync. Currently the maximum 237 that rsync will accept is 20, so that is used as the default. Rvm will 238 accept higher values, but higher values will cause rsync to exit with an 239 error. (Not enforcing a maximum in rvm allows for future changes in 240 rsync, but allows for user error.) 241 242 * Bugfix: A simple bug in implementation caused rsync-retry-count to be off 243 by one. Setting rsync-retry-count to 0 would cause rsync never to run at 244 all, and a default setting of 3 would mean that rsync is only retried 2 245 times. 246 247 * After a system upgrade to Debian/Etch, the first order of business is to 248 make sure that the latest version of RVM still compiles and passes all of 249 it's tests. 250 251 - The following error message was produced during compilation: 252 253 tortoise% make 254 cd . && /bin/sh /home/peek/prog/rvm/src/missing --run automake-1.10 255 --foreign --include-deps Makefile 256 Makefile.am:1090: `#' comment at start of rule is unportable 257 Makefile.am:1091: `#' comment at start of rule is unportable 258 Makefile.am:1092: `#' comment at start of rule is unportable 259 Makefile.am:1095: `#' comment at start of rule is unportable 260 Makefile.am:1096: `#' comment at start of rule is unportable 261 Makefile.am:1097: `#' comment at start of rule is unportable 262 Makefile.am:1100: `#' comment at start of rule is unportable 263 Makefile.am:1101: `#' comment at start of rule is unportable 264 Makefile.am:1102: `#' comment at start of rule is unportable 265 Makefile.am:1116: `#' comment at start of rule is unportable 266 make: *** [Makefile.in] Error 1 267 268 Removed '#' comments from Makefile.am to remove these errors. 269 270 - The following error message was produced during compilation: 271 272 tortoise% make 273 make all-am 274 make[1]: Entering directory `/home/peek/prog/rvm/src' 275 g++ -DHAVE_CONFIG_H -I. -g -Wall -MT estring.o -MD -MP -MF 276 .deps/estring.Tpo -c -o estring.o estring.cc 277 estring.h:259: error: extra qualification ‘estring::’ on member 278 ‘T_fraction_to_strings’ 279 estring.h:262: error: extra qualification ‘estring::’ on member 280 ‘T_integral_to_string’ 281 estring.h:265: error: extra qualification ‘estring::’ on member 282 ‘T_string_to_integral’ 283 estring.h:269: error: extra qualification ‘estring::’ on member 284 ‘T_string_to_signed_integral’ 285 estring.h:272: error: extra qualification ‘estring::’ on member 286 ‘T_string_to_fractional’ 287 estring.cc: In member function ‘void estring::T_integral_to_string(const 288 T&, std::string&) [with T = short unsigned int]’: 289 estring.cc:1130: instantiated from here 290 estring.cc:65: warning: comparison is always true due to limited range 291 of data type 292 make[1]: *** [estring.o] Error 1 293 make[1]: Leaving directory `/home/peek/prog/rvm/src' 294 make: *** [all] Error 2 295 296 Removed "estring::" from before T_* templated functions. 297 298 - The following error message was produced during compilation: 299 300 g++ -DHAVE_CONFIG_H -I. -g -Wall -MT rconfig.o -MD -MP -MF 301 .deps/rconfig.Tpo -c -o rconfig.o rconfig.cc 302 rconfig.h:269: error: using typedef-name 303 ‘configuration_manager::cfgfiles_type’ after ‘class’ 304 rconfig.h:225: error: ‘configuration_manager::cfgfiles_type’ has a 305 previous declaration here 306 rconfig.h:369: error: extra qualification ‘job_parser::’ on member 307 ‘read_job’ 308 rconfig.cc: In member function ‘void configuration_manager::clear()’: 309 rconfig.cc:1004: error: request for member ‘clear’ in 310 ‘((configuration_manager*)this)->configuration_manager::m_cfgfiles’, 311 which is of non-class type ‘int’ 312 rconfig.cc: In member function ‘void configuration_manager::init(int, 313 char**)’: 314 rconfig.cc:1111: error: request for member ‘push_back’ in 315 ‘((configuration_manager*)this)->configuration_manager::m_cfgfiles’, 316 which is of non-class type ‘int’ 317 rconfig.cc:1149: error: request for member ‘push_back’ in 318 ‘((configuration_manager*)this)->configuration_manager::m_cfgfiles’, 319 which is of non-class type ‘int’ 320 rconfig.cc:1178: error: request for member ‘begin’ in 321 ‘((configuration_manager*)this)->configuration_manager::m_cfgfiles’, 322 which is of non-class type ‘int’ 323 rconfig.cc:1178: error: request for member ‘end’ in 324 ‘((configuration_manager*)this)->configuration_manager::m_cfgfiles’, 325 which is of non-class type ‘int’ 326 make[1]: *** [rconfig.o] Error 1 327 make[1]: Leaving directory `/home/peek/prog/rvm/src' 328 make: *** [all] Error 2 329 330 Removed "class" from rconfig.h[269] declaration of m_cfgfiles, 331 Added "class" to rconfig.h[225] typedef of cfgfiles_type. 332 333 - The following error message was produced during compilation: 334 335 tortoise% make 336 make all-am 337 make[1]: Entering directory `/home/peek/prog/rvm/src' 338 g++ -DHAVE_CONFIG_H -I. -g -Wall -MT rconfig.o -MD -MP -MF 339 .deps/rconfig.Tpo -c -o rconfig.o rconfig.cc 340 rconfig.h:369: error: extra qualification ‘job_parser::’ on member 341 ‘read_job’ 342 make[1]: *** [rconfig.o] Error 1 343 make[1]: Leaving directory `/home/peek/prog/rvm/src' 344 make: *** [all] Error 2 345 346 Removed "job_parser::" from rconfig.h[369]. 347 348 - The following error message was produced during compilation: 349 350 g++ -DHAVE_CONFIG_H -I. -g -Wall -MT vaulter.o -MD -MP -MF 351 .deps/vaulter.Tpo -c -o vaulter.o vaulter.cc 352 table.h:124: error: extra qualification ‘table::’ on member ‘col_width’ 353 table.h:125: error: extra qualification ‘table::’ on member ‘row_width’ 354 table.h:126: error: extra qualification ‘table::’ on member ‘col_height’ 355 table.h:127: error: extra qualification ‘table::’ on member ‘row_height’ 356 make[1]: *** [vaulter.o] Error 1 357 make[1]: Leaving directory `/home/peek/prog/rvm/src' 358 make: *** [all] Error 2 359 360 Removed "table::" from these lines. 361 362 * Experimental: See if I can figure out why vault-locking fails. 363 364 -- Michael S. Peek <peek@tortoise.tiem.utk.edu> Thu, 7 Jun 2007 11:09:10 -0400 365 366rvm (1.01) stable; urgency=low 367 368 * Added code to check to see that the vault is writable before running 369 rsync. (In the event that something goes wrong with the filesystem, the 370 mount may be changed to read-only. 371 372 * Added a new global option: error-logging-level. When archiving a path for 373 a job, the logging-level is used unless rsync returns an error, then the 374 error-logging-level will be used for the subsequent retries of that path. 375 376 * Fixed bug that would not resolve wildcard vault paths when only one path 377 was found that matched the wildcard. 378 379 * Fixed bug in vault_manager::get_archive_list() that would incorrectly 380 construct the list of valid archive subdirectory names in a vault. 381 382 * Added new global option: vault-locking. This option takes a boolean 383 value. If enabled, rvm will lock the vault that it's using. Subsequent 384 instances of rvm will (if run with vault-locking enabled) check to see if 385 the vault is locked when selecting a vault. The behavior of rvm under 386 vault-locking is as follows: 387 388 - If a locked vault contains an archive with the same timestamp as the one 389 being used by rvm then rvm exists with an error. 390 391 - Otherwise, if a vault is locked, then that vault is ignored during the 392 selection process and the next best vault is chosen instead. 393 394 - If all vaults are locked then no vault is selected and rvm exits with an 395 error. 396 397 Rvm implements vault locking by creating a .rvm_lock file in the vault 398 directory containing rvm's PID. When rvm checks to see if the vault is 399 locked, it first checks to see if a .rvm_lock file exists, and if so, it 400 then checks to see if a process is running with the PID listed in the 401 .rvm_lock file. 402 403 NOTE: If vault-locking is disabled then any existing locks created by 404 other instances of rvm are ignored. (Be forewarned...) 405 406 * Fixed bug that would allow jobs that generate an empty job ID string for 407 status reports and the report log. If the generated job ID string is 408 empty, then the job's jobname is used. A check is now performed after 409 configuration that tests the job's generate_job_id() function to see if 410 the string generated is empty. If it is, an error message is produced 411 instructing the user to assign a descriptive jobname to the job in 412 question. 413 414 * Updated man page rvm.1 with any changes listed above that affect 415 configuration and/or behavior. 416 417 * Ran a spell checker on the man page -- Gads! I'm a terrible spellar. 418 419 * Added an introduction to the log file stating program name and version. 420 This is done by log_manager::init(). 421 422 * Added new global commands: delete-old-log-files and 423 delete-old-report-files. Both take a boolean value. If enabled, then 424 when rvm deleted old archives from a vault, it will also search for and 425 delete the log file and/or report file from the log-dir directory. 426 427 -- Michael Peek <peek@tiem.utk.edu> Wed, 21 Jun 2006 11:49:19 -0400 428 429rvm (1.0) stable; urgency=low 430 431 * Initial Release. 432 * Development continuing on a debian system as the old solaris systems are 433 being depreciated. As such, for convenience sake, the changelog format 434 will switch to the format used by debian package management tools. 435 * When deleting a file or directory, check that the file/directory exists. 436 If it does not, return without throwing an error. 437 * When deleting a file or directory, and deletion failed, check to see if 438 the file/directory still exists. It's possible that some user or some 439 other running process just deleted it. If it does not exist, return 440 without throwing an error. 441 * When deleting a file or directory, and deletion failed, check to see if we 442 have write privledges. This allows for a more descriptive error message. 443 * A debian/ directory with files to generate a single, debian-native rvm 444 package. 445 * Code to Makefile.am to generate a debian package. 446 * Code to Makefile.am to generate an RPM package. (Experimental) 447 * Code to Makefile.am to generate a binary tgz package. 448 449 * Contents of old changelog: 450 451 2005-04-15: Beta 0.90.4 452 453 Added: 454 455 - RVM will now log a warning message when deleting old, incomplete 456 archives. 457 458 - vault_manager::get_archive_list() now only includes directories of 459 archives with valid timestamp names, including incomplete archives. 460 461 - error::push_back(const error& a_e), which will copy all the error 462 instances from a_e to another error object. 463 464 Fixed: 465 466 - A bug in vault_manager::select() that prevented it from selecting the 467 correct vault in the event that an archive of the given timestamps 468 already existed in an incomplete form. 469 470 - A bug in vault_manager::delete_oldest_archive() that prevented it from 471 recognizing an archive of the current timestamp in incomplete form when 472 deleting old archives in the event of archive overflow. 473 474 - Many bug fixes in global_parser::parse_vault that prevented wildcard 475 vault pathnames from being resolved and processed correctly. 476 477 Among other things, this fix also prevents RVM from exiting with an 478 error in the event that one of the wildcard pathnames resolved points to 479 a directory with incorrect permissions. 480 481 - Changed: 482 std::string("stuff...") 483 to: 484 static_cast<std::string>("stuff...") 485 486 The former line causes an error on some compilers. 487 488 2004-05-10: Beta 0.90.3 489 490 Todo: 491 492 - Vaulter and archiver are supposed to work together to insure that an 493 archive by the same timestamp isn't created on two different vaults. 494 Somehow this got screwed up, because I have found this to be the case on 495 one occasion. Check the logs and possibly modify the code based on 496 what's found to insure that this doesn't happen again. 497 498 Added: 499 500 Changed: 501 502 - The archiver now creates the archive directory with the extension 503 ".incomplete". Once RVM completes archiving all jobs, the archive 504 directory is renamed, removing the ".incomplete" extension. 505 506 a) This allows the archive administrator to see at a glance if an archive 507 has been finished; or if RVM is no longer running, that an archive is 508 incomplete and possibly corrupted. 509 510 b) In the event that RVM is still running when another instance of RVM is 511 started, the second instance of RVM will know not to use the 512 still-incomplete previous archive as a hardlink source for the new 513 archive. 514 515 c) Incomplete archives are not listed in the link catalog. 516 517 - Cleaned up a lot of code associated with vault overflow handling. 518 519 - Rvm no longer exits when a vault cannot be found. An error is written 520 to stderr (since this is done in the configuration phase and the logger 521 has not been initialized yet) and the vault path is ignored. 522 523 This has prompted me to start designing modifications to the reporter 524 and how it interacts with the other objects... (See Ideas's above) 525 526 Fixed: 527 528 - Added missing subdirectory assignment member functions and operators. 529 This should fix a bug in vault_manager::delete_oldest_archive(). 530 531 - Fixed a nasty bug in global_parser::parse_vault() that would not 532 accept a full pathname to a vault(!?) (Don't know how that one slipped 533 through the tests.) 534 535 - Fixed numerous bugs in vault overflow detection and handling. 536 - Added "\n" to logs generated by vaulter::delete_oldest_archive() 537 538 - vault_manager::prepare() would delete more than one vault per 539 execution when vault-overflow-behavior was set to delete-oldest. 540 541 - Vault overflow would mysteriously disappear by the time rsync exited. 542 Culprit was traced to the overflow event being caused by a temporary 543 file created by rsync and then removed before exiting with an error 544 condition. Once rsync exited the overflow situation would no longer 545 exist and rvm would not attempt to delete an old archive. This would 546 result in rsync being re-run and the same situation happening over 547 again. Now if an overflow situation is detected a flag is set, and 548 even if the overflow situation no longer exists after rsync exits rvm 549 will still know to attempt to clean up the vault before retrying rsync. 550 551 - If rsync exits with an error, and a vault overflow event has been 552 detected, then the child process handling that job will not waste time 553 attempting to retry that path, but will move on to the next path. 554 555 - Fixed a bug in vault_manager::prepare() that would continue to delete 556 vaults after a single vault had already been deleted. 557 558 - If an overflow is detected, and vault-overflow-behavior is set to 559 delete-oldest, and the oldest archive found at the time of execution 560 has already been deleted, then attempt to continue archiving anyway. 561 Do not delete any more old archives, and do not abort the current 562 archive. Instead, let the jobs run and let rsync fail on it's own if 563 there is not enough space. 564 565 - Fixed a bug in vault_manager::prepare() that would not set the exit 566 status to 2 (vault full) when the vault fills up and no more archive 567 deletions can take place when vault-overflow-behavior is set to 568 delete-oldest. 569 570 - Fixed a bug where jobs exiting with an error are not rescheduled in 571 the event that the vault is also found to have exceeded it's overflow 572 threshold. 573 574 - Fixed a bug where jobs exiting with an error, and the vault has not 575 exceeded it's overflow threshold, and the job continues to sit in the 576 queue. 577 578 - Fixed a bug in job::generate_archive_path() that would create 579 duplicate directory structures for paths that are copied (as opposed to 580 paths whose *contents* are copied). This bug would be seen when a path 581 was specified without a '/' character at the end. For instance: 582 583 ... 584 <job> 585 ... 586 path /home/peek/download/ddd-3.3.1.tar.gz 587 ... 588 </job> 589 ... 590 591 This would result in the archive path name 592 <archive-dir>/home/peek/download/ddd-3.3.1.tar.gz/ being created, and 593 the file ddd-3.1.1.tar.gz being placed inside. 594 595 This bug would also affect directories: 596 597 ... 598 <job> 599 ... 600 path /home/peek/download 601 ... 602 </job> 603 ... 604 605 This would result in the archive path name 606 <archive-dir>/home/peek/download/ being created, and the directory 607 download/ being placed inside. 608 609 - Fixed a nasty bug in estring that would yield incorrect values for 610 some signed integral types. 611 612 - Fixed a minor bug in report generation. 613 614 - Fixed a minor bug in error logging. 615 616 - Fixed a bug that would allow the timestamp-resolution setting in a 617 configuration file override any timestamp resolution setting derived 618 from a --timestamp command line option. 619 620 - Updated man page to reflect behavioral changes and lessons learned. :) 621 622 2004-05-08: Beta 0.90.2 623 624 Changed: 625 626 - Miscilaneous spelleeng korrections 627 628 Bugs Fixed: 629 630 - Fixed a bug in rsync command line generation that the last bug fix 631 introduced. Deja-vu. Also wrote a test to make sure it's generating 632 what I think it is. (Should have done that the first time.) 633 634 2004-04-28: Beta 0.90.1 635 636 Changes: 637 638 - Man page corrections as provided by Tony Sequeira. 639 640 - Rsync uses --exclude-from and --include-from command line options, not 641 --exclude-file and --include-file. Updated RVM man page to avoid 642 confusion. 643 644 Bugs fixed: 645 646 - Fixed a bug in rsync command line generation -- again. I thought this 647 one was licked some time ago, but I must have goofed somewhere. 648 649 - estring::T_fraction_to_strings() would incorrectly convert some 650 fractional values without paying attention to the set precision value. 651 652 2004-04-22: Beta 0.90 653 654 This release incorporates all features originally planned for RVM by the 655 author, and concludes RVM's alpha development status. An extensive beta 656 testing phase will now begin where RVM will be used daily in a production 657 environment. 658 659 New features may be added as requested by the OpenSource community and as 660 time for development allows. As always patches for both feature requests 661 and bug reports are welcomed by the author with open arms. 662 663 - Added feature request: The rsync-options, when used in a job context, 664 now adds to any rsync options specified in the last default context 665 rather than overwriting any previous value. To override an 666 rsync-options command specified in a default context the clear command 667 must now be used. 668 669 - Replaced safe_add and safe_mul functions with (what I hope is) the 670 more robust, complete, generic, and faster safe_num class. 671 672 - New table object, used by the report manager in formatting text reports. 673 674 - Report manager completely rewritten. 675 676 - Statistics reported by rsync removed from reports. (Output from rsync 677 for runs involving hardlinks were erroneous, and thus RVM's reports 678 would confuse and annoy RVM's simpleton author.) The author considered 679 porting the analyzer from 0.79.1-alpha, but decided that it would only 680 add to RVM's runtime. 681 682 On the author's system, it currently takes over 18 hours to archive 683 334GB using two instances of rsync running in parallel on a virtual 684 200base-T connection consisting of two bonded 100base-T physical 685 connections. The author feels that this is taking too long already, and 686 is currently seeking ways to cut RVM's runtime. 687 688 Maybe if enough people want it it will incorporated in a future release. 689 690 - Added a new command: io-poll-interval. This command appears in a 691 global context and accepts any non-negative integer value. This command 692 instructs RVM to sleep some number of seconds between polling for I/O 693 from children. 694 695 - Modified job_archiver::mf_process_rsync_io() for more efficiency. 696 697 - Forgot to modify the log_manager -- if a previous log file already 698 exists by the same name, RVM will start appending numbers to the name 699 until it finds a filename that doesn't already exist. The same 700 modification has been made to the report_manager too. 701 702 - Modified the log file names generated. Before, RVM would use: 703 log.<timestamp> and report.<timestamp>. This has been reversed: 704 <timestamp>.log and <timestamp>.report. The author liked having both 705 reports together in an alphabetized directory listing. (The author is 706 wondering how many people this will confuse.) 707 708 Bugs fixed: 709 710 - A patch submitted by Ezra Peisach fixes a bug that prevented RVM from 711 constructing a valid command line when connecting to a remote rsync 712 server without specifying the port. 713 714 Platforms tested: 715 716 Platform Config Compile Passes Tests 717 ---------------------------------------- ------ ------- ----------------- 718 i386-unknown-freebsd4.10 FreeBSD-4.8 OK OK OK 719 i386-unknown-netbsdelf1.6.1 NetBSD-1.6.1 OK OK OK 720 i686-pc-linux-gnu Debian-2.2 OK OK 1 test fails [2] 721 i686-pc-linux-gnu RedHat-9.0 OK OK OK 722 powerpc-apple-darwin6.8 OS-X-10.2 OK OK 1 test fails [1] 723 sparc-sun-solaris2.9 Solaris-9 OK OK OK 724 x86_64-unknown-linux-gnu SuSE-8-ES OK OK 1 test fails [2] 725 726 727 [1]: test-rconfig-004 fails on Mac-OS-X-10.2 728 This test uses a set of configuration files that intentionally loop, 729 causing RVM to re-read the same files over and over again util memory is 730 exhausted. On this platform, when memory is exhausted, test-rconfig-004 731 fails at (backtrace): 732 733 1) _lookup_all_secure() 734 2) lu_getpwname() 735 3) getpw_internal() 736 4) getpw() 737 5) filestatus::path(std::string) 738 739 This failure is not seen in OS-X-10.1. To the best of the author's 740 knowledge this is an emergent feature of 10.2 that is caused by either a 741 lack of memory or a lack of stack space. 742 743 No error is thrown by test-rconfig-004, it merely exits with a 744 segmentation fault after receiving signal EXC_BAD_ACCESS (could not 745 access memory error). 10.2 seems to ignore the try {...} block that 746 wraps the code that causes this mess, meaning that if such a lack of 747 resource occurs in real life RVM will be unable to recover from the 748 error. 749 750 To the best of the author's knowledge, so long as a user does not run 751 out of memory or stack space then RVM should function as desired. 752 753 The author seeks a solution. 754 755 [2]: test-rvm fails 756 This has been seen on SuSE-8-ES running on an AMD64 Opteron, and on 757 Debian-2.2 running on an i686. 758 759 Test-rvm fails an assertion at line 580: in.is_open(). Errno is not set 760 when the failure occurs. At this time the cause of failure is unknown. 761 It is unknown whether or not RVM will function as desired on this 762 platform. 763 764 This behavior is not seen on Redhat-9.0 running on the same architecture 765 as Debian-2.2. 766 767 The author seeks a solution. 768 769 770 To-do: 771 772 - Comb through the code and make sure that all exceptions are being 773 handled gracefully. (*sigh*) 774 775 2004-03-23: Alpha 0.80.2 776 777 This is a bugfix release. 778 779 Bugs fixed: 780 781 - archive_manager now throws an exception if used uninitialized. 782 783 - Fixed an implicit conversion bug that would bite on some systems when 784 allocating a new job_archiver using a 785 configuration_manager::job_type::const_iterator as an argument. 786 787 - catalog_manager now throws an exception if used uninitialized. (Fixed 788 one point of usage where it was being used uninitialized.) 789 790 - Errno behaves in different ways on different systems. Therefore the 791 error object now takes an errno number as a parameter to the constructor 792 rather than taking it directly from the errno variable. This allows 793 greater control as errno can be used for errors that are expected to set 794 the errno variable, and 0 can be used for errors that are not expected 795 to set the errno variable. This provides a much more uniform behavior 796 of the error object from one system to another. 797 798 - error::str() now uses a char[] buffer and snprintf() to avoid a 799 possible buffer overflow segfault. 800 801 - estring now defines the alphabet length used in base conversions to 802 avoid possible data corruption. 803 804 - estring now uses a char[] buffer and snprintf() to avoid a possible 805 buffer overflow segfault. 806 807 - log_manager now throws an exception if used uninitialized. 808 809 - configuration_manager::clear() now resets the default log-dir to the 810 value determined by the configure script rather than to the hard-coded 811 value of "/var/log/rvm". 812 813 - configuration_manager now throws an exception if used uninitialized. 814 815 - rconfig.cc's parse_dirname() now checks for memory allocation error. 816 817 - report_manager now throws an exception if used uninitialized. 818 819 - Some tests altered to more accurately simulate an actual rvm run, code 820 simplification of the *_manager classes ensued. 821 822 - test-logger.cc expanded. 823 824 - Some minor bug fixes to several tests that would prevent them from 825 passing on various systems. 826 827 - timer.cc's timer::make_string_() now checks for memory allocation 828 error. 829 830 - Modified timer::eta() to check for math overflow error. Modified one 831 version of timer::eta() function to check that the timer has been both 832 started and stopped before attempting calculation. 833 834 - tstamp.cc's timestamp::str() now checks for memory allocation error. 835 836 - vault_manager now throws an error if used uninitialized. 837 838 - vault_manager::usage() now checks for math errors and writes an error 839 to the log file is one is detected. (No longer causes rvm to die with 840 an error.) 841 842 - Fixed a bug in vault_manager::select() that could affect the max-free 843 vault selection behavior. 844 845 - vault_manager::select() now checks for memory allocation error. 846 847 - Fixed a bug in fs.cc's mk_relative_path() that generated broken 848 relative symbolic links. 849 850 2004-02-16: Alpha 0.80 851 852 Completely rewrote much of the code, too many changes to list them all. 853 (In fact, with so many changes, 0.80 may have more bugs than 0.79.1...) 854 855 Rvm should now compile and run under gcc 2.95.x. 856 857 Configuration has been completely redesigned. Old configuration was 858 based on a Qmail style schema using "control files". The new 859 configuration method resembles something more akin to apache's 860 httpd.conf. Many options can be specified on a job-by-job basis with 861 defaults that can be set to be assumed by subsequent jobs. 862 863 Rvm can now use rsync to archive locally-mounted files or files on an 864 rsync server, as well as remote hosts accessed via RSH or SSH. 865 866 Added a timeout option, after which, if no I/O is received by rvm from 867 rsync, rvm will terminate the rsync process (and optionally re-try it). 868 869 Rvm will now give different exit codes depending on differing error 870 conditions that arise. 871 872 Previous versions of rvm included an analyzer run by the child process 873 to count the number and size of new files versus files hardlinked from 874 an older archive. This analyzer is no longer present in rvm. Instead, 875 rvm reads the output from rsync to obtain new vs. hardlinked files. 876 However, the statistics gathered through this new method could prove to 877 be confusing. If time permits, and the OpenSource community desires, 878 then I may recode the analyzer back in for a future version. 879 880 The report manager needs to be reworked. 881 882 The error manager needs to be reworked. 883 884 A lot of code needs minor work and general cleaning up. Much of the C++ 885 code in rvm is probably in violation of the philosophical spirit of OO 886 design. 887 888 At the moment there is only one log file. In 0.79.1 there was one 889 master log file and one log file for each client. However, in 0.80 890 RVM's design was moved from a client-based paradyme to the more abstract 891 job-based paradyme. At this time it is not obvious to the author how to 892 map each job to it's own log filename in an intuitive fashion, so the 893 multiple log file scheme has been discontinued. The new log file scheme 894 includes logging the PID and a uniq ID for each job on every line of the 895 log file, this way output from a particular child process or a 896 particular job can at least be grepped for. (If the OpenSource 897 community cares enough about it, the author could add a log-dir command 898 to the job context language.) 899 900 Fixed a bug in the estring object. It could have been caught by a 901 regression test, but wasn't because the tests are incomplete. Many, many 902 more tests should be written, but the author simply has not had the time. 903 904 Porting: 905 906 The author has tried to write rvm to be as portable as possible. 907 However, since the author's time and access to various systems is 908 limited, there could by problems. At the time of this writing the test 909 suite has been run on the following systems: 910 911 a) PowerPC / MacOS-X 10.1 : All 13 tests pass 912 913 b) PowerPC / MacOS-X 10.2 : 1 of 13 tests fails 914 915 test-rconfig segfaults at: 916 #0 0x9008ca74 in _lookup_all_secure () 917 #1 0x90031e64 in lu_getpwnam () 918 #2 0x9008b3f8 in getpw_internal () 919 #3 0x9008b268 in getpw () 920 #4 0x0005a7dc in filestatus::path(std::string) (this=0xbff84530, a_path={static npos = 4294967295, _M_dataplus = {<allocator<char>> = {<No data fields>}, _M_p = 0x3285cc "."}, static _S_empty_rep_storage = {0, 0, 357, 0}}) at fs.cc:726 921 922 The failure is in getpwuid() in the filestatus::path() function. The 923 uid number passed to getpwuid has been verified to be correct, and a 924 subsequent test of getpwuid from a separate program succeeds, which 925 leads me to suspect a memory leak somewhere. (*gag*) 926 927 c) Sparc / Solaris 8 : All 13 tests pass 928 929 d) Sparc / Solaris 9 : 1 of 13 tests failed 930 931 test-rvm throws the following error: 932 *** INTERNAL ERROR 933 Multiplication overflow error: 320646177970143232 * 100 934 in void safe_mul(T&, T) [with T = long long unsigned int] at rmath.h[56] 935 Assertion failed: !thrown, file test-rvm.cc, line 263 936 937 Most likely this means that I'm not initializing a variable somewhere. 938 939 e) AMD64 / SuSE 8 : All 13 tests passed 940 941 f) i686 / Debian 2.2 : All 13 tests passed 942 943 g) i686 / FreeBSD 4.8 : All 13 tests passed 944 945 h) i686 / RedHat 9.0 : All 13 tests passed 946 947 To-do: 948 949 1) Develop a tabular object that uses estring objects to lay out 950 reports. There's a minor, asthetic bug in the current layout code, and 951 I think this would be very helpful. 952 953 2) Rewrite the analyzer from 0.79.1 to gather statistics. Maybe the 954 use of rsync's output versus statistics gathered by the analyzer should 955 be a configurable option. 956 957 3) From (1) and (2), rewrite the reporter classes. 958 959 4) Clean up code. 960 961 5) Test, test, test, test, test... 962 963 2003-12-04: Alpha 0.79.1 964 965 Fixed a bug in timer object that would attempt to divide by zero if the 966 elapsed duration was zero seconds. (Duh!) 967 968 Modified and added to autoconf scripts to identify which flavor of 969 statfs a system has, and what include files are needed to use it. 970 971 Added tests to autoconf to determine whether a system uses struct 972 statfs.f_namemax or struct statfs.f_namelen. 973 974 Added an option to the timer object to force it to use Greenwich mean 975 time instead of localtime. ...Silly test programs failing because 976 people were in a different time zone. Modified test-timer.cc and 977 test-report.cc to suit. (Duh!) 978 979 2003-11-24: Alpha 0.79 980 981 Moved configuration file retry-count to rsync-retry-count. 982 983 Added the configuration file rsync-behavior to allow rvm to map 984 different exit error codes from rsync to commands telling rvm how to 985 handle them. For the errors that are retryable, rvm will retry up to 986 rsync-retry-count times. 987 988 Included tests in autoconf scripts to check for _FILE_OFFSET_BITS and 989 _LARGE_FILES for Large File support. 990 991 Lots of code cleanup, more still needed. Rvm is far from **optimized**, 992 but it seems to work. Overflow code and many error-recovery routines 993 still untested. 994 995 2003-10-31: Alpha 0.78 996 997 More work on backup.cc 998 999 Revamped logger. Didn't wind up using it the way I had anticipated. 1000 1001 Wrote help.cc which has version() to print a useless version message, 1002 and help() to print some useless help text. 1003 1004 Updated the man page, spellchecked the man page. 1005 1006 Rvm has not been rigorously tested yet. Feedback and patches are 1007 welcome. 1008 1009 2003-10-23: Alpha 0.77 1010 1011 Added --timestamp <timestamp> command line option to allow retry of a 1012 failed or interrupted previous archive attempt. 1013 1014 Moved backup code to backup.cc 1015 1016 Moved catalog code to catalog.cc 1017 1018 Revamped backup code, squashed bug that prevented proper flushing of IO 1019 from child to parent. Still needs lots of work. 1020 1021 2003-10-17: Alpha 0.76 1022 1023 rvm.cc revamped for parallel processing of clients using the class 1024 execute. 1025 1026 Report generation removed -- must be reworked for parallel code. 1027 1028 Wrote a man page. 1029 1030 Made a lot of changes to the code and to the autoconf scripts for various 1031 fixes. 1032 1033 2003-10-09: Alpha 0.75 1034 1035 Changed name to rvm, for Rsync Vault Manager. Vbk didn't really stand for 1036 anything, but I hadn't thought of anything better to name it at the time. 1037 1038 Introduced the --relink command line option to have rvm regenerate the 1039 link catalog. 1040 1041 Introduced class report for report generation, integrated into rvm.cc. 1042 1043 Modified timestamp class to include a resolution variable, integrated into 1044 config.cc and rvm.cc. 1045 1046 Completely revamped the way class config reads configuration files. 1047 Files can now contain comments, and line lengths are no longer limited. 1048 Modified the syntax of <logdir>/clients. Clients and pathnames are now 1049 specified as multiple lines of <client>:<path>, and there can be 1050 multiple lines with the same client name. This also resulted in a 1051 change in the way the config class stored this information. 1052 1053 Squashed many bugs. 1054 1055 2003-09-30: Alpha 0.50 1056 1057 (Alpha test code works!) 1058 1059 Code cleanup. 1060 1061 Rewrite of class error. Class error now contains an internal flag 1062 noting whether or not the error is internal or external, and contains a 1063 stack trace of the error all the way back up to main(). Also introduced 1064 some macros for ERROR(), INTERNAL_ERROR(), ERROR_INSTANCE(), and TRY() 1065 -- which is a generic try{} block to push information onto the error's 1066 stack trace. 1067 1068 Rewrite of all functions to utilize the new error object and macros. 1069 1070 Introduced class logger. Logger is used to give a generic interface to 1071 logging services in such a way that log messages could be sent to any 1072 output device, whether it be std::cout, std::cerr, a file, a file 1073 descriptor, or another ostream object. 1074 1075 Introduced mk_relative_symlink() in fs.cc, which will take two absolute 1076 paths, a source and a destination, and create the destination as a 1077 relative symbolic link to the source. 1078 1079 Introduced the linkdir configuration file, which holds a path to a 1080 directory in which to generate a catalog of backups, by both timestamp 1081 and client, as relative symbolic links to their real locations in the 1082 vaults. 1083 1084 Archiving code completely rewritten. Introduced the exclude and 1085 exclude-<client> configuration files. If either the exclude file or the 1086 exclude-<client> file exists, their complete pathnames are passed on to 1087 rsync using the --exclude-from=<file> command line argument. 1088 1089 Vault preparation moved from the vault class in vault.cc to rvm.cc, and 1090 rewritten to utilize the logger. (If a previous archive is deleted to 1091 make room for a new archive, I think that action should be logged.) 1092 1093 Fixed a bug in class timestamp that allowed a constant timestamp to 1094 change due to a misunderstanding of how localtime() worked. (The tm 1095 structure is now held as a static object in the class instead of as a 1096 link.) 1097 1098 Class execute modified to allow the user more control over stdio 1099 redirection in the child. 1100 1101 Introduced the to_string() template function to convert objects to a 1102 string, mostly used generating verbose error messages. 1103 1104 Removed all the functions in fs.cc that used const char* arguments. 1105 (New to C++, I didn't realize that char* would be automatically 1106 converted to std::string...) 1107 1108 2003-08-29: Alpha 0.00 1109 1110 Document designs, concepts, and implementation strategy, test prototype 1111 code necessary to complete details of the design. 1112 1113 -- Michael Peek <peek@tiem.utk.edu> Wed, 23 Nov 2005 14:53:05 -0500 1114 1115# vim:ts=3:shiftwidth=3:et: 1116