1New features with AN-2021-06-07: 2 3This is the first localization step for the schily source consolidation. Many 4programs now (hopefully) call gettext() for all strings that need localization. 5 6- The next step will include dgettext() calls for the libraries and the 7 missing programs 8 9- The following step will include the extracted strings 10 11- The last step will include German translations and install support 12 for the resulting binary message object files. 13 14----------> Please test and report compilation problems! <--------- 15 16***** NOTE: As mentioned since 2004, frontends to the tools should ***** 17***** call all programs in the "C" locale ***** 18***** by e.g. calling: LC_ALL=C cdrecord .... ***** 19***** unless these frontends support localized strings ***** 20***** used by the cdrtools with NLS support. ***** 21 22 *** WARNING *** 23 *** Need new smake *** 24 25 *** Due to the fact that schily-tools 2014-04-03 introduced to use new macro 26 *** expansions and a related bug fix in smake, you need a newer smake 27 *** to compile this source. If your smake is too old and aborts, ensure to 28 *** use the recent smake by calling: 29 30 cd ./psmake 31 ./MAKE-all 32 cd .. 33 psmake/smake 34 psmake/smake install 35 36 The new smake version mentioned above is smake-1.2.4 37 The recent smake version is smake-1.5 38 39 *** Due to the fact that schily-tools 2018-01-26 introduced 40 *** optimizations for the Schily version of SunPro Make, you 41 *** need at least the dmake version from 2018/01/11 with support 42 *** for the "export" directive to compile with this makefile system. 43 44For the beginning of the list of new features of the software in this tarball, 45please scroll down to "NEW FEATURES" 46 47 WARNING: the new version of the isoinfo program makes use of the 48 *at() series of functions that have been introduced by Sun 49 in August 2001 and added to POSIX.1-2008. For older platforms, 50 libschily now includes emulations for these functions but 51 these emulations have not yet been tested thoroughly. 52 Please report problems! 53 54 BUG WARNING: Please never report bugs only to Linux distributions as 55 they usually do not forward these bug reports upstream and 56 the Linux distributions typically do not let skilled people 57 check the bugs. We did not hear about a FIFO problem in star 58 for a long time. Then a problem on Linux occurred once 59 every 6000-10000 tries but it did not happen on Solaris after 60 even 10 million tries, so it was not known besides Linux and 61 not reported to the project. 62 63 BUG WARNING: *** GNU make *** starts too early with parallel 64 execution (when reading Makefiles and evaluating rules for 65 "include" statements already). Since GNU make does not 66 support a concept for a correct ordering of such actions, 67 you need to be prepared to see GNU make fail in parallel 68 mode. If you try to compile a maiden unpacked schilytools 69 tarball in parallel mode using GNU make, this will definitely 70 fail as a result of the GNU make timestamp caching bug. See 71 below for more information. 72 73 If you are interested in reliable parallel execution, 74 it is recommended to use the included "dmake" program with 75 a command line like: 76 77 dmake -j10 -r -f SMakefile 78 79 from the top level directory. Note that if you are on Linux, 80 you need the dmake version from schilytools 2021-06-07 or 81 newer, since that version introduced a solution for a kernel 82 caused performance problem with filesystems on Linux. Older 83 dmake versions will not be faster in parallel mode on Linux. 84 85 The "dmake" program included in the schilytools tarball is the 86 current version of the "new" SunOS make program that has been 87 introduced in January 1986 by Sun Microsystems. It also 88 introduced new features like the "include" directive that 89 3 years later have been copied by gmake in a partially buggy 90 way. As gmake does not fix showstopper bugs, it cannot be 91 supported. Current showstoppers are: 1) gmake executes 92 "include" related rules in the inverse order, causing rules 93 to fail if they depend on files created by an "earlier" action 94 2) gmake caches an outdated state of the directory and aborts 95 with a wrong complain about allegedly missing files that in 96 fact exist already, because they just have been remade. 97 98NEW FEATURES: 99 100- README.compile now explicitly mentions that using INS_BASE=xxx 101 requires a "make clean" followed by a recompile with INS_BASE=xxx to 102 make the change effective. 103 104 Thanks to Robert Clausecker for reporting. 105 106- COPYING: The license of the program "lndir" is now correctly 107 mentioned as "MIT licensed". Before, it incorrectly was reported 108 as "similar to BSD". 109 110 Thanks to Robert Clausecker for reporting. 111 112- RULES/MKLINKS: creates more ISA specific symlinks for FreeBSD 113 and ARM based platforms. 114 115- DEFAULTS/Defaults.freebsd now sets RUNPATH. Before, this was 116 commented out. 117 118 Thanks to Robert Clausecker for reporting. 119 120- RULES/*: the FreeBSD rules now pass RUNPATH to the linker. 121 This allows to compile schilytoos with LINKMODE=dynamic on 122 FreeBSD. 123 124 Thanks to Robert Clausecker for reporting. 125 126- RULES/rules.cnf: now always tries to update the configuration 127 test code in the directory "inc" that e.g. generates 128 129 align.h and avoffset.h 130 131 if the rules detect the need to automatically run "configure". 132 This permits to start compilation in a dedicated sub-directory 133 and to rely on the makefile system to automatically create all 134 primary dependendencies. 135 136 Note that there is no automatic creation of libraries that 137 may need to be compiled first. This is only handled automatically 138 when using the predefined compile order from the top level 139 directory. 140 141 Thanks to Robert Clausecker for reporting. 142 143- .links: The new FreeBSD links have been added to let them be 144 removed by calling: 145 146 ./.clean 147 148- RULES/rules.top: The NOECHO=@ has been moved more to the front of the 149 file to cover all rule commands. 150 151- makefiles.tar.bz2 was updated to include the above changes. 152 153- cpp: A new option -help (and an --help alias) has been implemented 154 to print a short usage information. 155 156- cpp: cpp -version (and --version) now prints a new version date to 157 give a hint to the changes. from above. 158 159- cpp: #ifdef linux and #ifdef __aarch64__ are now supported 160 161- cpp: The binary is now also installed as $INS_BASE/bin/krcpp 162 via a hardlink to lib/cpp (the historic location of the K&R cpp). 163 164 For manual use, it is now recommended to call: 165 166 krcpp ... 167 168- cpp: The man page now mentions $INS_BASE/bin/krcpp as well 169 170- cpp: The binary.../bin/krcpp is now also supported 171 by the CSW packaging 172 173- cpp: The man page krcpp.1 is now used by CSW packaging instead 174 of cpp.1. This makes it easier to avoid getting a GCC man page 175 of the same name. 176 177- man2html: This tool has been added to the packaging of the 178 CSW packet schilytools. 179 180- smake: the MAKEFLAGS parser has been enhanced to understand the 181 encoding used by the BSD make program "bmake". This is needed to 182 support something like "bmake INS_BASE=xxx" on FreeBSD while 183 compiling the schilytools. 184 185 This parser enhancement also covers some incompatibilities with 186 the MAKEFLAGS= values created by SunPro Make (e.g. a missing -- 187 before the make macro definitions are added). 188 189 Thanks to Robert Clausecker for reporting. 190 191- smake: The smake man page now also mentions SunPro make as dmake(1) 192 and make(1s) in the "SEE ALSO" section. 193 194- SunPro Make: A new hardlink .../bin/svr4.make is installed to make 195 it easier to get make "make" with SVr3 behavior. 196 197- SunPro Make: The binary.../bin/svr4.make is now also supported 198 by the CSW packaging 199 200- SunPro Make: The man page sysV-make.1 mentions the above change. 201 202- SunPro Make: The man page now also mentions smake(1) in the 203 "SEE ALSO" section. 204 205- SunPro Make: The word POSIX is now printed in bold to make it 206 easier to read the SunPro Make man page. 207 208- SunPro Make: Temporary output files, used for serializing stdout 209 and stderr with "make -j 2" or more parallel jobs are no longer 210 opened with O_DSYNC. 211 212 On platforms with bad filesystem performance, like Linux, using 213 O_DSYNC did cause a massive performance problem when running in 214 parallel mode on Linux. 215 216 Since using O_DSYNC just enforces a common view between the kernel 217 and the filesystem background storage and since the named temporary 218 files do not need to survive a reboot, there is no need to use O_DSYNC. 219 220 If you for whatever reason like to check whether using O_DSYNC 221 causes a difference, call: 222 223 cd sunpro 224 smake clean 225 smake COPTX=-DDO_DIRECTIO 226 227 to enforce the previous method. 228 229- SunPro Make: a new version date has been introduced for "make -version" 230 231- Bourne Shell: the man page now uses a correct notation for "[]" for the 232 arguments of the builtin "dosh" that has been introduced in May 2012. 233 234 235 236SCCS THOUGHTS: 237 238- SCCS: The current idea for converting a historic SCCS project into 239 a project oriented SCCS history bundle is the following: 240 241 - Create a user map file for "sccslog" by calling: 242 243 mkdir $HOME/.sccs 244 $EDITOR $HOME/.sccs/usermap 245 246 Enter the UNIX login names followed by a TAB, followed 247 by an E-mail notation. Use one line per user, e.g. 248 249 joerg J. Schilling <joerg@mail.com> 250 251 - Create a copy of the whole project to work on for this test. 252 Do not do this conversion on the original project until 253 sccs-6.0 is ready. 254 255 - chdir to the project home directory of the just created copy. 256 257 - Call "sccs init -i ." to make the project using an in-tree 258 project oriented repository. 259 260 - Call: 261 262 find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 - 263 264 for the CSRG BSD project use: 265 266 find * -path '*SCCS/s.*' | TZ=US/Pacific /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 - 267 268 to convert all history files into SCCSv6 history files. The 269 TZ=US/Pacific is important for the UCB conversion since SCCSv6 270 uses timezones but SCCSv4 does not and we need to have the 271 correct timezone entries in the SCCSv6 history files. 272 273 For the complete "schilytools" project with 4200 SCCS history 274 files in 55 Mbytes, this takes 12 seconds for the SCCS history 275 from 1984 .. 2020, but note that most of the edits from the 276 1980s are lost, so there are few entries from the time 277 before 1989. 278 279 An alternate example: the SCCS history from the BSD-4.4 project 280 from December 1979 up to June 1995 is in 12600 SCCS history 281 files that take up 125 MB. 282 The conversion time to the SCCSv6 history file format is 283 18 seconds. 284 285 - Call: 286 287 find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccslog -changeset - 288 289 to populate the changeset file from the existing deltas. 290 291 For the complete "schilytools" project with 19600 commits, 292 this takes 8 minutes. The resulting file .sccs/SCCS/s.changeset 293 has a size of approx. 7 MBytes. 294 295 An alternate example: the SCCS history from the BSD-4.4 project 296 from December 1979 up to June 1995 has approx. 47000 commits. 297 The conversion time is approx. 40 minutes. 298 The size of the resulting changeset file is approx. 14 MBytes. 299 300 - convert the in-tree repository into an off-tree repository. 301 This final step is not yet needed and there is currently no 302 code to do that automatically. 303 304 - If you like to check the resulting changeset file, there is 305 currently only one way to look at it, by calling: 306 307 sccs -O get -p -A -m .sccs/SCCS/s.changeset | more 308 309 This prints an annotated version of the changeset file. 310 The next task is to develop an enhancement to "sccs log" 311 that prints the changeset in a way similar to what "hg log -v" 312 prints. 313 314 - NOTE: Normal filesystems on Linux are slow, it is advised to 315 make the conversions on tmpfs for performance reasons in case 316 you are using Linux. 317 318 Please however keep in mind that this is still experimental and there is 319 absolutely no grant that a changelog created with current experimental 320 software will work correctly with the final SCCS version. The procedure 321 is just an example to check how it may look like. 322 323 The final conversion method will be more automated... most likely 324 by a command similar to "sccs import ..." 325 326 IMPORTANT: This is not yet the time to finally convert a project into 327 the project mode, because the project would be stuck in the current 328 state. What we need to continue work in that repository state in the 329 project mode is at least a working "sccs commit". Be prepared to remove 330 the changeset history file once "sccs commit" works and to re-create 331 the changeset file for that time. 332 333 334 335- SCCS TODO: 336 337 - Activate "fsdiff" as a "bdiff" replacement in delta(1) 338 to speed up delta(1) and to reduce the size of the SCCS 339 history files. 340 341 - Implement something that outputs similar information from 342 the changeset file as printed with "hg log -v". 343 344 This would be the next key feature. 345 346 - verify whether sccs.c uses -NSCCS in the back end programs 347 correctly, instead of converting g-file names from the command 348 line into s.file names in the frontend in order to forward 349 s.file names to the backend programs. This is needed for an 350 off-tree repository. 351 352 The related unit tests are already passed. 353 354 - Add code to to sccs(1) to send a list of files to admin(1) and 355 delta(1) with new or modified files in order to have all 356 important code for a "sccs commit" in a single program that 357 does not need to deal with ARG_MAX limitations. 358 359 - Add code to admin(1), delta(1), sccs-log(1) and get(1) to 360 maintain/understand the changeset file. 361 362 This is mainly writing out the sccschangeset(4) entries to an 363 intermediate store if a single file has been treated 364 successfully. For sccs-log(1), see below. 365 366 - Finish the work to allow normal line based diffs in SCCS even 367 for binary files. This are files that include nul bytes and 368 this needs to completely avoid fputs() and this needs an 369 initialized member p_line_length in struct packet even for 370 all content that does not result from a previous getline() call. 371 372 - sccs -R tell (and probably other subcommands?) does not yet 373 work in NewMode 374 375 - Add code to libcomobj to understand the changeset file. 376 This is needed in order to e.g. know the file names and file 377 specific SIDs/state that corresponds to a project global SID. 378 379 - Find/verify a complete transactional model that allows to repair 380 complex changes to the set of files for a project that have 381 been aborted in the middle. The current idea is to create the 382 file $PROJECTHOME/.sccs/changeset with the deltas to the 383 changeset during a complex update operation. 384 385 - Find a decision on how to deal with the admin flags that are 386 currently implemented as global flags and thus do not depend on 387 the SID (version) if the history file. 388 389 - Aborting a transaction via ^C currently requires a manual 390 removal of the global lock file. Find a way to avoid this in 391 case that a commit has been aborted while being prompted for 392 a commit message (which is before any real action happened). 393 394 - Implement a fully automated method to convert a SCCSv4 based 395 history with unrelated history files into a new SCCSv6 based 396 project mode history with a populated changeset history file. 397 398 This will most likely be done as a variant of the to be defined 399 new command "sccs sccsimport" that imports a whole existing old 400 SCCS project. 401 402 - Implement this "sccs sccsimport" based conversion in a way where 403 sccs(1) holds the global changeset lock for the whole time 404 of the conversion. 405 406 407 408 409- Bourne Shell Missing features for POSIX compliance: 410 411 - Support for $'...' quoting (this is not needed for the current 412 version of POSIX but for the next POSIX 413 version that will be named SUSv8). 414 The development of SUSv8 will start in 415 late 2016. 416 417 We are now expecting the Bourne Shell to be fully POSIX compliant. 418 419- Bourne Shell further TODO list: 420 421 - Finish loadable builtin support. 422 423 - POSIX does not allow us to implement ". -h", so we will 424 add a "source" builtin to be able to implement "source -h" 425 426- The following builtins (that are available in bsh) are still missing in 427 the Bourne Shell: 428 429 err echo with output going to stderr 430 glob echo with '\0' instead of ' ' between args 431 env a builtin version of /usr/bin/env 432 433 The following bsh intrinsics are still missing in the Bourne Shell: 434 435 - the restricted bsh has restriction features that 436 are missing in the Bourne shell. 437 438 - source -h read file into history but do not execute 439 440 and probably more features not yet identified to be bsh unique. 441 442 443 444Author: 445 446Joerg Schilling 447D-13353 Berlin 448Germany 449 450Email: joerg@schily.net 451 452Please mail bugs and suggestions to me. 453