From joseph at josephholsten.com Sun Sep 3 15:19:48 2023 From: joseph at josephholsten.com (Joseph Holsten) Date: Sat, 02 Sep 2023 22:19:48 -0700 Subject: [TUHS] Were any of the commercial unixes source-available? Message-ID: I’ve been playing around trying to link the OpenSolaris launch commit to various pieces in the Unix History Repository, and it’s making me wonder if we’ll ever have a chance to see the history of the systems. I’m less concerned about HPUX, AIX and SCO’s offerings since I presume someone has copy inside these companies. But what about A/UX, Irix, Tru64? Did these ever get sold with licenses to source tapes? Are there copies we need to preserve in-camera so something can exist 120 years after creation or whenever copyright expires? -- Joseph Holsten http://josephholsten.com mailto:joseph at josephholsten.com tel:+1-360-927-7234 From ron at ronnatalie.com Sun Sep 3 20:42:36 2023 From: ron at ronnatalie.com (Ron Natalie) Date: Sun, 03 Sep 2023 10:42:36 +0000 Subject: [TUHS] Were any of the commercial unixes source-available? In-Reply-To: References: Message-ID: Note that most UNIX was protected by Trade Secret rather than copyright so there’s no statutory expiration date. We were contractors for IBM so we had XENIX and AIX sources in our facility under NDAs, so I don’t believe IBM dealt with it other than OCO. I did have a license for the source code for Interactive Systems IS/1 but that was a fairly straight forward System V kernel. From andreww591 at gmail.com Sun Sep 3 21:25:54 2023 From: andreww591 at gmail.com (Andrew Warkentin) Date: Sun, 3 Sep 2023 05:25:54 -0600 Subject: [TUHS] Were any of the commercial unixes source-available? In-Reply-To: References: Message-ID: On 9/2/23, Joseph Holsten wrote: > I’ve been playing around trying to link the OpenSolaris launch commit to > various pieces in the Unix History Repository, and it’s making me wonder if > we’ll ever have a chance to see the history of the systems. > > I’m less concerned about HPUX, AIX and SCO’s offerings since I presume > someone has copy inside these companies. But what about A/UX, Irix, Tru64? > Did these ever get sold with licenses to source tapes? Are there copies we > need to preserve in-camera so something can exist 120 years after creation > or whenever copyright expires? > There are source leaks for quite a few commercial Unices floating around in various places. These are the ones I'm aware of: A/UX (0.7, which is complete, and 2.x, which is only the kernel) AIX 4.1.3 (most of the kernel and some of user space, possibly complete enough to build) BSD/OS (various versions, probably complete) DEC OSF/1 (1.0 and 2.0; these seem reasonably complete) DYNIX 3.x (several versions, possibly complete enough to build) DYNIX/PTX (4.x?; possibly complete enough to build) IRIX 6.5.5 (missing quite a few major packages and nowhere near complete enough to build) MIPS RISC/os 4.52 (possibly complete) SGI System V GL2-W3.7 (for the IRIS 3000 68K machines; probably complete) SunOS 4.1.3 (seems to be the complete base system) System V for the 3b2 (several 3.x versions, possibly complete) System V for the UNIX PC (3.51, possibly complete) System V/386 4.2 (possibly complete) ULTRIX-11 (at least 3.1) ULTRIX-32 (2.0, which has been confirmed to build by someone else, and 4.2, which is also fairly complete) I haven't looked at any of these in depth, so I'm not completely sure of the status of any of them except for A/UX and ULTRIX From marc.donner at gmail.com Sun Sep 3 22:40:59 2023 From: marc.donner at gmail.com (Marc Donner) Date: Sun, 3 Sep 2023 08:40:59 -0400 Subject: [TUHS] Were any of the commercial unixes source-available? In-Reply-To: References: Message-ID: My team at Morgan Stanley had a source license to SunOS in the early ‘90s. We tried to secure a license to AIX source but never succeeded. On Sun, Sep 3, 2023 at 7:26 AM Andrew Warkentin wrote: > On 9/2/23, Joseph Holsten wrote: > > I’ve been playing around trying to link the OpenSolaris launch commit to > > various pieces in the Unix History Repository, and it’s making me wonder > if > > we’ll ever have a chance to see the history of the systems. > > > > I’m less concerned about HPUX, AIX and SCO’s offerings since I presume > > someone has copy inside these companies. But what about A/UX, Irix, > Tru64? > > Did these ever get sold with licenses to source tapes? Are there copies > we > > need to preserve in-camera so something can exist 120 years after > creation > > or whenever copyright expires? > > > There are source leaks for quite a few commercial Unices floating > around in various places. These are the ones I'm aware of: > > A/UX (0.7, which is complete, and 2.x, which is only the kernel) > AIX 4.1.3 (most of the kernel and some of user space, possibly > complete enough to build) > BSD/OS (various versions, probably complete) > DEC OSF/1 (1.0 and 2.0; these seem reasonably complete) > DYNIX 3.x (several versions, possibly complete enough to build) > DYNIX/PTX (4.x?; possibly complete enough to build) > IRIX 6.5.5 (missing quite a few major packages and nowhere near > complete enough to build) > MIPS RISC/os 4.52 (possibly complete) > SGI System V GL2-W3.7 (for the IRIS 3000 68K machines; probably complete) > SunOS 4.1.3 (seems to be the complete base system) > System V for the 3b2 (several 3.x versions, possibly complete) > System V for the UNIX PC (3.51, possibly complete) > System V/386 4.2 (possibly complete) > ULTRIX-11 (at least 3.1) > ULTRIX-32 (2.0, which has been confirmed to build by someone else, and > 4.2, which is also fairly complete) > > I haven't looked at any of these in depth, so I'm not completely sure > of the status of any of them except for A/UX and ULTRIX > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rawallis at panix.com Mon Sep 4 01:09:50 2023 From: rawallis at panix.com (Andy Wallis) Date: Sun, 03 Sep 2023 11:09:50 -0400 Subject: [TUHS] Were any of the commercial unixes source-available? In-Reply-To: References: Message-ID: <3bca93296e95eb1e53fcc6408246678f@panix.com> On 2023-09-03 08:40, Marc Donner wrote: > My team at Morgan Stanley had a source license to SunOS in the early > ‘90s. We tried to secure a license to AIX source but never > succeeded. We had access to the AIX source at my work place. The AIX 3.2.5 source delivery included a build machine because we had requirements to be able to support and redistribute AIX 3.2.5 for up to 20 years. As I recall, it required a lot of high level negotiations between us and IBM to get that. Access was limited to select people on an air-gapped machine. The source code itself was locked away on 8mm tape and required signatures to check it out. IBM did have a public announcement letter for AIX 3.2.5 source access. http://bio.gsi.de/DOCS/AIX/ENUSZP93-0158.printable.html We also had access to the various AIX versions that we had to support our government customers. I know we had access to AIX 4.3.3, 5.1,5,2, and 5.3 source code. One of our developers would fix bugs that he found and file PMRs to tell the AIX developers what exactly to fix. A strip command change to support very large executables took 18 months for them to fix because they didn't understand why failing to strip very large binaries was a problem. -Andy Wallis From imp at bsdimp.com Mon Sep 4 01:48:43 2023 From: imp at bsdimp.com (Warner Losh) Date: Sun, 3 Sep 2023 09:48:43 -0600 Subject: [TUHS] Were any of the commercial unixes source-available? In-Reply-To: References: Message-ID: On Sun, Sep 3, 2023, 4:42 AM Ron Natalie wrote: > Note that most UNIX was protected by Trade Secret rather than copyright > so there’s no statutory expiration date. > But only so long as they remain secret... there are a few leaks of early stuff in bitsavers... the leader broke the law, perhaps, but once the secret is out there, its no longer secret. See the 32V preliminary rulings for variations on that theme. System V and newer did have copyright protection as well as trade secret contract stuff... It isn't hard to find at least one copy of most of the major and many minor players. It's fair use to copy for study and commentary. But building a full system for commercial benefit is not. We were contractors for IBM so we had XENIX and AIX sources in our > facility under NDAs, so I don’t believe IBM dealt with it other than > OCO. > I did have a license for the source code for Interactive Systems IS/1 > but that was a fairly straight forward System V kernel. > Several variations on that are out there... What isn't out there are the full source code control trees. At least Sun distributed them. But so far only a symlink farm for that has been found. Warner > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Mon Sep 4 01:59:47 2023 From: imp at bsdimp.com (Warner Losh) Date: Sun, 3 Sep 2023 09:59:47 -0600 Subject: [TUHS] Were any of the commercial unixes source-available? In-Reply-To: References: Message-ID: On Sun, Sep 3, 2023 at 5:26 AM Andrew Warkentin wrote: > On 9/2/23, Joseph Holsten wrote: > > I’ve been playing around trying to link the OpenSolaris launch commit to > > various pieces in the Unix History Repository, and it’s making me wonder > if > > we’ll ever have a chance to see the history of the systems. > > > > I’m less concerned about HPUX, AIX and SCO’s offerings since I presume > > someone has copy inside these companies. But what about A/UX, Irix, > Tru64? > > Did these ever get sold with licenses to source tapes? Are there copies > we > > need to preserve in-camera so something can exist 120 years after > creation > > or whenever copyright expires? > > > There are source leaks for quite a few commercial Unices floating > around in various places. These are the ones I'm aware of: > > A/UX (0.7, which is complete, and 2.x, which is only the kernel) > AIX 4.1.3 (most of the kernel and some of user space, possibly > complete enough to build) > BSD/OS (various versions, probably complete) > DEC OSF/1 (1.0 and 2.0; these seem reasonably complete) > DYNIX 3.x (several versions, possibly complete enough to build) > DYNIX/PTX (4.x?; possibly complete enough to build) > IRIX 6.5.5 (missing quite a few major packages and nowhere near > complete enough to build) > MIPS RISC/os 4.52 (possibly complete) > SGI System V GL2-W3.7 (for the IRIS 3000 68K machines; probably complete) > SunOS 4.1.3 (seems to be the complete base system) > There is (or was) a github repo that has Solaris 2.5, 2.6, 8 and 11 sources. System III sources are out there, lots of copies, but they all appear to come from the same root source and have just been repackaged. > System V for the 3b2 (several 3.x versions, possibly complete) > System V for the UNIX PC (3.51, possibly complete) > System V/386 4.2 (possibly complete) > Bitsavers also has sources for the IS/1 or similar to a number of different 68k machines, but it's only the kernel. It's V7 based, but with bits of system III and system V tossed in. Also, it's not hard to find SystemVr1 (for VAX and PDP-11), r2, r3 (several) and r4 online. > ULTRIX-11 (at least 3.1) > This is buildable, and legit with permission from DEC. At least parts of it are buildable, I've not tried to do a full system gen from sources. > ULTRIX-32 (2.0, which has been confirmed to build by someone else, and > 4.2, which is also fairly complete) > > I haven't looked at any of these in depth, so I'm not completely sure > of the status of any of them except for A/UX and ULTRIX > hpux has a lot of docs out there. And a lot of binaries, but little source. It's one of the few I've not found sources for when I was doing research on how sync behaved, for example. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcus.e.clark at outlook.com Mon Sep 4 02:21:32 2023 From: marcus.e.clark at outlook.com (Marcus Clark) Date: Sun, 3 Sep 2023 16:21:32 +0000 Subject: [TUHS] Source code for Research Unix Version 7 checkers game Message-ID: Research Unix Version 7 has a checkers game ( /usr/games/checkers). I can't find the source code for the game, does anyone know where I can find it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Mon Sep 4 19:57:48 2023 From: tuhs at tuhs.org (Paul Ruizendaal via TUHS) Date: Mon, 4 Sep 2023 11:57:48 +0200 Subject: [TUHS] Unix install & "standalone" package Message-ID: Recently, I was looking into the “Das U-Boot” boot loader package. Summarised with great simplification, u-boot bundles device drivers, file systems, commands and a Bourne-like shell into a standalone package. Normally it auto-runs a script that brings up a system, but when used in interactive mode it allows a great deal of poking around. It made me think of the “standalone” set of programs for installing early Unix. On 16-bit understandably each basic command has to be a separate standalone program, but after the shift to 32-bit bundling more functionality in a single binary would have become possible. How did the Unix “standalone” package evolve in the 80’s, both in the research and BSD lineages? Is there any retrospective paper about that? Or is it a case of “Use the source, Luke”? From norman at oclsc.org Tue Sep 5 00:44:21 2023 From: norman at oclsc.org (Norman Wilson) Date: Mon, 4 Sep 2023 10:44:21 -0400 (EDT) Subject: [TUHS] Unix install & "standalone" package Message-ID: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> I don't remember any special many-programs-in-one binary like busybox in any Unix from the days when Unix was simple enough for me to understand. That covers the entire lifetime of the Research systems, but also System V and the BSDs and their sundry offspring up into at least the 1990s. I'm pretty sure OpenBSD at least still has nothing like busybox. The nearest thing was to make sure certain programs were linked statically (or existed in alternate statically- linked versions) so they would work before shared libraries were available. (It seems to be common wisdom that `sbin' means `system bin' these days, but I remember once, long ago, being told it stood for `static bin'.) Perhaps the question to ask is why such a magic program is needed at all. Is it just because programs like the shell have become so large and unwieldy that they won't fit in a small environment suitable for loading into an initramfs? Older UNIXes, even on the VAX, didn't use an initramfs. the boot code had just enough understanding of devices and file systems to load the kernel and to point it at the real root file system. The VAX hardware designers had a clever scheme on many (but, strangely, not all) VAX variants by which the hardware had several little boot ROMs, each containing a bare-bones read-only device driver for a particular device along with a few instructions to read the first sector into memory and start it, with pointers to the boot-rom driver and device ID in specified registers. That was enough to support a device-independent Unix boot block that could read unix (or another file name typed on the console) from the root of the file system at the start of the disk. I know it was because I wrote such a boot block, though I don't know whether anyone else did. (Other systems, I think, just used it to load /boot, a larger and more-capable program.) Maybe it was just that the boot environment was simpler in older systems, without the need to load kernel modules or support multiple locations and means of access for the root? Norman Wilson Toronto ON From emu at e-bbes.com Tue Sep 5 00:53:09 2023 From: emu at e-bbes.com (emanuel stiebler) Date: Mon, 4 Sep 2023 10:53:09 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: Message-ID: <7300cbcd-c88a-781e-2cf2-8010298570a6@e-bbes.com> On 2023-09-04 05:57, Paul Ruizendaal via TUHS wrote: > > Recently, I was looking into the “Das U-Boot” boot loader package. Summarised with great simplification, u-boot bundles device drivers, file systems, commands and a Bourne-like shell into a standalone package. Normally it auto-runs a script that brings up a system, but when used in interactive mode it allows a great deal of poking around. U-Boot has his roots in embedded systems, where you have a nice environment to debug your system. You have already everything there to download files with s-record from serial, boot from tftp, etc. Actually also setting up the memory controllers, timing, setting environment variables ... This became so comfortable, that sometimes the U-Boot code is larger than the target system. So, after development is done, the system jumps into the real application or code ... From katolaz at freaknet.org Tue Sep 5 00:55:53 2023 From: katolaz at freaknet.org (Vincenzo Nicosia) Date: Mon, 4 Sep 2023 14:55:53 +0000 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: rather, back then you didn't need the same kernel to run on a wide variety of hardware, with all the possible different combinations of peripherals, requiring all sorts of different drivers. I think that's the only real reason why initramfs came to existence: allowing a selection of kernel modules to be loaded at init time, depending on the hardware at disposal on that machine. Then things went south, and more recent initramfs have everything and the kitchen sink. But that's another story. HND Enzo Nicosia -- From imp at bsdimp.com Tue Sep 5 03:07:46 2023 From: imp at bsdimp.com (Warner Losh) Date: Mon, 4 Sep 2023 11:07:46 -0600 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: Message-ID: On Mon, Sep 4, 2023 at 3:58 AM Paul Ruizendaal via TUHS wrote: > > Recently, I was looking into the “Das U-Boot” boot loader package. > Summarised with great simplification, u-boot bundles device drivers, file > systems, commands and a Bourne-like shell into a standalone package. > Normally it auto-runs a script that brings up a system, but when used in > interactive mode it allows a great deal of poking around. > > It made me think of the “standalone” set of programs for installing early > Unix. On 16-bit understandably each basic command has to be a separate > standalone program, but after the shift to 32-bit bundling more > functionality in a single binary would have become possible. > > How did the Unix “standalone” package evolve in the 80’s, both in the > research and BSD lineages? Is there any retrospective paper about that? Or > is it a case of “Use the source, Luke”? > The stand package continued in research and BSD to be those programs needed to install and/or recover badly damaged systems. You could create a new file system, copy a file from the tape to a partition, etc. You couldn't do general scripting with this, by and large. Originally, they were tape programs. This made sense because of its original focus. In time, some systems could load the stand alone programs instead of the kernel, but they continued the original focus. This is, imho, due in large part due to the miniroot. The miniroot evolved into both a full-enough system to do the installation scripts in shell instead of C (Venix, at least, had their install program written in C). You'd copy the minroot to swap and then install the system. But a number of additional programs were placed into the miniroot so you could do some limited filesystem repair, file editing, etc. In addition, many vendor's ROMs grew in complexity. Solbourne's ROMs, for example, could do basic repair of UFS (clri level, not fsck level), and copy files from one place to another. I often recovered a Solbourne system I screwed up by attaching an external SCSI drive that had a known good kernel, init, etc. The 'stand' environment was a whole set of tools that could be used to build stand-alone programs that shared much code of their full unix brethren, despite not having a full kernel under them. Kernel services were provided by different libraries that did filesystem things, block driver things, network things, etc in a similar way to Unix, but with a much reduced footprint. initramfs, as has been mentioned elsewhere, is pretty much a Linux invention. It was designed to 'punt' on the choose where to load things from and have a very minimal interface between the boot loader and the system. In time, it grew to support more interfaces, more ways of loading, and better ways to mount something that you could then 'pivot' onto. Few other unix systems went this route, though many adopted some variation on the pivot_root functionality. Linux has moved beyond the pivot root after having booted the correct kernel into being able to take over the machine early in, say, UEFI startup with a minimal kernel and initramfs that just knows how to load the next kernel. They skipped the complex boot loader stage, and went straight to the 'run linux earlier' stage which is how things like LinuxBoot, coreboot and others have put the boot logic into bash scripts. The ability to 'kexec' a kernel and replace the current running kernel originated in the 'non-stop' world that wanted to reduce downtime. Now, it's used to reduce firmware complexity by eliminating large swaths of UEFI from the boot process, but also generalizes in the embedded space. FreeBSD, from around FreeBSD 2 (1995 or so), had /rescue which largely took over form the stand alone environment for the repair duties of things. FreeBSD also adopted a more complex boot loader that would load the kernel, modules, set tunables, etc prior to kicking off the kernel. Between /rescue having all the tools needed to repair bad updates, repair failing disks, get that one last backup before the drive is dead while you wait for the new drive to be delivered, etc, and /boot/loader being able to script loading the kernel while the BIOS was still around so the need for drivers in the loader was lessened. However, as the BIOS evolved into UEFI and FreeBSD pushed into the embedded space whose firmware provided a less rich environment to the boot loader, so it was able to load things off fewer and fewer devices, it became clear that it would need a pivot root feature to allow it to boot all the way into FreeBSD, load some drivers from an included ram disk, and then use that mount a new root and then 'reroot' to that by killing everything and running init from that new root. FreeBSD also moved from Forth to Lua in its scripting language for the boot loader, giving 'pre boot' environment support better features. I also added the ability to use FreeBSD boot loader as a Linux binary to load FreeBSD and its metadata from a LinuxBoot environment. Finally, FreeBSD has 'spun out' and generalized the /rescue feature to allow creation of any 'BeastyBox' environment, similar to what you get in a busy box, or clone, environment. This environment, though, is meant in large part on both Linux and FreeBSD to be in constrained environments where a full install is prohibitive (even those that never pivot to something more, like ap, routers and nas boxes). NetBSD retains many of the old BSD stand-alone programs that started on the vax. I've not studied things beyond noticing this. OpenBSD is similar. Their boot chain is a bit simpler than FreeBSD's, though there's noises about porting FreeBSD's boot there. There's a port of /boot/loader to illumos too, but I don't know if it is the default, or just available. So I'll not chat about it more. So the original 'standalone' environment where you had one program running on a system has evolved into either a rich boot loader environment that lets one do a lot to decide what kernel to load, or towards having a minimal selection of unix programs faster and using /bin/sh or similar to do scripting. These reduced environments are often called standalone, though all they share just the name with the earlier 'stand' programs: they are full unix programs, but with reduced feature sets and 'linker magic' to package them in a way that's faster, smaller, etc (eg all in one binary). FreeBSD's boot loader is an outgrowth of the original standalone env, by way of a port of NetBSD's libsa. I suspect in the future, we'll see more and more of a trend for low-level init and then handing off to some built-in kernel (be it Linux, BSD-based (there's now kexec), or whatever) to reuse more of the vetted code rather than re-inventing Unix inside the boot loader (which is a valid criticism of FreeBSD's boot loader, though it's rich feature set is what you get for the complexity). Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually do the research on the areas I was hand-wavy about? Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Tue Sep 5 03:18:18 2023 From: imp at bsdimp.com (Warner Losh) Date: Mon, 4 Sep 2023 11:18:18 -0600 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: On Mon, Sep 4, 2023 at 8:44 AM Norman Wilson wrote: > Maybe it was just that the boot environment was simpler > in older systems, without the need to load kernel modules > or support multiple locations and means of access for > the root? > Older systems had fewer choices. On the PDP-11 you had Q-Bus or Unibus. You booted off a small selection of disks that converged to MSCP(?), so there was THE boot environment and THE driver and THE filesystem. And the kernels were often tuned to be exactly what the machine needed. The VAX, Sun, hp, ibm, etc continued this early trend. Though the boot process was still fairly narrow, with a limited list of supported boot devices. In large part this was because the machine was exactly the same every time you booted. But with USB, PC Card, CardBus, ExpressCard, SCSI, Thunderbolt, and a host of other removable technologies with a dizzying array of cards, like PCI, ISA, etc, the game changed. Also, the number of environments that a FreeBSD GENERIC[*] kernel can boot in is huge due to combinatoric explosion due to three or four boot environments, vm alternate startup paths, several supported root file system, BIOS, UEFI, OpenFirmware, u-boot, coreboot, etc as well as several thousand supported boot controller devices, things got complex, despite there being several common interfaces that made things simpler. I'm running a summer of code project this year to help tame the combinatoric explosion to provide better test coverage because though attempts were made to make things the same, variations exist that can cause unexpected breakage in different environments. Warner [*] I use GENERIC here as a catch all, including the more recent MINIMAL kernels that try to include just the core functionality, and omit the vast majority of drivers. Linux has similar issues, but more, and also solves them in many interesting ways... But I'm the FreeBSD boot loader guy in large part (though there's others that work on it), not a Linux person. -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Tue Sep 5 03:20:31 2023 From: imp at bsdimp.com (Warner Losh) Date: Mon, 4 Sep 2023 11:20:31 -0600 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: On Mon, Sep 4, 2023 at 8:56 AM Vincenzo Nicosia wrote: > rather, back then you didn't need the same kernel to run on a wide > variety of hardware, with all the possible different combinations of > peripherals, requiring all sorts of different drivers. > > I think that's the only real reason why initramfs came to existence: > allowing a selection of kernel modules to be loaded at init time, > depending on the hardware at disposal on that machine. Then things went > south, and more recent initramfs have everything and the kitchen sink. > But that's another story. Yea, it was an effort to move mounting of root out of the kernel. The earliest scripts just mounted the right disk and moved on, and didn't load any new drivers: they just had the logic to pick the desired root. But at the same time, there were a lot of people that were running on 4MB and 8MB systems that noticed they could put all the router software in the initramfs and never pivot to something else and they could have quite the product with that. And those were the first few bricks that paved the road to hell :) Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Tue Sep 5 04:21:21 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 4 Sep 2023 14:21:21 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: Message-ID: On Mon, Sep 4, 2023 at 1:08 PM Warner Losh wrote: > [snip] > There's a port of /boot/loader to illumos too, but I don't know > if it is the default, or just available. So I'll not chat about it more. I can confirm that it is the default on i86pc, though not universally. At Oxide, for example, we boot directly into a very small loader held in flash that has a compressed cpio archive containing the kernel and a few necessary kernel modules compiled into it (as a byte blob); we uncompress that blob into physical memory, locate and load the kernel like a normal ELF binary, and jump to the kernel's ELF entry point, passing a few basic arguments (notably, the location and length of the cpio archive in physical memory). Entry to the kernel is thus in 64-bit mode with paging enabled. > So the original 'standalone' environment where you had one program running on a system has evolved > into either a rich boot loader environment that lets one do a lot to decide what kernel to load, or towards > having a minimal selection of unix programs faster and using /bin/sh or similar to do scripting. These > reduced environments are often called standalone, though all they share just the name with the earlier > 'stand' programs: they are full unix programs, but with reduced feature sets and 'linker magic' to package > them in a way that's faster, smaller, etc (eg all in one binary). FreeBSD's boot loader is an outgrowth > of the original standalone env, by way of a port of NetBSD's libsa. > > I suspect in the future, we'll see more and more of a trend for low-level init and then handing off to some > built-in kernel (be it Linux, BSD-based (there's now kexec), or whatever) to reuse more of the vetted code > rather than re-inventing Unix inside the boot loader (which is a valid criticism of FreeBSD's boot loader, > though it's rich feature set is what you get for the complexity). > > Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually > do the research on the areas I was hand-wavy about? That would be interesting. I can still remember booting IBM 6150 RTs into a miniroot environment and using that to create and initialize filesystems when installing AOS (4.3BSD for the RT) back in the day. To my mind, the standalone programs were always oriented towards solving the related problems of bootstrap initialization onto a fresh machine, and disaster recovery when things were really, really messed up. As I recall, the RT miniroot could either load from tape, or one could `dd` it into swap and boot from that. In either case, I seem to recall it was copied into memory and run as a RAM disk. The idea of busy-box like "a bunch of utilities compiled into the same binary" was to save space, particularly since this would be copied into RAM; even with demand paging, redundant copies of bits from `libc` in each binary were a waste for what was intended to be a minimal environment anyway. - Dan C. From clemc at ccc.com Tue Sep 5 05:05:25 2023 From: clemc at ccc.com (Clem Cole) Date: Mon, 4 Sep 2023 15:05:25 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: I can not add too much to what happened later, but I think modern users don't get it why the things like the standalone system were needed back in the day. There were a few simple differences thanks to Moore's law, modern folks really may not grok: 1. The number of peripherals that needed to be supported was small [each firm made only a few for each target system] 2. The capacity/capabilities of those peripherals are minimal compared to today's world 3. Cheap ROMs, as we know it, were coming on the scene [the EPROM is only invented in 1971] One-time programmable and Mask ROMs were around but they were expensive and small. I hope Warren's emailer lets this pass through. Here is a picture of the original root 'ROM' for the PDP-11 M792-xx board, which gives you 32 words (64 bytes) of memory. It used individual diodes in or out arranged an array to represent individual bits with a diode present being a '1' and '0' if missing. As the Gunkies web page suggests, the board came in two versions - customer programmable which was fully populated with diodes, and the user removed unneeded diodes to program it. The other variants were programmed with the designed boot functions - designated M792-Yx (where 'x' is a capital letter, starting with 'A') [image: Read_Only_Memory_DEC_M792_Diode_Matrix_ROM_Aceware_Attribution.jpg] Also, remember that a $3-5k/drive RK05 is only 2.5Mbytes [4872 512 byte 'sectors' *a.k.a*. disk blocks -- 2 hd/disk * 15 sec/cyl * 203 cyl/disk], which is why we had separated /bin and /usr/bin and /lib and /usr/lib. Plus, as Norman points our dynamic linking only comes later, so all of these are static bound programs (*i.e.,* Sun create /bin and /sbin to separate those programs). The whole idea was what was in root was enough to get the system running in /bin, /lib and /etc and everything else was one the mounted file system, which often was another RK05. Many of our V6 and V7 systems ran with 3 RK05 /, /usr /home, and we might have an extra RK05 for /mnt. Tape was the standard thing. DEC tape was cheaper than 9-track, although in my history, we all went for 9-track because it was more portable to other systems, but often used after-market 9-track transports that emulated the DEC ones. Even with the VAX, the front-end runs on an LSI-11 with floppies, so things like microcode were loaded into the VAX and stored in ROM. By then, the RP and RM series had become more normal, and eventually, the RK07 and RL02 replaced the RK05. But we are not talking about substantial capacity in the storage systems. Only ten years later, in the early 1980s, a Fujisu 'Eagle' is just 470Mbytes and costs 12-15K per drive, plus another $6-8K for the controller for your Vax [also often after-market from SI or Emulex]. Sun and Apollo build 'diskless' systems because a 100MB ST-506 style disk was considered too expensive, and was worried about the entry-level price of their systems (I famously made a typo describing how those systems performed BTW with a dyslexic change of the s to c on a whole company email at Masscomp in the mid-1980s - to Sun's credit, it was the best marketing ploy ever - people bought diskless and discovered they sucked and then all added a disk - which cost more $s than the original Masscomp system which was had one built-in). But I digress... Those new workstations did have real ROM chips, unlike the original PDP-11s, but they were small, maybe 8Kbytes of NS2764, so the ROMs were tight and only had enough in them to recognize a couple of peripherals. So to answer more of your question, you build the tools you need that make sense at the time. /stand was a simple solution and was small. For V7 and the original BSD 3 and 4 released, the user needed to read a distribution media, traditionally a 9-track tape, and set up a local disk. The number of peripherals was small. One other thought ... in the case of the PDP-11 and Vaxen, the disks were often dismountable, like the different RK, RL, RP, and RM series drives. We often had more than one system. So once the system was up and running, it was not usual to use a different system with better tools to help repair things if bad stuff occurred. That is also why we often used 9-track tape, just because that was even easier to move back and forth. With the Wokstrations, the disks were generally sealed. Clem A small PS on another note -- I have memories of taking an ST-506 disk with me to France to work with some engineers in Grenoble and explaining to the French authorities that they could not open it up to look inside. BTW: this pre-internet - imagine what we had to do if we flew to Canada, which was taxing SW. Even bringing tapes in and out of Toronto airport could be treacherous. ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Read_Only_Memory_DEC_M792_Diode_Matrix_ROM_Aceware_Attribution.jpg Type: image/jpeg Size: 154411 bytes Desc: not available URL: From tytso at mit.edu Tue Sep 5 05:59:18 2023 From: tytso at mit.edu (Theodore Ts'o) Date: Mon, 4 Sep 2023 15:59:18 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: <20230904195918.GC701295@mit.edu> On Mon, Sep 04, 2023 at 11:20:31AM -0600, Warner Losh wrote: > > Yea, it was an effort to move mounting of root out of the > kernel. The earliest scripts just mounted the right disk and moved > on, and didn't load any new drivers: they just had the logic to pick > the desired root. But at the same time, there were a lot of people > that were running on 4MB and 8MB systems that noticed they could put > all the router software in the initramfs and never pivot to > something else and they could have quite the product with that. And > those were the first few bricks that paved the road to hell :) The other reason why you might need something like an initial ramdisk is if you don't yet *have* a root file system on the hardware, and you are trying to install a system using TFTP boot. At that point, you have three choices: (a) Use a NFS root (b) Use a read-only network block device (for example, MIT Project Athena had a read-only Remote Virtual Disk which was a network block device that was used to provide a read-only system image used for installation and update, as well as a read-only /usr image). (c) Use a built-in ramdisk. (And here's not just Linus which has adopted this strategy; OpenBSD's installer does this as well.) The other dynamic at play was that ramdisks were needed when you were installing on a 386 PC with, say, 16 megs of memory, a 320 megabyte hard disk, and a *single* 1.44MB floppy --- and no ethernet, because this was for the home PC user, so the best that you had might be a 38kbps dialup link with PPP --- if you were lucky. So putting the kernel on the first floppy disk, and putting a ramdisk image on the second floppy disk, so you could then eject the floppy disk and insert subsequent floppy disks so you could install rest of the installation binaries (including the shell utilities, the C compiler, X windows systems, etc.) and then reboot onto the HDD with a fully set up system, was a pretty natural evolution. And since we had the ramdisk infrastructure, and most users didn't want to configure their own kernel as part of the installation process, that begat kernel modules and the initial ramdisk being used to store the kernel modules. In the very early days, this made Linux *far* more user friendly for non-system programmers to install, compared to say, FreeBSD during that era, which was still stuck in the BSD 4.x days of a generic kernel, followed by a kernel configuration step, etc. These days, given that some enterprise setups what their servers to be installed from a Fibre Channel Storage Area Network, or iSCSI, perhaps using Kerberos to authenticate to the iSCSI target, it makes absolutely a huge amount of sense to have an initial ramdisk as an option. That being said, for a long time, the initial ramdisk was an *option*. You didn't have to use it, if you were willing to have a custom kernel and you are using a device which has a fixed and stable boot-time enumeration. This tended to only be used by people who knew what they were doing, but for example, my file system test appliance VM system doesn't use an initial ramdisk at all. It uses qemu, and has a built-in kernel configuration and build system, and this works perfectly fine booting into the latest Debian stable distribution, with no initial ramdisk used at all: https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md - Ted the root filesystem on the second floppy disk (the first floppy disk contained the kernel), which was then copied into a ramdisk, so you could Trying to fit the kernel and the root file system on a single 1.44MB floppy, was (barely) doable, but you certainly wouldn't have space for a C compiler so you build or link your own custom kernel. So one of the first things that we did was to put the kernel on a single floppy disk, and then but the root on a single floppy disk, and then copy the root to the HDD, and then reboot onto the HDD, and then take the installation procedure from there. But that was awkward and From steffen at sdaoden.eu Tue Sep 5 08:10:59 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Tue, 05 Sep 2023 00:10:59 +0200 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: <20230904221059.sF2G0%steffen@sdaoden.eu> Norman Wilson wrote in <9A989054DE79CE5059CBA74797391E39.for-standards-violators at oclsc.org>: |I don't remember any special many-programs-in-one binary |like busybox in any Unix from the days when Unix was simple |enough for me to understand. That covers the entire lifetime |of the Research systems, but also System V and the BSDs and |their sundry offspring up into at least the 1990s. ... |Perhaps the question to ask is why such a magic program is |needed at all. Is it just because programs like the shell |have become so large and unwieldy that they won't fit in |a small environment suitable for loading into an initramfs? AlpineLinux as used on my vserver has busybox by default and can cover most utitilities like that. The lead developer Copa once said something like "The idea is you install explicitly [if you want something better]". (It is a symlink farm that is selectively replaced by installing "real" packages iirc.) For my laptop it allows me easy boot management. To save you the chatter ("Chatten" is the name of my tribe .. most likely; could be Franken, Sueben .. and you know how it is): this approach is much easier and smaller than having lots of static binaries to copy around etc. I do not use secure boot, i have on EFI only a kernel, busybox and cryptsetup, and scripts (the laptop is named "kent"):: ... drwxr-xr-x 4 root root 4096 Jul 15 2021 EFI/ ... -rwxr-xr-x 1 root root 272 Feb 1 2022 kent.sh* -rwxr-xr-x 1 root root 313 Feb 1 2022 kent-direct.sh* drwxr-xr-x 1 root root 252 Oct 9 2022 ../ -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* So kent.sh can be init(8) for the ideapad-stage1.efi Linux kernel started via EFI as setup via efibootmgr(8) Boot0001* kent HD(1,GPT,5d6d756b-5de2-4e5d-b043-8d4ae1bb6eb0,0x800,0x82000)/File(\ideapad-stage1.efi)root=/dev/nvme0n1p1 rootfstype=vfat init=/kent.sh #!/busybox.static sh #@ kent, step 1., via EFI. PART_ROOT=/dev/nvme0n1p8 ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' INIT_S2=/boot/kent-2.sh . /linux-init-s1.sh and that allows me to unlock the harddisk. We then boot via $INIT_S2 and kexec(8) a kernel from the encrypted harddisk, so no code from EFI partition keeps on running. (We byte-compare the data from EFI with equal /boot/ files after booting the real system.) This allows nice and easy properties: only three files to track (cryptsetup, busybox, kernel), almost same set of files in /boot/ and /media/efi aka EFI. And ideapad-stage1.efi is the same kernel that later runs, but later we have also additional dynamic modules available. Ie, every few weeks i copy /boot/ideapad-6_1.efi over to be the new -stage1.efi. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From imp at bsdimp.com Tue Sep 5 09:51:33 2023 From: imp at bsdimp.com (Warner Losh) Date: Mon, 4 Sep 2023 17:51:33 -0600 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230904195918.GC701295@mit.edu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904195918.GC701295@mit.edu> Message-ID: On Mon, Sep 4, 2023 at 1:59 PM Theodore Ts'o wrote: > On Mon, Sep 04, 2023 at 11:20:31AM -0600, Warner Losh wrote: > > > > Yea, it was an effort to move mounting of root out of the > > kernel. The earliest scripts just mounted the right disk and moved > > on, and didn't load any new drivers: they just had the logic to pick > > the desired root. But at the same time, there were a lot of people > > that were running on 4MB and 8MB systems that noticed they could put > > all the router software in the initramfs and never pivot to > > something else and they could have quite the product with that. And > > those were the first few bricks that paved the road to hell :) > > The other reason why you might need something like an initial ramdisk > is if you don't yet *have* a root file system on the hardware, and you > are trying to install a system using TFTP boot. At that point, you > have three choices: > > (a) Use a NFS root > > (b) Use a read-only network block device (for example, MIT Project > Athena had a read-only Remote Virtual Disk which was a network block > device that was used to provide a read-only system image used for > installation and update, as well as a read-only /usr image). > > (c) Use a built-in ramdisk. (And here's not just Linus which has > adopted this strategy; OpenBSD's installer does this as well.) > Yes. FreeBSD did this originally, and some installers still do that. > The other dynamic at play was that ramdisks were needed when you were > installing on a 386 PC with, say, 16 megs of memory, a 320 megabyte > hard disk, and a *single* 1.44MB floppy --- and no ethernet, because > this was for the home PC user, so the best that you had might be a > 38kbps dialup link with PPP --- if you were lucky. > > So putting the kernel on the first floppy disk, and putting a ramdisk > image on the second floppy disk, so you could then eject the floppy > disk and insert subsequent floppy disks so you could install rest of > the installation binaries (including the shell utilities, the C > compiler, X windows systems, etc.) and then reboot onto the HDD with a > fully set up system, was a pretty natural evolution. > Yes. FreeBSD 2.0 definitely did that, and I think FreeBSD 1.0 as well. > And since we had the ramdisk infrastructure, and most users didn't > want to configure their own kernel as part of the installation > process, that begat kernel modules and the initial ramdisk being used > to store the kernel modules. In the very early days, this made Linux > *far* more user friendly for non-system programmers to install, > compared to say, FreeBSD during that era, which was still stuck in the > BSD 4.x days of a generic kernel, followed by a kernel configuration > step, etc. > GENERIC worked fine in the early days: it was small enough that the savings from removing all the extra devices was small... but that's because the number of devices supported was also relatively small. By the time it became an issue FreeBSD did have dynamic loading of drivers, but retained the GENERIC kernel for install (it should have gone to a minimal kernel, but hasn't yet completed the migration: the savings wasn't big enough due to Moore's Law growing machine memories faster than the kernel grew: FreeBSD's kernel only doubled every 2 years...). But yea, it was a bit of a kludge. And FreeBSD grew the ability to configure the morass of ISA devices in the boot loader early on so you could at least boot the generic kernel, and have the loader remember your choices. > These days, given that some enterprise setups what their servers to be > installed from a Fibre Channel Storage Area Network, or iSCSI, perhaps > using Kerberos to authenticate to the iSCSI target, it makes > absolutely a huge amount of sense to have an initial ramdisk as an > option. That being said, for a long time, the initial ramdisk was an > *option*. You didn't have to use it, if you were willing to have a > custom kernel and you are using a device which has a fixed and stable > boot-time enumeration. > Yea. FreeBSD can use it to this day (and many interesting use cases use it), but generally the loader tells the kernel what root filesystem to use. These smarts in the boot loader have also kept the pressure of the vast majority of cases where initramfs makes sense in Linux land. > This tended to only be used by people who knew what they were doing, > but for example, my file system test appliance VM system doesn't use > an initial ramdisk at all. It uses qemu, and has a built-in kernel > configuration and build system, and this works perfectly fine booting > into the latest Debian stable distribution, with no initial ramdisk > used at all: > > > https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md > Ah, cool. Every single distro I've run in the past has had an initramfs. I thought it had become no longer an option... I do similar things for my VM setup with FreeBSD :) Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsg at jsg.id.au Tue Sep 5 11:07:46 2023 From: jsg at jsg.id.au (Jonathan Gray) Date: Tue, 5 Sep 2023 11:07:46 +1000 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: On Mon, Sep 04, 2023 at 10:44:21AM -0400, Norman Wilson wrote: > I don't remember any special many-programs-in-one binary > like busybox in any Unix from the days when Unix was simple > enough for me to understand. That covers the entire lifetime > of the Research systems, but also System V and the BSDs and > their sundry offspring up into at least the 1990s. > > I'm pretty sure OpenBSD at least still has nothing like > busybox. The nearest thing was to make sure certain programs look at ls -i output on the install media/bsd.rd https://man.openbsd.org/crunchgen.8 Added to FreeBSD and NetBSD in 1994. From tuhs at tuhs.org Tue Sep 5 21:15:34 2023 From: tuhs at tuhs.org (Paul Ruizendaal via TUHS) Date: Tue, 5 Sep 2023 13:15:34 +0200 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: Message-ID: <8D99EBF7-CB1D-48EF-B6AD-914B45D48A5F@planet.nl> > Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually > do the research on the areas I was hand-wavy about? That certainly answered the prompt, much appreciated the walkthrough. Currently just trying to get a view of the field and to collect recollections on how it was done back in the day. Part of it is finding a conceptual framework that can make sense of it all. One pitfall I would like to avoid in my own thinking is conflating “installing” and “booting”, even though the two seem related (one loads bits into permanent storage, the other into volatile storage; both are built around incrementally adding capability). When variable hardware comes into play, the two mix even more. Also related is the topic of recovery. The starting point seems to be a setup where a small set of standalone programs is used to load or repair the bits, as was done for 16-bit unix. A next conceptual step seems to be where first a very basic system is installed that is then used for further installation or for repair. This step seems to have come early, if this 32V install page is reflective of how it was done back in 1980: https://gunkies.org/wiki/Installing_32V_on_SIMH The idea to use disk swap space for this also seems to have come early (and I suppose the concept lives on in “rescue partitions”). Another conceptual step might be where early installer phases run a different, smaller kernel (or even OS) than the one being installed. There seems to be much potential for a “wheel of reincarnation” here where as an installer grows large, a pre-installer is created to load the installer and then the pre-installer grows large, etc. For booting, this wheel seems to have turned about 5 times in current Linux. Installing that from scratch on a fully blank SBC (without prepping a removable disk on another computer) also appears to have 4 or 5 revolutions. That is one revolution every 5 years for the 25 years from the late seventies to the early 2000’s. Then there is the question of where in the "installer stack" to stop. My current interest excludes (precursors to) package managers, containers, etc. In short: much to ponder, thanks to all for sharing recollections. > On 4 Sep 2023, at 19:07, Warner Losh wrote: > > > > On Mon, Sep 4, 2023 at 3:58 AM Paul Ruizendaal via TUHS wrote: > > Recently, I was looking into the “Das U-Boot” boot loader package. Summarised with great simplification, u-boot bundles device drivers, file systems, commands and a Bourne-like shell into a standalone package. Normally it auto-runs a script that brings up a system, but when used in interactive mode it allows a great deal of poking around. > > It made me think of the “standalone” set of programs for installing early Unix. On 16-bit understandably each basic command has to be a separate standalone program, but after the shift to 32-bit bundling more functionality in a single binary would have become possible. > > How did the Unix “standalone” package evolve in the 80’s, both in the research and BSD lineages? Is there any retrospective paper about that? Or is it a case of “Use the source, Luke”? > > The stand package continued in research and BSD to be those programs needed to install and/or recover > badly damaged systems. You could create a new file system, copy a file from the tape to a partition, etc. > You couldn't do general scripting with this, by and large. > > Originally, they were tape programs. This made sense because of its original focus. In time, some systems > could load the stand alone programs instead of the kernel, but they continued the original focus. > > This is, imho, due in large part due to the miniroot. The miniroot evolved into both a full-enough system > to do the installation scripts in shell instead of C (Venix, at least, had their install program written in C). > You'd copy the minroot to swap and then install the system. But a number of additional programs were > placed into the miniroot so you could do some limited filesystem repair, file editing, etc. > > In addition, many vendor's ROMs grew in complexity. Solbourne's ROMs, for example, could do basic > repair of UFS (clri level, not fsck level), and copy files from one place to another. I often recovered a > Solbourne system I screwed up by attaching an external SCSI drive that had a known good kernel, > init, etc. > > The 'stand' environment was a whole set of tools that could be used to build stand-alone programs that > shared much code of their full unix brethren, despite not having a full kernel under them. Kernel services > were provided by different libraries that did filesystem things, block driver things, network things, etc > in a similar way to Unix, but with a much reduced footprint. > > initramfs, as has been mentioned elsewhere, is pretty much a Linux invention. It was designed to > 'punt' on the choose where to load things from and have a very minimal interface between the boot > loader and the system. In time, it grew to support more interfaces, more ways of loading, and better > ways to mount something that you could then 'pivot' onto. Few other unix systems went this route, though > many adopted some variation on the pivot_root functionality. Linux has moved beyond the pivot root after > having booted the correct kernel into being able to take over the machine early in, say, UEFI startup with > a minimal kernel and initramfs that just knows how to load the next kernel. They skipped the complex boot > loader stage, and went straight to the 'run linux earlier' stage which is how things like LinuxBoot, coreboot > and others have put the boot logic into bash scripts. The ability to 'kexec' a kernel and replace the current > running kernel originated in the 'non-stop' world that wanted to reduce downtime. Now, it's used to reduce > firmware complexity by eliminating large swaths of UEFI from the boot process, but also generalizes in > the embedded space. > > FreeBSD, from around FreeBSD 2 (1995 or so), had /rescue which largely took over form the stand alone environment > for the repair duties of things. FreeBSD also adopted a more complex boot loader that would load the > kernel, modules, set tunables, etc prior to kicking off the kernel. Between /rescue having all the tools needed > to repair bad updates, repair failing disks, get that one last backup before the drive is dead while you wait > for the new drive to be delivered, etc, and /boot/loader being able to script loading the kernel while the BIOS > was still around so the need for drivers in the loader was lessened. However, as the BIOS evolved into UEFI > and FreeBSD pushed into the embedded space whose firmware provided a less rich environment to the boot > loader, so it was able to load things off fewer and fewer devices, it became clear that it would need a pivot > root feature to allow it to boot all the way into FreeBSD, load some drivers from an included ram disk, and then > use that mount a new root and then 'reroot' to that by killing everything and running init from that new root. > FreeBSD also moved from Forth to Lua in its scripting language for the boot loader, giving 'pre boot' > environment support better features. I also added the ability to use FreeBSD boot loader as a Linux binary > to load FreeBSD and its metadata from a LinuxBoot environment. Finally, FreeBSD has 'spun out' and > generalized the /rescue feature to allow creation of any 'BeastyBox' environment, similar to what you get > in a busy box, or clone, environment. This environment, though, is meant in large part on both Linux and > FreeBSD to be in constrained environments where a full install is prohibitive (even those that never pivot > to something more, like ap, routers and nas boxes). > > NetBSD retains many of the old BSD stand-alone programs that started on the vax. I've not studied things > beyond noticing this. OpenBSD is similar. Their boot chain is a bit simpler than FreeBSD's, though there's > noises about porting FreeBSD's boot there. There's a port of /boot/loader to illumos too, but I don't know > if it is the default, or just available. So I'll not chat about it more. > > So the original 'standalone' environment where you had one program running on a system has evolved > into either a rich boot loader environment that lets one do a lot to decide what kernel to load, or towards > having a minimal selection of unix programs faster and using /bin/sh or similar to do scripting. These > reduced environments are often called standalone, though all they share just the name with the earlier > 'stand' programs: they are full unix programs, but with reduced feature sets and 'linker magic' to package > them in a way that's faster, smaller, etc (eg all in one binary). FreeBSD's boot loader is an outgrowth > of the original standalone env, by way of a port of NetBSD's libsa. > > I suspect in the future, we'll see more and more of a trend for low-level init and then handing off to some > built-in kernel (be it Linux, BSD-based (there's now kexec), or whatever) to reuse more of the vetted code > rather than re-inventing Unix inside the boot loader (which is a valid criticism of FreeBSD's boot loader, > though it's rich feature set is what you get for the complexity). > > Does that answer the prompt? Should I try to make this into more of a retrospective paper and actually > do the research on the areas I was hand-wavy about? > > Warner From clemc at ccc.com Wed Sep 6 00:15:17 2023 From: clemc at ccc.com (Clem Cole) Date: Tue, 5 Sep 2023 10:15:17 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <8D99EBF7-CB1D-48EF-B6AD-914B45D48A5F@planet.nl> References: <8D99EBF7-CB1D-48EF-B6AD-914B45D48A5F@planet.nl> Message-ID: below.. On Tue, Sep 5, 2023 at 7:16 AM Paul Ruizendaal via TUHS wrote: > One pitfall I would like to avoid in my own thinking is conflating > “installing” and “booting”, > That is an excellent point. I think that the real question comes back to how 'cold' the 'system' is. Booting is the process of making the processor 'live' including the loading of the 'initial memory contents' (IBM used to call it IPL for Initial Program Load), while 'installation' is setting up a 'system' which 'lives' in the peripherals as long-term storage. When you 'Power-On' a CPU, you often need to IPL a system even if the peripherals have been set up [back in the day of core memory, you might not even have that as the OS was not lost when power was removed]. > The starting point seems to be a setup where a small set of standalone > programs is used to load or repair the bits, as was done for 16-bit unix. > You are solving the 'chicken and egg' issue. Truth is Ken's famous "Reflections on Trusting Trust" comes to bear here. The fact is you are 'setting up' (or restoring) a system for bits that were generated on another. The critical point is that you are setting the initial 'systems' bits into the peripherals for that specific system. You are relying on making copies of those bits from another system. The question is how to get them into the new media. All the standalone system is doing is offering you a minimum way of making a copy of that other system without having access to its hardware directly. > A next conceptual step seems to be where first a very basic system is > installed that is then used for further installation or for repair. This > step seems to have come early, if this 32V install page is reflective of > how it was done back in 1980: > https://gunkies.org/wiki/Installing_32V_on_SIMH The idea to use disk > swap space for this also seems to have come early (and I suppose the > concept lives on in “rescue partitions”). > Newer versions of the system setup scheme offer more and more features, as the options of how you set up might be different from the origin system become greater and greater. Modern OS implementations of the Unix technologies now run on a much more comprehensive range of devices, and frankly, 'IPL' much less might be set up from so many different types of sources, it's not surprising that the IPL schemes and the system setup scheme are a lot more sophisticated. But remember, when you examine the past scheme, you must also consider the constraints of the timeframe (particularly the economics of certain choices). It's not that you could not have built something like some of today's schemes — it was expensive with respect to what was available and, frankly, not wholly necessary. > > Another conceptual step might be where early installer phases run a > different, smaller kernel (or even OS) than the one being installed. There > seems to be much potential for a “wheel of reincarnation” here where as an > installer grows large, a pre-installer is created to load the installer and > then the pre-installer grows large, etc. For booting, this wheel seems to > have turned about 5 times in current Linux. Installing that from scratch on > a fully blank SBC (without prepping a removable disk on another computer) > also appears to have 4 or 5 revolutions. That is one revolution every 5 > years for the 25 years from the late seventies to the early 2000’s. > The circle started long before that. For instance, the boot/IPL got more flexible (like Sam's autoconfiguration work developed for VAX). Why? Because the HW configurations had begun to branch and get bushy. Even with the PDP-11's peripheral set in V7, it was getting more complex. But those kernel configurations were statically linked. Sun added dynamic linking, making both IPL, much less setup/installation more complex. But frankly, to do that on an earlier machine took a lot of work. BSD2.9 did backport much of the VAX work but not all of it. It was often too difficult, and the 'gain' for the effort was low. So my modern UNIX implementations, be it macOS, *BSD, or the Linux family, the schemes have matured and been recreated over and over. For instance, you can look at what Apple's done with its system install scheme. It hides the setup/tear down as of the root and then makes a read-only root system under the covers. This is both a blessing and a curse. Sure, it helps make things a lot more secure for them and basic users, but that choice breaks traditional admin scripts [which drives me nuts think - /etc/periodic] because the new Apple system implementors don't understand why the core system works the way it does and thus those of us that have scripts have worked since V7 in the late 1970s, now don't. Clem ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steffen at sdaoden.eu Wed Sep 6 01:53:01 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Tue, 05 Sep 2023 17:53:01 +0200 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230904221059.sF2G0%steffen@sdaoden.eu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> Message-ID: <20230905155301.mIziN%steffen@sdaoden.eu> Steffen Nurpmeso wrote in <20230904221059.sF2G0%steffen at sdaoden.eu>: |Norman Wilson wrote in | <9A989054DE79CE5059CBA74797391E39.for-standards-violators at oclsc.org>: ... ||Perhaps the question to ask is why such a magic program is ||needed at all. Is it just because programs like the shell ||have become so large and unwieldy that they won't fit in ||a small environment suitable for loading into an initramfs? ... |For my laptop it allows me easy boot management. ... | -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* | -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* | -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* | -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* | -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* Only to add that this is because of Linux and the way it is doing things. If i would use FreeBSD on bare metal, then i would have an EFI boot loader on EFI that knows (only) enough to ask for passphrase (correct me if i am wrong), and can then boot the kernel from FFS or ZFS. (You have to choose dedicated ZFS boot loader iirc, but despite that...) I know GRUB (and maybe other) Linux bootloaders can do all that, but they are huge, are badly maintained, or under-documented, let alone with local manuals, and i am too stupid to configure them (due to all that). refind is ok, however. But.. be aware of typos in the configuration.. But anyhow. With an EFI_STUB Linux kernel i can save me all that, with busybox i get a complete environment (i then even create an initrd in /boot/ on the fly so i do not have to type the password a second time, that can (optionally) be cached, and is, actually -rw------- 1 root root 4495987 May 29 16:29 .kent.initrd.0 Unfortunately cryptsetup is needed even though, i think, the kernel has anything needed; you just cannot access it. cryptsetup is only needed for "$cs open $PART_ROOT p_root --key-file -". Of course i am no real Linux expert but only a do-it-yourself guy. busybox allows me to manage this easily, to answer your question. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From paul.winalski at gmail.com Wed Sep 6 03:03:29 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 5 Sep 2023 13:03:29 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: On 9/4/23, Clem Cole wrote: > > Even with the VAX, the front-end runs on an LSI-11 with floppies, so things > like microcode were loaded into the VAX and stored in ROM. I think you meant either "stored in RAM" or "not stored in ROM". The LSI-11 front end for the VAX-11/780 (and other early VAXen?) was connected to the SBI (Synchronous Backplane Interconnect--the system bus) and performed three main functions: microcode load and system bootstrap initiation, interface for the system console (originally a LA-36 dot matrix terminal), and some system diagnostics. CPU microcode was stored on a 5" floppy disk connected to the LSI-11 and was loaded into RAM at system power-up. -Paul W. From imp at bsdimp.com Wed Sep 6 03:03:52 2023 From: imp at bsdimp.com (Warner Losh) Date: Tue, 5 Sep 2023 11:03:52 -0600 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <8D99EBF7-CB1D-48EF-B6AD-914B45D48A5F@planet.nl> References: <8D99EBF7-CB1D-48EF-B6AD-914B45D48A5F@planet.nl> Message-ID: On Tue, Sep 5, 2023 at 5:15 AM Paul Ruizendaal wrote: > > Does that answer the prompt? Should I try to make this into more of a > retrospective paper and actually > > do the research on the areas I was hand-wavy about? > > That certainly answered the prompt, much appreciated the walkthrough. > Currently just trying to get a view of the field and to collect > recollections on how it was done back in the day. > > Part of it is finding a conceptual framework that can make sense of it all. > Yes. V7 and other early unix had a 'stand alone' framework. So that's one piece of the puzzle. It could be used for anything (I've seen private diagnostic stand alone programs at Solbourne that would do more extensive tests than the boot roms did for RMA and manufacturing line acceptance). But as they come down to us from AT&T, they are just the bits needed to install a system from scratch (or in some cases fix a system by booting the distribution tapes). > One pitfall I would like to avoid in my own thinking is conflating > “installing” and “booting”, even though the two seem related (one loads > bits into permanent storage, the other into volatile storage; both are > built around incrementally adding capability). When variable hardware comes > into play, the two mix even more. Also related is the topic of recovery. > There's "booting" and then there's "bootstrapping". booting happens all the time, but installation happens rarely. The problem is that the install process for V7 and earlier systems was to load a series of programs into memory from tape that did each step of the process to stay within the 64k limits imposed by the platform. That process was done by a series of standalone programs that could, in some cases, act as system repair for certain cases of 'afu' system. > The starting point seems to be a setup where a small set of standalone > programs is used to load or repair the bits, as was done for 16-bit unix. > Yea, that's where stand came from originally: An environment that could run a limited program w/o booting the kernel... > A next conceptual step seems to be where first a very basic system is > installed that is then used for further installation or for repair. This > step seems to have come early, if this 32V install page is reflective of > how it was done back in 1980: > https://gunkies.org/wiki/Installing_32V_on_SIMH The idea to use disk > swap space for this also seems to have come early (and I suppose the > concept lives on in “rescue partitions”). > Yes. It was quite common for the next step to be when demand paging became a thing. Once demand paging became a thing and you didn't need a dedicated swap partition, you could 'borrow' the installed system's swap partition (or what would become the installed system's swap partition) to have a richer set of tools. This would use otherwise wasted space for the install, then you'd overwrite it with whatever workload you put on your system. > Another conceptual step might be where early installer phases run a > different, smaller kernel (or even OS) than the one being installed. There > seems to be much potential for a “wheel of reincarnation” here where as an > installer grows large, a pre-installer is created to load the installer and > then the pre-installer grows large, etc. For booting, this wheel seems to > have turned about 5 times in current Linux. Installing that from scratch on > a fully blank SBC (without prepping a removable disk on another computer) > also appears to have 4 or 5 revolutions. That is one revolution every 5 > years for the 25 years from the late seventies to the early 2000’s. > Indeed, the whole linuxboot saga where you boot a minimal linux kernel that has a shell script (or similar) that decides where to load the complete kernel from is also interesting (doubly so to me because it means I can sneak in at that point and load FreeBSD onto the system). > Then there is the question of where in the "installer stack" to stop. My > current interest excludes (precursors to) package managers, containers, etc. > Yes. For me, the 'preboot environment' is a better mental framework to look at it with. Or 'pre-kernel' or 'non-kernel' environments. Though that paradigm runs out of steam in the 90s when people started booting kernels with bundled ram disks (in various flavors) to do the installation to allow boot medium to be used to load additional data (eg the system). The read/write nature of RAM made this vector useful, even when CDROMs appeared on the scene. It was easier to run off a ram disk than off the CDROM because you don't have to have symlinks to a (small) RAM disks for the bits of the system that needed to be read/write during the install process... The 'standalone' stuff, as typified by the src/stand directory, was all pre-kernel or non-kernel use-cases. Sometimes it was used to build a bit of code that had to be small, but was also used to build code to load the kernel (even dating back to the original boot loader for V7, though I'm relying on my memory which may be confusing that with the 2BSD bootstrap)... The stuff that was 'standalone-but-on-a-unix-kernel' came later. It might also be amusing to note: I once contemplated porting V7 unix to replace FreeBSD's stand environment to replace the various ad-hock hacks that had grown up. But I found that those hacks were smaller in the end, and needed less code space to support more things (excluding crypto and compression) than the V7 did once you started looking at modifying it to support multiple filesystem types, adding a network stack etc. While V7 is nice and clean, it's also immature in terms of features needed for a modern loader... Warner > In short: much to ponder, thanks to all for sharing recollections. > > > > On 4 Sep 2023, at 19:07, Warner Losh wrote: > > > > > > > > On Mon, Sep 4, 2023 at 3:58 AM Paul Ruizendaal via TUHS > wrote: > > > > Recently, I was looking into the “Das U-Boot” boot loader package. > Summarised with great simplification, u-boot bundles device drivers, file > systems, commands and a Bourne-like shell into a standalone package. > Normally it auto-runs a script that brings up a system, but when used in > interactive mode it allows a great deal of poking around. > > > > It made me think of the “standalone” set of programs for installing > early Unix. On 16-bit understandably each basic command has to be a > separate standalone program, but after the shift to 32-bit bundling more > functionality in a single binary would have become possible. > > > > How did the Unix “standalone” package evolve in the 80’s, both in the > research and BSD lineages? Is there any retrospective paper about that? Or > is it a case of “Use the source, Luke”? > > > > The stand package continued in research and BSD to be those programs > needed to install and/or recover > > badly damaged systems. You could create a new file system, copy a file > from the tape to a partition, etc. > > You couldn't do general scripting with this, by and large. > > > > Originally, they were tape programs. This made sense because of its > original focus. In time, some systems > > could load the stand alone programs instead of the kernel, but they > continued the original focus. > > > > This is, imho, due in large part due to the miniroot. The miniroot > evolved into both a full-enough system > > to do the installation scripts in shell instead of C (Venix, at least, > had their install program written in C). > > You'd copy the minroot to swap and then install the system. But a number > of additional programs were > > placed into the miniroot so you could do some limited filesystem repair, > file editing, etc. > > > > In addition, many vendor's ROMs grew in complexity. Solbourne's ROMs, > for example, could do basic > > repair of UFS (clri level, not fsck level), and copy files from one > place to another. I often recovered a > > Solbourne system I screwed up by attaching an external SCSI drive that > had a known good kernel, > > init, etc. > > > > The 'stand' environment was a whole set of tools that could be used to > build stand-alone programs that > > shared much code of their full unix brethren, despite not having a full > kernel under them. Kernel services > > were provided by different libraries that did filesystem things, block > driver things, network things, etc > > in a similar way to Unix, but with a much reduced footprint. > > > > initramfs, as has been mentioned elsewhere, is pretty much a Linux > invention. It was designed to > > 'punt' on the choose where to load things from and have a very minimal > interface between the boot > > loader and the system. In time, it grew to support more interfaces, more > ways of loading, and better > > ways to mount something that you could then 'pivot' onto. Few other unix > systems went this route, though > > many adopted some variation on the pivot_root functionality. Linux has > moved beyond the pivot root after > > having booted the correct kernel into being able to take over the > machine early in, say, UEFI startup with > > a minimal kernel and initramfs that just knows how to load the next > kernel. They skipped the complex boot > > loader stage, and went straight to the 'run linux earlier' stage which > is how things like LinuxBoot, coreboot > > and others have put the boot logic into bash scripts. The ability to > 'kexec' a kernel and replace the current > > running kernel originated in the 'non-stop' world that wanted to reduce > downtime. Now, it's used to reduce > > firmware complexity by eliminating large swaths of UEFI from the boot > process, but also generalizes in > > the embedded space. > > > > FreeBSD, from around FreeBSD 2 (1995 or so), had /rescue which largely > took over form the stand alone environment > > for the repair duties of things. FreeBSD also adopted a more complex > boot loader that would load the > > kernel, modules, set tunables, etc prior to kicking off the kernel. > Between /rescue having all the tools needed > > to repair bad updates, repair failing disks, get that one last backup > before the drive is dead while you wait > > for the new drive to be delivered, etc, and /boot/loader being able to > script loading the kernel while the BIOS > > was still around so the need for drivers in the loader was lessened. > However, as the BIOS evolved into UEFI > > and FreeBSD pushed into the embedded space whose firmware provided a > less rich environment to the boot > > loader, so it was able to load things off fewer and fewer devices, it > became clear that it would need a pivot > > root feature to allow it to boot all the way into FreeBSD, load some > drivers from an included ram disk, and then > > use that mount a new root and then 'reroot' to that by killing > everything and running init from that new root. > > FreeBSD also moved from Forth to Lua in its scripting language for the > boot loader, giving 'pre boot' > > environment support better features. I also added the ability to use > FreeBSD boot loader as a Linux binary > > to load FreeBSD and its metadata from a LinuxBoot environment. Finally, > FreeBSD has 'spun out' and > > generalized the /rescue feature to allow creation of any 'BeastyBox' > environment, similar to what you get > > in a busy box, or clone, environment. This environment, though, is meant > in large part on both Linux and > > FreeBSD to be in constrained environments where a full install is > prohibitive (even those that never pivot > > to something more, like ap, routers and nas boxes). > > > > NetBSD retains many of the old BSD stand-alone programs that started on > the vax. I've not studied things > > beyond noticing this. OpenBSD is similar. Their boot chain is a bit > simpler than FreeBSD's, though there's > > noises about porting FreeBSD's boot there. There's a port of > /boot/loader to illumos too, but I don't know > > if it is the default, or just available. So I'll not chat about it more. > > > > So the original 'standalone' environment where you had one program > running on a system has evolved > > into either a rich boot loader environment that lets one do a lot to > decide what kernel to load, or towards > > having a minimal selection of unix programs faster and using /bin/sh or > similar to do scripting. These > > reduced environments are often called standalone, though all they share > just the name with the earlier > > 'stand' programs: they are full unix programs, but with reduced feature > sets and 'linker magic' to package > > them in a way that's faster, smaller, etc (eg all in one binary). > FreeBSD's boot loader is an outgrowth > > of the original standalone env, by way of a port of NetBSD's libsa. > > > > I suspect in the future, we'll see more and more of a trend for > low-level init and then handing off to some > > built-in kernel (be it Linux, BSD-based (there's now kexec), or > whatever) to reuse more of the vetted code > > rather than re-inventing Unix inside the boot loader (which is a valid > criticism of FreeBSD's boot loader, > > though it's rich feature set is what you get for the complexity). > > > > Does that answer the prompt? Should I try to make this into more of a > retrospective paper and actually > > do the research on the areas I was hand-wavy about? > > > > Warner > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Wed Sep 6 04:02:49 2023 From: clemc at ccc.com (Clem Cole) Date: Tue, 5 Sep 2023 14:02:49 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> Message-ID: On Tue, Sep 5, 2023 at 1:03 PM Paul Winalski wrote: > On 9/4/23, Clem Cole wrote: > > > > Even with the VAX, the front-end runs on an LSI-11 with floppies, so > things > > like microcode were loaded into the VAX and stored in ROM. > > I think you meant either "stored in RAM" or "not stored in ROM". Yep -- thanks - dyslexia-r-me. ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Thu Sep 7 03:50:45 2023 From: imp at bsdimp.com (Warner Losh) Date: Wed, 6 Sep 2023 11:50:45 -0600 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230905155301.mIziN%steffen@sdaoden.eu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> Message-ID: On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso wrote: > Steffen Nurpmeso wrote in > <20230904221059.sF2G0%steffen at sdaoden.eu>: > |Norman Wilson wrote in > | <9A989054DE79CE5059CBA74797391E39.for-standards-violators at oclsc.org>: > ... > ||Perhaps the question to ask is why such a magic program is > ||needed at all. Is it just because programs like the shell > ||have become so large and unwieldy that they won't fit in > ||a small environment suitable for loading into an initramfs? > ... > |For my laptop it allows me easy boot management. > ... > | -rwxr-xr-x 1 root root 4596 Feb 4 2023 linux-init-s1.sh* > | -rwxr-xr-x 1 root root 3646 Feb 4 2023 linux-init-lib.sh* > | -rwxr-xr-x 1 root root 5480120 Feb 11 2023 cryptsetup.static* > | -rwxr-xr-x 1 root root 1978368 Aug 15 18:51 busybox.static* > | -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* > > Only to add that this is because of Linux and the way it is doing > things. If i would use FreeBSD on bare metal, then i would have > an EFI boot loader on EFI that knows (only) enough to ask for > passphrase (correct me if i am wrong), and can then boot the > kernel from FFS or ZFS. (You have to choose dedicated ZFS boot > loader iirc, but despite that...) > No, you don't have to choose the dedicated ZFS boot loader, at least not anymore. Also, you can use boot1.efi to load loader.efi from the root filesystem to load the kernel, or you could use loader.efi directly on the ESP to load the kernel. boot1 barely knows anything (and has only one choice of what to boot). loader.efi is the full deal, and can do rather a lot of sophisticated things. > I know GRUB (and maybe other) Linux bootloaders can do all that, > but they are huge, are badly maintained, or under-documented, let > alone with local manuals, and i am too stupid to configure them > (due to all that). refind is ok, however. But.. be aware of > typos in the configuration.. > > But anyhow. With an EFI_STUB Linux kernel i can save me all that, > with busybox i get a complete environment (i then even create an > initrd in /boot/ on the fly so i do not have to type the password > a second time, that can (optionally) be cached, and is, actually > > -rw------- 1 root root 4495987 May 29 16:29 .kent.initrd.0 > > Unfortunately cryptsetup is needed even though, i think, the > kernel has anything needed; you just cannot access it. cryptsetup > is only needed for "$cs open $PART_ROOT p_root --key-file -". > Of course i am no real Linux expert but only a do-it-yourself guy. > busybox allows me to manage this easily, to answer your question. > You could do that on FreeBSD with a loader.efi that has a ram disk built into it as well, including a 'beastie box' thing that's akin to busybox. It will boot in one step and no no further I/O to get a running system. Others have used this for secure boot and to boot a small ram disk that's later discarded as userland decides what root should be. But it's much less automated than in Linux... Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From steffen at sdaoden.eu Thu Sep 7 10:11:57 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Thu, 07 Sep 2023 02:11:57 +0200 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> Message-ID: <20230907001157.Qld6B%steffen@sdaoden.eu> Hello! Warner Losh wrote in : |On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso wrote: |> Steffen Nurpmeso wrote in |> <20230904221059.sF2G0%steffen at sdaoden.eu>: |>|Norman Wilson wrote in |>| <9A989054DE79CE5059CBA74797391E39.for-standards-violators at oclsc.org>: |> ... |>||Perhaps the question to ask is why such a magic program is |>||needed at all. Is it just because programs like the shell |>||have become so large and unwieldy that they won't fit in |>||a small environment suitable for loading into an initramfs? |> ... |>|For my laptop it allows me easy boot management. ... |> Only to add that this is because of Linux and the way it is doing |> things. If i would use FreeBSD on bare metal, then i would have |> an EFI boot loader on EFI that knows (only) enough to ask for |> passphrase (correct me if i am wrong), and can then boot the |> kernel from FFS or ZFS. (You have to choose dedicated ZFS boot |> loader iirc, but despite that...) | |No, you don't have to choose the dedicated ZFS boot loader, at least not |anymore. Ah! It seems regarding FreeBSD i am stuck in BIOS world, then. (zfsboot and gptzfsboot.) |Also, you can use boot1.efi to load loader.efi from the root filesystem to The former deprecated i see. |load the kernel, or you could use loader.efi directly on the ESP to load |the kernel. boot1 barely knows anything (and has only one choice of |what to boot). loader.efi is the full deal, and can do rather a lot of |sophisticated things. Great. ... |> But anyhow. With an EFI_STUB Linux kernel i can save me all that, |> with busybox i get a complete environment (i then even create an |> initrd in /boot/ on the fly so i do not have to type the password |> a second time, that can (optionally) be cached, and is, actually ... |> Unfortunately cryptsetup is needed even though, i think, the |> kernel has anything needed; you just cannot access it. cryptsetup |> is only needed for "$cs open $PART_ROOT p_root --key-file -". |> Of course i am no real Linux expert but only a do-it-yourself guy. Well, and user space only, and never having read BIOS or UEFI standards / implementations. I have reread (almost) all of FreeBSDs documentation (loader, uefi etc) today, and it is always great to see such good documentation! Of course some things are beyond what a "normal" user would understand, but at least there is documentation available. That is really great. |> busybox allows me to manage this easily, to answer your question. | |You could do that on FreeBSD with a loader.efi that has a ram disk |built into it as well, including a 'beastie box' thing that's akin to |busybox. |It will boot in one step and no no further I/O to get a running system. |Others have used this for secure boot and to boot a small ram disk that's |later discarded as userland decides what root should be. But it's much less |automated than in Linux... Hm, but wait. Isn't it easier for FreeBSD? I must admit i never used FreeBSD/geli on bare metal, i started using fully encrypted hard disks in mid 2021, no sooner. So when i read [1] it seems FreeBSD's boot procedure is so nicely linked with geli that the boot loader asks itself for the password of the encrypted partition, simply by setting some variables in loader.conf? I mean, the /boot/ must be available? [1] https://man.freebsd.org/cgi/man.cgi?query=geli&apropos=0&sektion=0&manpath=FreeBSD+13.2-RELEASE+and+Ports&arch=default&format=html These simple scripts i use for Linux also do not support true suspend, only suspend-to-ram. Ie, i have no idea how i could "suspend" the encrypted device and then "resume" it again; The "warm" security is only (initiated via ACPI event from root account) auto-unload of SSH and PGP keys etc, unload of encfs fuse volumes, and then slock of all X displays etc. Ie, all programs which run live on the encrypted volume. When the laptop wakes up, X is running etc. One would need the possibility to wake up to "a password prompt". Well, likely it would work to mount EFI vfat before the suspend, create a console and auto-switch to it, and then sit there, ready for reading in the password once the system resumes. That is to say: i always believed in FreeBSD this works just as easy as the boot procedure? --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From imp at bsdimp.com Fri Sep 8 02:05:37 2023 From: imp at bsdimp.com (Warner Losh) Date: Thu, 7 Sep 2023 10:05:37 -0600 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230907001157.Qld6B%steffen@sdaoden.eu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> <20230907001157.Qld6B%steffen@sdaoden.eu> Message-ID: On Wed, Sep 6, 2023 at 6:12 PM Steffen Nurpmeso wrote: > Hello! > > Warner Losh wrote in > : > |On Tue, Sep 5, 2023 at 9:53 AM Steffen Nurpmeso > wrote: > |> Steffen Nurpmeso wrote in > |> <20230904221059.sF2G0%steffen at sdaoden.eu>: > |>|Norman Wilson wrote in > |>| <9A989054DE79CE5059CBA74797391E39.for-standards-violators at oclsc.org>: > |> ... > |>||Perhaps the question to ask is why such a magic program is > |>||needed at all. Is it just because programs like the shell > |>||have become so large and unwieldy that they won't fit in > |>||a small environment suitable for loading into an initramfs? > |> ... > |>|For my laptop it allows me easy boot management. > ... > |> Only to add that this is because of Linux and the way it is doing > |> things. If i would use FreeBSD on bare metal, then i would have > |> an EFI boot loader on EFI that knows (only) enough to ask for > |> passphrase (correct me if i am wrong), and can then boot the > |> kernel from FFS or ZFS. (You have to choose dedicated ZFS boot > |> loader iirc, but despite that...) > | > |No, you don't have to choose the dedicated ZFS boot loader, at least not > |anymore. > > Ah! It seems regarding FreeBSD i am stuck in BIOS world, then. > (zfsboot and gptzfsboot.) > Yes, for MBR, things are necessarily more complicated because the ZFS boot loader is too big. The first few sectors live in the 8k space that the traditional boot loader lives, and the rest is DD'd to a special area of the ZFS uberblock reserved for boot blocks. For gptboot vs gptzfsboot, they were written to be either or and combining them presented challenges, not least was the partition size that gptboot lives in was created for many releases too small to hold gptzfsboot and making it tricky to upgrade... > |Also, you can use boot1.efi to load loader.efi from the root filesystem > to > > The former deprecated i see. > > |load the kernel, or you could use loader.efi directly on the ESP to load > |the kernel. boot1 barely knows anything (and has only one choice of > |what to boot). loader.efi is the full deal, and can do rather a lot of > |sophisticated things. > > Great. > > ... > |> But anyhow. With an EFI_STUB Linux kernel i can save me all that, > |> with busybox i get a complete environment (i then even create an > |> initrd in /boot/ on the fly so i do not have to type the password > |> a second time, that can (optionally) be cached, and is, actually > ... > |> Unfortunately cryptsetup is needed even though, i think, the > |> kernel has anything needed; you just cannot access it. cryptsetup > |> is only needed for "$cs open $PART_ROOT p_root --key-file -". > |> Of course i am no real Linux expert but only a do-it-yourself guy. > > Well, and user space only, and never having read BIOS or UEFI > standards / implementations. > I have reread (almost) all of FreeBSDs documentation (loader, uefi > etc) today, and it is always great to see such good documentation! > Of course some things are beyond what a "normal" user would > understand, but at least there is documentation available. > That is really great. > > |> busybox allows me to manage this easily, to answer your question. > | > |You could do that on FreeBSD with a loader.efi that has a ram disk > |built into it as well, including a 'beastie box' thing that's akin to > |busybox. > |It will boot in one step and no no further I/O to get a running system. > |Others have used this for secure boot and to boot a small ram disk that's > |later discarded as userland decides what root should be. But it's much > less > |automated than in Linux... > > Hm, but wait. Isn't it easier for FreeBSD? I must admit i never > used FreeBSD/geli on bare metal, i started using fully encrypted > hard disks in mid 2021, no sooner. So when i read [1] it seems > FreeBSD's boot procedure is so nicely linked with geli that the > boot loader asks itself for the password of the encrypted > partition, simply by setting some variables in loader.conf? > I mean, the /boot/ must be available? > > [1] > https://man.freebsd.org/cgi/man.cgi?query=geli&apropos=0&sektion=0&manpath=FreeBSD+13.2-RELEASE+and+Ports&arch=default&format=html > > These simple scripts i use for Linux also do not support true > suspend, only suspend-to-ram. Ie, i have no idea how i could > "suspend" the encrypted device and then "resume" it again; > The "warm" security is only (initiated via ACPI event from root > account) auto-unload of SSH and PGP keys etc, unload of encfs fuse > volumes, and then slock of all X displays etc. Ie, all programs > which run live on the encrypted volume. When the laptop wakes up, > X is running etc. > One would need the possibility to wake up to "a password prompt". > Well, likely it would work to mount EFI vfat before the suspend, > create a console and auto-switch to it, and then sit there, ready > for reading in the password once the system resumes. That is to > say: i always believed in FreeBSD this works just as easy as the > boot procedure? > Yea. I find it easier :). But there's some better automation available in Linux for the 'complicated' situations. I was also trying to be generous to Linux because I know the passion of its supporters in the face of even mild criticism. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From e5655f30a07f at ewoof.net Fri Sep 8 23:56:00 2023 From: e5655f30a07f at ewoof.net (Michael =?utf-8?B?S2rDtnJsaW5n?=) Date: Fri, 8 Sep 2023 13:56:00 +0000 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230905155301.mIziN%steffen@sdaoden.eu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> Message-ID: On 5 Sep 2023 17:53 +0200, from steffen at sdaoden.eu (Steffen Nurpmeso): > Unfortunately cryptsetup is needed even though, i think, the > kernel has anything needed; you just cannot access it. cryptsetup > is only needed for "$cs open $PART_ROOT p_root --key-file -". > Of course i am no real Linux expert but only a do-it-yourself guy. If your need is restricted to a highly specific use case and you are trying to keep it as small as possible, then it should be possible to write a custom wrapper around whatever libcryptsetup functionality you need and avoid the extra code that you get with cryptsetup proper. -- Michael Kjörling 🔗 https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?” From tytso at mit.edu Sat Sep 9 00:58:01 2023 From: tytso at mit.edu (Theodore Ts'o) Date: Fri, 8 Sep 2023 10:58:01 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> <20230907001157.Qld6B%steffen@sdaoden.eu> Message-ID: <20230908145801.GA1054673@mit.edu> On Thu, Sep 07, 2023 at 10:05:37AM -0600, Warner Losh wrote: > > Yea. I find it easier :). But there's some better automation available in > Linux for the 'complicated' situations. Yeah, this is why most of the Linux distributions always use an initramfs, at least by default, because it handles all (or at least most) of these complicated sitautions automatically. So this leads to people assuming that Linux "requires" using a initramfs, when it would be more accurate to say that most Linux distributions makes it super easy to use an initramfs, and much more difficult (because the documentation is a bit scattered, and many may be somewhat out of date) to do things without an Linux distribution-engineered initramfs. And the other clear difference between Linux and FreeBSD is that things like initramfs tends to be distribution-specific, so there is competition between distributions to see who can make a simpler/easier installer and boot sequnce, and so market forces tends to strongly encourage developers (who after all, tend to be experts and who don't need the installer and boot up automation), to improve things much more aggressively. (And since there are product managers involved, often features and time to market often trump issues like technical debt.) The downside with this approach, though, is that distributions tend to reinvent the wheel multiple times, and the initramfs infrastructures are different. So learning how things work in say, Debian, isn't going to help you understand the fine details of how Fedora or Red Hat Enterprise Linux works except at a very basic, high level. And the same is most definitely true if you compare the installers between Debian, Ubuntu, Fedora, etc. FreeBSD has the advantage that it has a much more centralized development model, especially as it relates to kernel/libc integration, the installer, and the boot loader path. But while there is a certain amount of competitive forces of FreeBSD, NetBSD, OpenBSD, et. al., improving and simplifying the boot loader, from an outsider looking in, it would appear that those forces aren't quite as strong, and the level of convergence beween the *BSD seems to be at least as strong, if not stronger, in some areas, than between Linux distributions. Cheers, - Ted From steffen at sdaoden.eu Sat Sep 9 09:38:54 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Sat, 09 Sep 2023 01:38:54 +0200 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> Message-ID: <20230908233854.Xni_j%steffen@sdaoden.eu> Michael Kjörling wrote in : |On 5 Sep 2023 17:53 +0200, from steffen at sdaoden.eu (Steffen Nurpmeso): |> Unfortunately cryptsetup is needed even though, i think, the |> kernel has anything needed; you just cannot access it. cryptsetup |> is only needed for "$cs open $PART_ROOT p_root --key-file -". |> Of course i am no real Linux expert but only a do-it-yourself guy. | |If your need is restricted to a highly specific use case and you are |trying to keep it as small as possible, then it should be possible to |write a custom wrapper around whatever libcryptsetup functionality you |need and avoid the extra code that you get with cryptsetup proper. It is nicely documented, and my Linux distribution ships the static library anyhow. But i am a lazy sort regarding such, i just take the thing of my distribution and copy it over (they do build it statically also by default). You know, things change, and if you do not follow closely, you stand in the rain. I am not a paid Linux engineer that follows this rapidly moving target in the end. For example the (no longer) new random developer chose to disable feeding entropy via /dev/urandom, here (distribution) still is # Load random seed /bin/cat /var/lib/urandom/seed > /dev/urandom for almost two decades (it is a rather young one), but the code path was mutilated (i read the kernel source once he had rewritten that to be blake2/some 32-byte block thing based), now one needs to use some ioctl interface fwiw. Or once here cryptsetup was updated to use OpenSSL 3.0 suddenly ripemd160 was no longer available on EFI (aka purely static, without the filesystem avaialable), even though its release notes explicitly mentioned the problem as solved, and OpenSSL 3's libcrypto.a _had_ ripemd160... I had to switch to sha512 .. then to sha256 once cryptsetup started warning args had to be explicit in the future. Mind you, i in fact use it twice, also for encrypted swap, i only wrongly searched for $cs $2 open --type plain --cipher aes --key-size 256 \ --hash sha256 $PART_SWAP p_swap --key-file - && i said on IRC cryptsetup does EVP_DigestInit_ex(h->md, h->hash_id, NULL), i presume that does load additional things. That surely is it, i did not track it further. So no, to answer you, i have no highly specific use case at all. This is only an encrypted volume with my own boot-style that requires no boot loader but Linux itself. Maybe i should really look deeply in how cryptsetup then attaches a LUKS2 volume to the kernel, maybe it actually _would_ be possible to do this simply in some other way. But truly writing a program? I feel much saver in the horde, with so many people, specialists even, working on Linux, LUKS2, cryptsetup, OpenSSL, .. these are all moving targets. (I mean, i am lucky if i _can_ do a bit of programming on at least the MUA i maintain; so much to do! And roff hopefully somewhere on the horizon, somewhen; today it was zero minutes.) --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From meillo at marmaro.de Sun Sep 10 05:51:32 2023 From: meillo at marmaro.de (markus schnalke) Date: Sat, 09 Sep 2023 21:51:32 +0200 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? Message-ID: <1qf3zQ-7oK-00@marmaro.de> Hoi, I just discovered that one of my favorite computer books about my best liked programming language (besides C) releases in a second edition. Does anyone know what the differences of 1st and 2nd edition are? As the original book is almost perfect, the only rework and extension direction I can think of is towards different implementations like gawk, mawk, portability and such things. Does anyone know more about it? Maybe some inside information? ;-) meillo From christopher.fujino at gmail.com Sun Sep 10 06:13:30 2023 From: christopher.fujino at gmail.com (christopher fujino) Date: Sat, 9 Sep 2023 13:13:30 -0700 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <1qf3zQ-7oK-00@marmaro.de> References: <1qf3zQ-7oK-00@marmaro.de> Message-ID: Oh wow, this is really interesting. Haven't read the original, maybe I should pick this up. On Sat, Sep 9, 2023 at 12:51 PM markus schnalke wrote: > Hoi, > > I just discovered that one of my favorite computer books about my > best liked programming language (besides C) releases in a second > edition. Does anyone know what the differences of 1st and 2nd > edition are? > > As the original book is almost perfect, the only rework and > extension direction I can think of is towards different > implementations like gawk, mawk, portability and such things. > > Does anyone know more about it? Maybe some inside information? ;-) > > > meillo > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steffen at sdaoden.eu Sun Sep 10 08:43:12 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Sun, 10 Sep 2023 00:43:12 +0200 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230908233854.Xni_j%steffen@sdaoden.eu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> <20230908233854.Xni_j%steffen@sdaoden.eu> Message-ID: <20230909224312.gGToQ%steffen@sdaoden.eu> Steffen Nurpmeso wrote in <20230908233854.Xni_j%steffen at sdaoden.eu>: |Michael Kjörling wrote in | : ||On 5 Sep 2023 17:53 +0200, from steffen at sdaoden.eu (Steffen Nurpmeso): ... ||> Of course i am no real Linux expert but only a do-it-yourself guy. || ||If your need is restricted to a highly specific use case and you are ||trying to keep it as small as possible, then it should be possible to ... Actually maybe someone may find it funny, even though it is neither historic nor very sophisticated. Let me post it. In the end it takes some time to get there. I would expect almost all of you will not be interested in the lengthy rest of this mail. Ciao, and a nice Sunday i wish from Germany! All you need are statically linked busybox (and cryptsetup with encryption), and a kernel with EFI_STUB on the EFI partition, so it can be booted directly. The distribution kernels i have (ArchLinux, AlpineLinux) have it enabled; if your EFI is large enough you could simply copy it and all its masses of modules to EFI. Otherwise the kernel needs all the firmware and modules to boot and to access all filesystems where the real stuff is. My own kernel has a lot of things statically built-in, but it can load more via modules (stage2 uses "the same" kernel later, and does). Kernels on stage2 need kexec, i have CONFIG_KEXEC=y CONFIG_KEXEC_FILE=y CONFIG_ARCH_HAS_KEXEC_PURGATORY=y # CONFIG_KEXEC_SIG is not set CONFIG_KEXEC_CORE=y -s1.sh and -s2.sh are docu-commented at the top; they can also be used to create the necessary environment. If i recall correctly cd /boot sh linit-init-s2.sh PATH-TO-BUSYBOX PATH-TO-CRYPTSETUP mount EFI-PARTITION cd EFI-PARTITION sh linit-init-s1.sh PATH-TO-BUSYBOX PATH-TO-CRYPTSETUP This only creates files and directories etc, and beat me if it does any harm otherwise. I use a specific naming scheme, the script reacts in particular on -old and -new to select sensible defaults. #?0|kent:/boot# ll ideapad*.efi -rwxr-xr-x 1 root root 10112672 Aug 26 18:44 ideapad-stage1.efi* -rw-r--r-- 1 root root 10120512 Sep 2 19:52 ideapad-6_1.efi -rw-r--r-- 1 root root 10120512 Sep 9 18:18 ideapad-6_1-old.efi -rw-r--r-- 1 root root 10121376 Sep 9 18:18 ideapad-6_1-new.efi That very kernel (series) is in fact shared in between several computers of a similar series, kent is the one i write this on: #?1|kent:/boot# v kent.sh #!/busybox.static sh #@ kent, step 1., via EFI. PART_ROOT=/dev/nvme0n1p? ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' INIT_S2=/boot/kent-2.sh . /linux-init-s1.sh #?0|kent:/boot# v kent-2.sh #@ kent, stage 2 KERNEL_ID=ideapad ^ restrict kernel selection for kexec: ^ [ "${k##*$KERNEL_ID-*.efi}" != "$k" ] || continue PART_SWAP=/dev/nvme0n1p6 SWAP_INPUT=y ^ you can have additional interactive random for swap encryption ^ (it is a lie i use that in practice) INITRD_PATH=/boot/.kent.initrd KEXEC_ARGS="--append=\"rtw88_pci.disable_aspm=1 rc.hostname=kent\"" FILE_CHECK='kent-direct.sh kent.sh ideapad-stage1.efi' SWITCH_ROOT=media/initrd Stage 2 config file is sourced automatically, so no need to source linux-init-s2.sh yourself (stage 2 uses switch_root). I also have a direct boot, but have not used it for long: #?0|kent:~# v /boot/kent-direct.sh #!/busybox.static sh #@ kent-direct, sole step 1., via EFI. PART_ROOT=/dev/nvme0n1p8 ROOT_DECRYPT='-t btrfs -o defaults,subvol=/crux/kent/root' PART_ROOT1=/dev/nvme0n1p8 ROOT_DECRYPT1='-t btrfs -o defaults,subvol=/crux/kent/root.old' PART_SWAP=/dev/nvme0n1p6 PIVOT_OLDROOT=media/initrd . /linux-init-s1.sh This does not use kexec, but only pivot_root's to the decrypted filesystem, starting sbin/init there. The kernel from EFI continues to run. It is pretty cool how i now can drive several computers easily simply by creating a shell script with some variable assignments. In fact the filesystem is shared in between multiple computers (with only selective partitions being truly unique, but still backed up everywhere), and all those text configs are in /boot. Only the VFAT EFI partition is truly system-specific (ie, in that it only holds the stage1 config file for $HOSTNAME). I like that very much, and hope Linux continues to be so flexible. (Not that one day you need systemd-early to get into Linux to get to systemd-later, or something.) The EFI boot is driven via efibootmgr(8) and has config like #@ /root/hosts/self/efiboot d=/dev/nvme0n1p1 maxno=1 b1=0x01 L1=kent l1=ideapad-stage1 u1='root='${d}' rootfstype=vfat init=/kent.sh' driven by /root/bin/efiboot.sh (shortened): #!/bin/sh : ${HOSTNAME:=$(uname -n)} : ${DBG:=} if [ -f /root/hosts/${HOSTNAME}/efiboot ]; then . /root/hosts/${HOSTNAME}/efiboot else echo >&2 "MISS /root/hosts/${HOSTNAME}/efiboot" exit 1 fi if [ "$1" = create ]; then obo=$(efibootmgr | grep -E ^BootOrder: | sed -E 's/^BootOrder:[[:space:]]*//') if [ -z "$obo" ]; then echo >&2 'Cannot determine previous boot order' exit 1 fi nbo= i=1 while [ $i -le $maxno ]; do eval ${DBG} efibootmgr -c -b \$b$i -L \\\"\$L$i\\\" -d \$d$i -l \\\"\$l$i.efi\\\" -u \\\"\$u$i\\\" #>/dev/null [ -n "$nbo" ] && nbo=$nbo, eval nbo=\$nbo\$b$i i=$((i + 1)) done ${DBG} efibootmgr -o $nbo,$obo #> /dev/null elif [ "$1" = delete ]; then i=1 while [ $i -le $maxno ]; do eval ${DBG} efibootmgr -B -b \$b$i #>/dev/null i=$((i + 1)) done else echo >&2 'Synsopsis: efiboot.sh create|delete' exit 1 fi ${DBG} efibootmgr -v -u This is of course git managed, so i can distribute it easily. (Further even, the entire ideapad series configurations differ in only 193 lines, and most of those are the different SSH keys. The rest are different device names, ie network, display, volume etc, disk partitions, host names.) I am pretty lucky i found a way to manage my systems like this. I have several git branches in root.git, linux.kconfig, bin, iwd.network, and one for each computer. I can then "git merge" selectively together what is necessary. For example my web vserver only needs bin and its own configuration, whereas the laptops also need iwd.network. Etc etc. Like this every computer only knows the selective minimum. The full git repo is on an encrypted volume stored away. And it is not Puppet or Ansible or any of those programs, it is nothing but git and its branches. But now i really left manually driven cryptsetup by far. Ciao! --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) -------------- next part -------------- A non-text attachment was scrubbed... Name: linux-init-s1.sh Type: application/x-sh Size: 4596 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: linux-init-s2.sh Type: application/x-sh Size: 7186 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: linux-init-lib.sh Type: application/x-sh Size: 3646 bytes Desc: not available URL: From arnold at skeeve.com Mon Sep 11 05:41:34 2023 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 10 Sep 2023 13:41:34 -0600 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <1qf3zQ-7oK-00@marmaro.de> References: <1qf3zQ-7oK-00@marmaro.de> Message-ID: <202309101941.38AJfYOc019233@freefriends.org> Hi. markus schnalke wrote: > Hoi, > > I just discovered that one of my favorite computer books about my > best liked programming language (besides C) releases in a second > edition. Does anyone know what the differences of 1st and 2nd > edition are? > > As the original book is almost perfect, the only rework and > extension direction I can think of is towards different > implementations like gawk, mawk, portability and such things. > > Does anyone know more about it? Maybe some inside information? ;-) > > meillo Inside information? As it happens, yes, I do have some. :-) (I was a reviewer.) [In the below, "awk" means Brian Kernighan's awk.] In the 36 (!) years since the first edition was published, awk has undergone, shall we say, a large number of small changes. These are listed in the FIXES file currently in the master branch of https://github.com/onetrueawk/awk. In addition, Brian Kernighan decided to add support for UTF-8 input, which is what awk now expects, and support for CSV input files when invoked with the --csv option. Furthermore, there is a new \u escape sequence which must be followed by 1-8 hexadecimal digits for specifying Unicode code points. The book itself has been carefully revised. The large second chapter which was a reference to the full language was moved to an appendix. Many of the example programs from the first edition were retained and updated, but there is also quite of lot of pleasing new material. There is mention of, and occasional comparison with, gawk, mawk and Ben Hoyt's GoAwk, but by and large the focus is on the authors' version. The new code is currently in the "csv" branch of the above Github repo. The maintainer is in the process of tidying up the repo (dealing with issues and pull requests) and will merge the csv branch into master sometime in the very near future. I'm told that the printed books with get to the publisher's warehouse towards the end of September. The book is available now on O'Reilly's Safari learning site (safari.oreilly.com) for anyone who has a subscription. Matching code (--csv and \u) are in gawk's master branch now. I will make a release this fall, after the new code has moved into master in BWK's awk. I heartily recommend the book; it is totally up to Brian Kernighan's usual very high standard. Enjoy, Arnold From norman at oclsc.org Mon Sep 11 05:59:15 2023 From: norman at oclsc.org (Norman Wilson) Date: Sun, 10 Sep 2023 15:59:15 -0400 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <202309101941.38AJfYOc019233@freefriends.org> References: <1qf3zQ-7oK-00@marmaro.de> <202309101941.38AJfYOc019233@freefriends.org> Message-ID: <54626B532B36A1FDDBC4AD6352A3FD77.for-standards-violators@oclsc.org> Thanks, both for this message and for all your hard work! From tuhs at tuhs.org Mon Sep 11 11:11:32 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Mon, 11 Sep 2023 01:11:32 +0000 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs Message-ID: Hello folks, I'm here today with a question that sprung off of some 3B20 research. When 1984 happened and ATTIS rose from the ashes of former Bell System computing efforts, presumably ATTIS received all IP rights from Western Electric for 3B processors, WE32000, and so on, and continued to sell related products through to the 3B2 line. Is this the case, is ATTIS the formal recipient of both computing software *and* hardware IPs after the breakup? Given that, plus subsequent market flow, "old AT&T" scooped up and paraded around in effigy by SBC, other old Bell stuff cannibalized by other RBOCs, spinoffs of stuff to Novell, then Caldera/SCO on the other side...who all wound up with the hardware IPs? The story as it "concludes" concerning UNIX is of course tied up in all the subsequent lawsuits, what with Novell and Caldera conflicts on ownership, transfer to the Open Group, so on and so forth, and SCO and progeny wind up with the Sys V "trunk." Is there a clear, current owner of these WECo hardware IPs, or have those waters grown even murkier than those of UNIX in the times after AT&T proper? Thanks everyone! - Matt G. P.S. As an aside (even though it's the more directly UNIX thing...) is anything after SVR4 developments that would've involved the same folks as were working up to that point in the USL group? Or did the transfer of System V to Novell also involve their own in house folks starting to take it over, then over to SCO, is there anything post SVR4 (4.2, 5, UnixWare stuff) that would even remotely be considered the logical next step by the same folks that engineered SVR4, or was it basically just another face in the crowd of "UNIX " when USL wasn't involved anymore? Probably not the first time this has been asked either so to a finer point I'm basically fishing for whether anything post the initial SVR4 releases in the early 90s is generally considered "pure" in any way or if the Bell streams pretty much terminate with Research V10 and SVR4, (and IX) at the turn of the 90s. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan at alanlee.org Mon Sep 11 11:35:46 2023 From: alan at alanlee.org (alan at alanlee.org) Date: Sun, 10 Sep 2023 21:35:46 -0400 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs In-Reply-To: References: Message-ID: <2884e81581b595ec2c4fc11fc6b2a8ab@alanlee.org> My understanding is all the WE IP was retained through the Alcatel-Lucent mergers and is now owned by Nokia. That would include all the 3Bx systems and WE32k. -A On 2023-09-10 21:11, segaloco via TUHS wrote: > Hello folks, I'm here today with a question that sprung off of some > 3B20 research. > > When 1984 happened and ATTIS rose from the ashes of former Bell System > computing efforts, presumably ATTIS received all IP rights from Western > Electric for 3B processors, WE32000, and so on, and continued to sell > related products through to the 3B2 line. Is this the case, is ATTIS > the formal recipient of both computing software *and* hardware IPs > after the breakup? > > Given that, plus subsequent market flow, "old AT&T" scooped up and > paraded around in effigy by SBC, other old Bell stuff cannibalized by > other RBOCs, spinoffs of stuff to Novell, then Caldera/SCO on the other > side...who all wound up with the hardware IPs? The story as it > "concludes" concerning UNIX is of course tied up in all the subsequent > lawsuits, what with Novell and Caldera conflicts on ownership, transfer > to the Open Group, so on and so forth, and SCO and progeny wind up with > the Sys V "trunk." > > Is there a clear, current owner of these WECo hardware IPs, or have > those waters grown even murkier than those of UNIX in the times after > AT&T proper? > > Thanks everyone! > > - Matt G. > > P.S. As an aside (even though it's the more directly UNIX thing...) is > anything after SVR4 developments that would've involved the same folks > as were working up to that point in the USL group? Or did the transfer > of System V to Novell also involve their own in house folks starting to > take it over, then over to SCO, is there anything post SVR4 (4.2, 5, > UnixWare stuff) that would even remotely be considered the logical next > step by the same folks that engineered SVR4, or was it basically just > another face in the crowd of "UNIX " when USL wasn't involved > anymore? Probably not the first time this has been asked either so to a > finer point I'm basically fishing for whether anything post the initial > SVR4 releases in the early 90s is generally considered "pure" in any > way or if the Bell streams pretty much terminate with Research V10 and > SVR4, (and IX) at the turn of the 90s. -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Mon Sep 11 11:43:27 2023 From: clemc at ccc.com (Clem Cole) Date: Sun, 10 Sep 2023 21:43:27 -0400 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs In-Reply-To: References: Message-ID: On Sun, Sep 10, 2023 at 9:11 PM segaloco via TUHS wrote: > Is there a clear, current owner of these WECo hardware IPs, or have those > waters grown even murkier than those of UNIX in the times after AT&T proper? I have never seen an unofficial, much less an official, reckoning, but if you discover/unearth something, it will be interesting to read. That said, you left out one piece of history. Please remember that AT&T bought NCR in the mid-1980s (and eventually spun it off a few years later). The UNIX HW development was moved into the new division of the old NCR, including the 3B series work, the WE32000, and some other semiconductor IPs. FWIW: That occurred when I consulted for NCR's Chief Architect (Lee Hovel) in the mid-late 80s (I did some of the analysis for Lee on what IP was there). But that all settled out after my contract expired, so I don't know how it finally settled - other than I'm reasonably sure that most of the 3B and chip development reported up through an ex-NCR exec after purchase. Those teams and their associated IP were folded into things like the old NCR semi-conductor, NCR Computer, *etc*.. IIRC Also, a few NCR communications products were moved out of the old NCR team and into the old WE folks. So .... I would not be surprised if when NCR was later spun back out, some of the old AT&T IP (such as the computer HW and chip IP) went with them, just as when Novell was sold the UNIX SW IP. Of course, later, Lucent, ney Alcatel, ney Nokia - got the communications IP. ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan at alanlee.org Mon Sep 11 12:02:55 2023 From: alan at alanlee.org (alan at alanlee.org) Date: Sun, 10 Sep 2023 22:02:55 -0400 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs In-Reply-To: <2884e81581b595ec2c4fc11fc6b2a8ab@alanlee.org> References: <2884e81581b595ec2c4fc11fc6b2a8ab@alanlee.org> Message-ID: <689bc9fc3e5cc8d6772b98f12b96ba92@alanlee.org> Let me clarify. A few of us have reached out to various people about both 3B2 and WE32k information. The main response them has been it's still in-use to varying degrees by customers. I assume through emulation to keep legacy systems running. And they cannot release any details. The two people that would know the most about the status of that IP is William Caughlin [1] at the AT&T Archives and Ed Eckert - Archivist at Nokia Bell Labs. -Alan On 2023-09-10 21:35, alan at alanlee.org wrote: > My understanding is all the WE IP was retained through the > Alcatel-Lucent mergers and is now owned by Nokia. That would include > all the 3Bx systems and WE32k. > > -A Links: ------ [1] https://www.linkedin.com/in/william-caughlin-59571546/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Mon Sep 11 12:57:50 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Mon, 11 Sep 2023 02:57:50 +0000 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs In-Reply-To: References: Message-ID: <8cLEEhdmkOHtKBxAfMVu6TQNyw3XQ6paWadcOywtb43zx--LshnfV0tYzcIqZGf_IBODnckLjTG-TETOHBWq8sOABYbJBXlzaYDahigXpm4=@protonmail.com> Whoops, my bad, hadn't considered NCR in this train of thought. And it sounds like the Nokia stream is a compelling direction, what with the 3B20 emulation and all that, it was a telecom processor after all. Glad to see I'm not just stumped, that it can be a little unclear. - Matt G. P.S. I realize my last P.S. may be more opinion oriented and/or controversial..just wanna recognize that oversight, no harm intended, different discussion for a different time ------- Original Message ------- On Sunday, September 10th, 2023 at 6:43 PM, Clem Cole wrote: > On Sun, Sep 10, 2023 at 9:11 PM segaloco via TUHS wrote: > >> Is there a clear, current owner of these WECo hardware IPs, or have those waters grown even murkier than those of UNIX in the times after AT&T proper? > > I have never seen an unofficial, much less an official, reckoning, but if you discover/unearth something, it will be interesting to read. > > That said, you left out one piece of history. Please remember that AT&T bought NCR in the mid-1980s (and eventually spun it off a few years later). The UNIX HW development was moved into the new division of the old NCR, including the 3B series work, the WE32000, and some other semiconductor IPs. > > FWIW: That occurred when I consulted for NCR's Chief Architect (Lee Hovel) in the mid-late 80s (I did some of the analysis for Lee on what IP was there). But that all settled out after my contract expired, so I don't know how it finally settled - other than I'm reasonably sure that most of the 3B and chip development reported up through an ex-NCR exec after purchase. Those teams and their associated IP were folded into things like the old NCR semi-conductor, NCR Computer, etc.. IIRC Also, a few NCR communications products were moved out of the old NCR team and into the old WE folks. > > So .... I would not be surprised if when NCR was later spun back out, some of the old AT&T IP (such as the computer HW and chip IP) went with them, just as when Novell was sold the UNIX SW IP. Of course, later, Lucent, ney Alcatel, ney Nokia - got the communications IP. > > ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.bowling at kev009.com Mon Sep 11 13:17:34 2023 From: kevin.bowling at kev009.com (Kevin Bowling) Date: Sun, 10 Sep 2023 20:17:34 -0700 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs In-Reply-To: <2884e81581b595ec2c4fc11fc6b2a8ab@alanlee.org> References: <2884e81581b595ec2c4fc11fc6b2a8ab@alanlee.org> Message-ID: On Sun, Sep 10, 2023 at 6:36 PM wrote: > > My understanding is all the WE IP was retained through the Alcatel-Lucent mergers and is now owned by Nokia. That would include all the 3Bx systems and WE32k. I agree with this. Lucent was supporting 3B systems at least into the millennium, they would certainly have some rights to do so. WE32k I've less certainty. > -A > > On 2023-09-10 21:11, segaloco via TUHS wrote: > > Hello folks, I'm here today with a question that sprung off of some 3B20 research. > > When 1984 happened and ATTIS rose from the ashes of former Bell System computing efforts, presumably ATTIS received all IP rights from Western Electric for 3B processors, WE32000, and so on, and continued to sell related products through to the 3B2 line. Is this the case, is ATTIS the formal recipient of both computing software *and* hardware IPs after the breakup? > > Given that, plus subsequent market flow, "old AT&T" scooped up and paraded around in effigy by SBC, other old Bell stuff cannibalized by other RBOCs, spinoffs of stuff to Novell, then Caldera/SCO on the other side...who all wound up with the hardware IPs? The story as it "concludes" concerning UNIX is of course tied up in all the subsequent lawsuits, what with Novell and Caldera conflicts on ownership, transfer to the Open Group, so on and so forth, and SCO and progeny wind up with the Sys V "trunk." > > Is there a clear, current owner of these WECo hardware IPs, or have those waters grown even murkier than those of UNIX in the times after AT&T proper? > > Thanks everyone! > > - Matt G. > > P.S. As an aside (even though it's the more directly UNIX thing...) is anything after SVR4 developments that would've involved the same folks as were working up to that point in the USL group? Or did the transfer of System V to Novell also involve their own in house folks starting to take it over, then over to SCO, is there anything post SVR4 (4.2, 5, UnixWare stuff) that would even remotely be considered the logical next step by the same folks that engineered SVR4, or was it basically just another face in the crowd of "UNIX " when USL wasn't involved anymore? Probably not the first time this has been asked either so to a finer point I'm basically fishing for whether anything post the initial SVR4 releases in the early 90s is generally considered "pure" in any way or if the Bell streams pretty much terminate with Research V10 and SVR4, (and IX) at the turn of the 90s. > > From clemc at ccc.com Mon Sep 11 13:37:39 2023 From: clemc at ccc.com (Clem Cole) Date: Sun, 10 Sep 2023 23:37:39 -0400 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs In-Reply-To: <8cLEEhdmkOHtKBxAfMVu6TQNyw3XQ6paWadcOywtb43zx--LshnfV0tYzcIqZGf_IBODnckLjTG-TETOHBWq8sOABYbJBXlzaYDahigXpm4=@protonmail.com> References: <8cLEEhdmkOHtKBxAfMVu6TQNyw3XQ6paWadcOywtb43zx--LshnfV0tYzcIqZGf_IBODnckLjTG-TETOHBWq8sOABYbJBXlzaYDahigXpm4=@protonmail.com> Message-ID: My point was it was not clear at all other than then NCR deal would have added to the confusion. For instance I know that one of my brothers who was doing communications chips at ATT Allentown PA that we originally for the 3B but later licensed to Apple (became FireWire I think) stayed with ATT but one or more of his buddies ended up moving to Ohio to become part of NCR because that was were more SCSI and FC chips were being done. So like lots of big firms things were added and subtracted as different deals were made. So I’m not sure there is an easy linear progression and while Nokia at this point clearly owns some of this but don’t be surprised that like Unix - there are multiple firms with IP claims and unless something happens where it takes a court to sort it out, I’m not so sure any of here is going to be able to give you a definitive answer. Sent from a handheld expect more typos than usual On Sun, Sep 10, 2023 at 10:58 PM segaloco via TUHS wrote: > Whoops, my bad, hadn't considered NCR in this train of thought. > > And it sounds like the Nokia stream is a compelling direction, what with > the 3B20 emulation and all that, it was a telecom processor after all. Glad > to see I'm not just stumped, that it can be a little unclear. > > - Matt G. > > P.S. I realize my last P.S. may be more opinion oriented and/or > controversial..just wanna recognize that oversight, no harm intended, > different discussion for a different time > > ------- Original Message ------- > > On Sunday, September 10th, 2023 at 6:43 PM, Clem Cole > wrote: > > > > On Sun, Sep 10, 2023 at 9:11 PM segaloco via TUHS wrote: > >> Is there a clear, current owner of these WECo hardware IPs, or have those >> waters grown even murkier than those of UNIX in the times after AT&T proper? > > I have never seen an unofficial, much less an official, reckoning, but if > you discover/unearth something, it will be interesting to read. > > That said, you left out one piece of history. Please remember that AT&T > bought NCR in the mid-1980s (and eventually spun it off a few years later). > The UNIX HW development was moved into the new division of the old NCR, > including the 3B series work, the WE32000, and some other semiconductor IPs. > > FWIW: That occurred when I consulted for NCR's Chief Architect (Lee Hovel) > in the mid-late 80s (I did some of the analysis for Lee on what IP was > there). But that all settled out after my contract expired, so I don't know > how it finally settled - other than I'm reasonably sure that most of the 3B > and chip development reported up through an ex-NCR exec after purchase. > Those teams and their associated IP were folded into things like the old > NCR semi-conductor, NCR Computer, *etc*.. IIRC Also, a few NCR > communications products were moved out of the old NCR team and into the old > WE folks. > > So .... I would not be surprised if when NCR was later spun back out, some > of the old AT&T IP (such as the computer HW and chip IP) went with them, > just as when Novell was sold the UNIX SW IP. Of course, later, Lucent, ney > Alcatel, ney Nokia - got the communications IP. > > ᐧ > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tytso at mit.edu Mon Sep 11 14:10:59 2023 From: tytso at mit.edu (Theodore Ts'o) Date: Mon, 11 Sep 2023 00:10:59 -0400 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230908233854.Xni_j%steffen@sdaoden.eu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> <20230908233854.Xni_j%steffen@sdaoden.eu> Message-ID: <20230911041059.GH701295@mit.edu> On Sat, Sep 09, 2023 at 01:38:54AM +0200, Steffen Nurpmeso wrote: > You know, things change, and if you do not follow closely, you > stand in the rain. I am not a paid Linux engineer that follows > this rapidly moving target in the end. > For example the (no longer) new random developer chose to disable > feeding entropy via /dev/urandom, here (distribution) still is > > # Load random seed > /bin/cat /var/lib/urandom/seed > /dev/urandom > > for almost two decades (it is a rather young one), but the code > path was mutilated (i read the kernel source once he had rewritten > that to be blake2/some 32-byte block thing based), now one needs > to use some ioctl interface fwiw. Huh? That's not correct. You can still introduce entropy into the random pool by writing to /dev/urandom or /dev/random. It is true that there has been some changes to the design of /dev/random, but I assure you as the original /dev/random developer, was consulted and involved in the design discussions. Anyway, this is off-topic for TUHS, but if you have specific questions/complaints, feel free to address them to me and Jason. Your comments do make me suspect that there are some fundamental misunderstandings at work. Cheers, - Ted From meillo at marmaro.de Mon Sep 11 15:59:44 2023 From: meillo at marmaro.de (markus schnalke) Date: Mon, 11 Sep 2023 07:59:44 +0200 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <202309101941.38AJfYOc019233@freefriends.org> References: <1qf3zQ-7oK-00@marmaro.de> <202309101941.38AJfYOc019233@freefriends.org> Message-ID: <1qfZxY-3Sk-00@marmaro.de> Hoi, thanks a lot for your helpful answer. This was the kind of answer I was wishing to get. It's hard to resist getting the book, after your favorable words. ;-) meillo [2023-09-10 21:41] arnold at skeeve.com > > Hi. > > markus schnalke wrote: > > > Hoi, > > > > I just discovered that one of my favorite computer books about my > > best liked programming language (besides C) releases in a second > > edition. Does anyone know what the differences of 1st and 2nd > > edition are? > > > > As the original book is almost perfect, the only rework and > > extension direction I can think of is towards different > > implementations like gawk, mawk, portability and such things. > > > > Does anyone know more about it? Maybe some inside information? ;-) > > > > meillo > > Inside information? As it happens, yes, I do have some. :-) > (I was a reviewer.) > > [In the below, "awk" means Brian Kernighan's awk.] > > In the 36 (!) years since the first edition was published, awk > has undergone, shall we say, a large number of small changes. These > are listed in the FIXES file currently in the master branch of > https://github.com/onetrueawk/awk. > > In addition, Brian Kernighan decided to add support for UTF-8 input, > which is what awk now expects, and support for CSV input files when > invoked with the --csv option. Furthermore, there is a new \u escape > sequence which must be followed by 1-8 hexadecimal digits for specifying > Unicode code points. > > The book itself has been carefully revised. The large second chapter > which was a reference to the full language was moved to an appendix. > Many of the example programs from the first edition were retained > and updated, but there is also quite of lot of pleasing new material. > > There is mention of, and occasional comparison with, gawk, mawk and > Ben Hoyt's GoAwk, but by and large the focus is on the authors' version. > > The new code is currently in the "csv" branch of the above Github > repo. The maintainer is in the process of tidying up the repo (dealing > with issues and pull requests) and will merge the csv branch into > master sometime in the very near future. > > I'm told that the printed books with get to the publisher's warehouse > towards the end of September. The book is available now on O'Reilly's > Safari learning site (safari.oreilly.com) for anyone who has a > subscription. > > Matching code (--csv and \u) are in gawk's master branch now. I will > make a release this fall, after the new code has moved into master > in BWK's awk. > > I heartily recommend the book; it is totally up to Brian Kernighan's > usual very high standard. > > Enjoy, > > Arnold > From stuff at riddermarkfarm.ca Mon Sep 11 23:52:55 2023 From: stuff at riddermarkfarm.ca (Stuff Received) Date: Mon, 11 Sep 2023 09:52:55 -0400 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <1qfZxY-3Sk-00@marmaro.de> References: <1qf3zQ-7oK-00@marmaro.de> <202309101941.38AJfYOc019233@freefriends.org> <1qfZxY-3Sk-00@marmaro.de> Message-ID: <758df571-6151-d5c8-642e-b3fcc83aa4f2@riddermarkfarm.ca> On 2023-09-11 01:59, markus schnalke wrote: > Hoi, > > thanks a lot for your helpful answer. This was the kind of answer > I was wishing to get. More tidbits here: https://www.awk.dev/ S. > It's hard to resist getting the book, after your favorable words. > ;-) > > > meillo > > > > [2023-09-10 21:41] arnold at skeeve.com >> >> Hi. >> >> markus schnalke wrote: >> >>> Hoi, >>> >>> I just discovered that one of my favorite computer books about my >>> best liked programming language (besides C) releases in a second >>> edition. Does anyone know what the differences of 1st and 2nd >>> edition are? >>> >>> As the original book is almost perfect, the only rework and >>> extension direction I can think of is towards different >>> implementations like gawk, mawk, portability and such things. >>> >>> Does anyone know more about it? Maybe some inside information? ;-) >>> >>> meillo >> >> Inside information? As it happens, yes, I do have some. :-) >> (I was a reviewer.) >> >> [In the below, "awk" means Brian Kernighan's awk.] >> >> In the 36 (!) years since the first edition was published, awk >> has undergone, shall we say, a large number of small changes. These >> are listed in the FIXES file currently in the master branch of >> https://github.com/onetrueawk/awk. >> >> In addition, Brian Kernighan decided to add support for UTF-8 input, >> which is what awk now expects, and support for CSV input files when >> invoked with the --csv option. Furthermore, there is a new \u escape >> sequence which must be followed by 1-8 hexadecimal digits for specifying >> Unicode code points. >> >> The book itself has been carefully revised. The large second chapter >> which was a reference to the full language was moved to an appendix. >> Many of the example programs from the first edition were retained >> and updated, but there is also quite of lot of pleasing new material. >> >> There is mention of, and occasional comparison with, gawk, mawk and >> Ben Hoyt's GoAwk, but by and large the focus is on the authors' version. >> >> The new code is currently in the "csv" branch of the above Github >> repo. The maintainer is in the process of tidying up the repo (dealing >> with issues and pull requests) and will merge the csv branch into >> master sometime in the very near future. >> >> I'm told that the printed books with get to the publisher's warehouse >> towards the end of September. The book is available now on O'Reilly's >> Safari learning site (safari.oreilly.com) for anyone who has a >> subscription. >> >> Matching code (--csv and \u) are in gawk's master branch now. I will >> make a release this fall, after the new code has moved into master >> in BWK's awk. >> >> I heartily recommend the book; it is totally up to Brian Kernighan's >> usual very high standard. >> >> Enjoy, >> >> Arnold >> From paul.winalski at gmail.com Tue Sep 12 01:34:40 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Mon, 11 Sep 2023 11:34:40 -0400 Subject: [TUHS] Current Ownership of 3B/WECo Computer IPs In-Reply-To: References: <8cLEEhdmkOHtKBxAfMVu6TQNyw3XQ6paWadcOywtb43zx--LshnfV0tYzcIqZGf_IBODnckLjTG-TETOHBWq8sOABYbJBXlzaYDahigXpm4=@protonmail.com> Message-ID: Regarding the possible Western Electric -> Lucent -> Nokia IP path, has anyone tried contacting Nokia's legal department and asking whether they think they own the rights to the 3B/WECo computer IP, and if not, do they know who does (or might) own it? -Paul W. From mis at seiden.com Tue Sep 12 03:46:22 2023 From: mis at seiden.com (Mark Seiden) Date: Mon, 11 Sep 2023 13:46:22 -0400 Subject: [TUHS] nassau smelting was Re: Current Ownership of 3B/WECo Computer IPs In-Reply-To: References: <8cLEEhdmkOHtKBxAfMVu6TQNyw3XQ6paWadcOywtb43zx--LshnfV0tYzcIqZGf_IBODnckLjTG-TETOHBWq8sOABYbJBXlzaYDahigXpm4=@protonmail.com> Message-ID: <9ADDEFE7-F5DC-44D9-9FC7-142D31890492@seiden.com> hi, old friends (and i mean that both figuratively and literally) the recent/continuing brouhaha involving lead sheathed cables made me wonder about nassau smelting and refining in staten island (a sub of WEco which ended up with Lucent) (and apparently another location at W. 29th st which was a superfund site for a while.) wonderful chaplainesque photo: https://www.facebook.com/classicstatenisland/photos/a.286586221935407/516688818925145/?type=3 also regarding the Staten Island location, quoting from https://www.silive.com/news/2020/02/former-nassau-smelting-site-sells-for-306m.html "The cleanup was to entail covering 450,000 cubic yards of contaminated soil with a thick, unbreakable plastic liner, as well as layers of clean soil.” (hah, what could go wrong with that?) > On Sep 11, 2023, at 11:34 AM, Paul Winalski wrote: > > Regarding the possible Western Electric -> Lucent -> Nokia IP path, > has anyone tried contacting Nokia's legal department and asking > whether they think they own the rights to the 3B/WECo computer IP, and > if not, do they know who does (or might) own it? > > -Paul W. -------------- next part -------------- An HTML attachment was scrubbed... URL: From douglas.mcilroy at dartmouth.edu Tue Sep 12 05:58:20 2023 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Mon, 11 Sep 2023 15:58:20 -0400 Subject: [TUHS] nassau smelting was Re: Current Ownership of 3B/WECo Computer IPs In-Reply-To: <9ADDEFE7-F5DC-44D9-9FC7-142D31890492@seiden.com> References: <8cLEEhdmkOHtKBxAfMVu6TQNyw3XQ6paWadcOywtb43zx--LshnfV0tYzcIqZGf_IBODnckLjTG-TETOHBWq8sOABYbJBXlzaYDahigXpm4=@protonmail.com> <9ADDEFE7-F5DC-44D9-9FC7-142D31890492@seiden.com> Message-ID: Way off topic, but too nostalgic to pass up. I was involved in after-hours training courses and persuaded Bob Morris to organize the first one ever to be held off campus, "Bell System with field trips". The highlight of the series was Nassau Smelting and Refining. They proudly told us about their new environmental consciousness: aqua regia (used to reclaim gold) was now being adjusted to pH 7 before being dumped into the Kill van Kull, and stack emissions of lead had been reduced from hundreds of pounds per year to eight. The scene was almost straight out of Agricola. An enormous steel jousting pole mounted on a backhoe was used to shove scrap copper and live-cut tree trunks into a ferocious green-flaming furnace. Men in moon suits and respirators puddled slag floating on open vats of molten lead. A Dickensian crone snipped gold tips off the old relays cradled in her lap. Clothes, including shoes, exchanged at the door of the gloomy gold room, were collected periodically and thrown into the aqua regia pots to extract every last milligram. The only process that really deviated from Agricola was pyrolysis, for reclaiming modern cable cladding. But he surely would have been impressed by the mechanism for casting continuous copper bar and collecting it on a giant spool. They delighted in telling about the time the spool stopped turning while the copper feed continued, filling the hall with an enormous tangle of 1" copper stock. Doug On Mon, Sep 11, 2023 at 1:47 PM Mark Seiden wrote: > > hi, old friends (and i mean that both figuratively and literally) > > the recent/continuing brouhaha involving lead sheathed cables made me wonder about > nassau smelting and refining in staten island (a sub of WEco which ended up with Lucent) > (and apparently another location at W. 29th st which was a superfund site for a while.) > > wonderful chaplainesque photo: > > https://www.facebook.com/classicstatenisland/photos/a.286586221935407/516688818925145/?type=3 > > also regarding the Staten Island location, quoting from > > https://www.silive.com/news/2020/02/former-nassau-smelting-site-sells-for-306m.html > > "The cleanup was to entail covering 450,000 cubic yards of contaminated soil with a thick, unbreakable plastic liner, as well as layers of clean soil.” > > (hah, what could go wrong with that?) > > On Sep 11, 2023, at 11:34 AM, Paul Winalski wrote: > > Regarding the possible Western Electric -> Lucent -> Nokia IP path, > has anyone tried contacting Nokia's legal department and asking > whether they think they own the rights to the 3B/WECo computer IP, and > if not, do they know who does (or might) own it? > > -Paul W. > > From douglas.mcilroy at dartmouth.edu Tue Sep 12 07:06:42 2023 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Mon, 11 Sep 2023 17:06:42 -0400 Subject: [TUHS] nassau smelting was Re: Current Ownership of 3B/WECo Computer IPs In-Reply-To: References: <8cLEEhdmkOHtKBxAfMVu6TQNyw3XQ6paWadcOywtb43zx--LshnfV0tYzcIqZGf_IBODnckLjTG-TETOHBWq8sOABYbJBXlzaYDahigXpm4=@protonmail.com> <9ADDEFE7-F5DC-44D9-9FC7-142D31890492@seiden.com> Message-ID: > i’m having trouble identifying “Agricola”. See WIkipedia on "De re metallica", or for the real thing https://www.gutenberg.org/files/38015/38015-h/38015-h.htm On Mon, Sep 11, 2023 at 4:29 PM Mark Seiden wrote: > > i’m so happy you replied. > > have you considered a memoir? > > your writing is very vivid (and literate). > > but i’m having trouble identifying “Agricola”. > > the strategy board game? the roman emperor? the farm to table restaurant in princeton? > > > > > > > > On Sep 11, 2023, at 3:58 PM, Douglas McIlroy wrote: > > > > Way off topic, but too nostalgic to pass up. I was involved in > > after-hours training courses and persuaded Bob Morris to organize the > > first one ever to be held off campus, "Bell System with field trips". > > The highlight of the series was Nassau Smelting and Refining. They > > proudly told us about their new environmental consciousness: aqua > > regia (used to reclaim gold) was now being adjusted to pH 7 before > > being dumped into the Kill van Kull, and stack emissions of lead had > > been reduced from hundreds of pounds per year to eight. > > > > The scene was almost straight out of Agricola. An enormous steel > > jousting pole mounted on a backhoe was used to shove scrap copper and > > live-cut tree trunks into a ferocious green-flaming furnace. Men in > > moon suits and respirators puddled slag floating on open vats of > > molten lead. A Dickensian crone snipped gold tips off the old relays > > cradled in her lap. Clothes, including shoes, exchanged at the door of > > the gloomy gold room, were collected periodically and thrown into the > > aqua regia pots to extract every last milligram. > > > > The only process that really deviated from Agricola was pyrolysis, for > > reclaiming modern cable cladding. But he surely would have been > > impressed by the mechanism for casting continuous copper bar and > > collecting it on a giant spool. They delighted in telling about the > > time the spool stopped turning while the copper feed continued, > > filling the hall with an enormous tangle of 1" copper stock. > > > > Doug > > > > > > On Mon, Sep 11, 2023 at 1:47 PM Mark Seiden wrote: > >> > >> hi, old friends (and i mean that both figuratively and literally) > >> > >> the recent/continuing brouhaha involving lead sheathed cables made me wonder about > >> nassau smelting and refining in staten island (a sub of WEco which ended up with Lucent) > >> (and apparently another location at W. 29th st which was a superfund site for a while.) > >> > >> wonderful chaplainesque photo: > >> > >> https://www.facebook.com/classicstatenisland/photos/a.286586221935407/516688818925145/?type=3 > >> > >> also regarding the Staten Island location, quoting from > >> > >> https://www.silive.com/news/2020/02/former-nassau-smelting-site-sells-for-306m.html > >> > >> "The cleanup was to entail covering 450,000 cubic yards of contaminated soil with a thick, unbreakable plastic liner, as well as layers of clean soil.” > >> > >> (hah, what could go wrong with that?) > >> > >> On Sep 11, 2023, at 11:34 AM, Paul Winalski wrote: > >> > >> Regarding the possible Western Electric -> Lucent -> Nokia IP path, > >> has anyone tried contacting Nokia's legal department and asking > >> whether they think they own the rights to the 3B/WECo computer IP, and > >> if not, do they know who does (or might) own it? > >> > >> -Paul W. > >> > >> > From steffen at sdaoden.eu Tue Sep 12 08:05:47 2023 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Tue, 12 Sep 2023 00:05:47 +0200 Subject: [TUHS] Unix install & "standalone" package In-Reply-To: <20230911041059.GH701295@mit.edu> References: <9A989054DE79CE5059CBA74797391E39.for-standards-violators@oclsc.org> <20230904221059.sF2G0%steffen@sdaoden.eu> <20230905155301.mIziN%steffen@sdaoden.eu> <20230908233854.Xni_j%steffen@sdaoden.eu> <20230911041059.GH701295@mit.edu> Message-ID: <20230911220547.atSg7%steffen@sdaoden.eu> Theodore Ts'o wrote in <20230911041059.GH701295 at mit.edu>: |On Sat, Sep 09, 2023 at 01:38:54AM +0200, Steffen Nurpmeso wrote: |> You know, things change, and if you do not follow closely, you |> stand in the rain. I am not a paid Linux engineer that follows |> this rapidly moving target in the end. |> For example the (no longer) new random developer chose to disable |> feeding entropy via /dev/urandom, here (distribution) still is |> |> # Load random seed |> /bin/cat /var/lib/urandom/seed > /dev/urandom |> |> for almost two decades (it is a rather young one), but the code |> path was mutilated (i read the kernel source once he had rewritten |> that to be blake2/some 32-byte block thing based), now one needs |> to use some ioctl interface fwiw. | |Huh? That's not correct. You can still introduce entropy into the |random pool by writing to /dev/urandom or /dev/random. It is true |that there has been some changes to the design of /dev/random, but I |assure you as the original /dev/random developer, was consulted and |involved in the design discussions. Ah, lesser and lesser freedom for man! That is the truth! Neither was (last i looked) that counted no more to unlock man from the chokehold of missing entropy, nor do newer kernels then decrement entropy when you read random. |Anyway, this is off-topic for TUHS, but if you have specific |questions/complaints, feel free to address them to me and Jason. Your |comments do make me suspect that there are some fundamental |misunderstandings at work. Surely for the latter. So much i give in on this song line. Greetings. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From tuhs at tuhs.org Tue Sep 12 12:38:50 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Tue, 12 Sep 2023 02:38:50 +0000 Subject: [TUHS] Some BSP Numbers Related to 3B/1A "Attached Processor Interface" Message-ID: Hello, today I've received a 3B/1A "Attached Processor Interface" or "API" (as I will refer to it in this email, note API will not mean Application Programming Interface in my contributions to this discussion) pamphlet, published by WECo in November 1981, concerning installation and maintenance of the API. From the introductory page: > This lesson is a self-paced package that can be administered individually or in a classroom environment. There is a helpful glossary in the back of this booklet that you may use while going through the material. > > At the conclusion of this lesson, the installer will be able to: > > - State the purpose of the API as it applies to the 3B Processor. > > - Identify the different types of circuit packs used in the API. > > - List the number of API diagnostic phases. > > - Name the API circuit pack that interfaces the Dual Serial Channel in the 3B Processor. Among the many pages (which I'll be doing a formal scan on in the future) are some references to BSPs and schematic drawings relevant to this information, I wanted to get that information out in the email archive now so the bibliographic references are preserved somewhere: 3B Processor: 254-001-031 - Test Equipment List 254-301-000 - System Documentation 254-301-005 - General Description 254-301-115 - TTY System 254-301-010 - Central Control 254-301-020 - Power Systems 254-301-100 - Input Output Interfaces 254-301-105 - Input Output Processor 254-301-110 - Input Output Processor Peripheral Controller 254-301-200 - Main Store 254-301-210 - Moving Head Disk Drive 254-301-215 - Disk File Controller 254-301-220 - Magnetic Tape System 254-301-800 - Emergency Action Procedures 254-301-809 - Acceptance Test Plan 254-301-811 - Task Oriented Practice-Routine 254-301-812 - Task Oriented Practice-Trouble Clearing 254-341-000 - DMERT 254-341-220 - MTCE Management and Diagnostics API: 234-100-020 - APS Description and Theory 254-201-003 - API Frame Theory 254-280-114 - APS Software Description 254-281-030 - API Growth Misc: 234-153-055 - File Store Relocation 3B Schematics: SD-4C050-01 - 3B Control Frame, DMA Unit, CC Unit, MASM 0, MASM 1, Power Unit, Cooling Unit, Filter Unit SD-4C059-02 - 3B Peripheral Control Frame SD-4C052-01 - IOP Basic SD-4C049-01 - IOP Growth SD-4C051-01 - DFC Unit SD-4C056-01 - Disk Inverter Frame SD-4C058-01 - Tape Transport Frame SD-4C057-01 - Inverter Unit SD-4C070-02 - Inverter Control Unit SD-4C054-01 - Duct & Cabling SD-4C053-01 - Power Distributing, AC Requirements SD-4C069-01 - Micro Level Test Set SD-4C065-01 - Port Switch API Schematics: SD-5A056-01 - API Frame SD-5A055-01 - API Unit There are then references to a few "Handbooks": HB 309 - 3B Proc HB 263(A) - API ITS (Mainly mentions tests and diagnostic stuff) HB 264(A) - API Generic (Not sure what ITS and Generic distinguish here) Finally, a bit of fun. Towards the back of the pamphlet is a small crossword puzzle concerning the API: https://i.imgur.com/0pLcK9p.jpg Answers to come when I do a scan of this. If anyone has any particular information they want me to try and find in it sooner rather than later just let me know, it's 88 pages but small form. - Matt G. From kevin.bowling at kev009.com Tue Sep 12 12:50:30 2023 From: kevin.bowling at kev009.com (Kevin Bowling) Date: Mon, 11 Sep 2023 19:50:30 -0700 Subject: [TUHS] Some BSP Numbers Related to 3B/1A "Attached Processor Interface" In-Reply-To: References: Message-ID: On Mon, Sep 11, 2023 at 7:39 PM segaloco via TUHS wrote: > > Hello, today I've received a 3B/1A "Attached Processor Interface" or "API" (as I will refer to it in this email, note API will not mean Application Programming Interface in my contributions to this discussion) pamphlet, published by WECo in November 1981, concerning installation and maintenance of the API. From the introductory page: > > > This lesson is a self-paced package that can be administered individually or in a classroom environment. There is a helpful glossary in the back of this booklet that you may use while going through the material. > > > > At the conclusion of this lesson, the installer will be able to: > > > > - State the purpose of the API as it applies to the 3B Processor. > > > > - Identify the different types of circuit packs used in the API. > > > > - List the number of API diagnostic phases. > > > > - Name the API circuit pack that interfaces the Dual Serial Channel in the 3B Processor. > > Among the many pages (which I'll be doing a formal scan on in the future) are some references to BSPs and schematic drawings relevant to this information, I wanted to get that information out in the email archive now so the bibliographic references are preserved somewhere: > > 3B Processor: > 254-001-031 - Test Equipment List > 254-301-000 - System Documentation > 254-301-005 - General Description > 254-301-115 - TTY System > 254-301-010 - Central Control > 254-301-020 - Power Systems > 254-301-100 - Input Output Interfaces > 254-301-105 - Input Output Processor > 254-301-110 - Input Output Processor Peripheral Controller > 254-301-200 - Main Store > 254-301-210 - Moving Head Disk Drive > 254-301-215 - Disk File Controller > 254-301-220 - Magnetic Tape System > 254-301-800 - Emergency Action Procedures > 254-301-809 - Acceptance Test Plan > 254-301-811 - Task Oriented Practice-Routine > 254-301-812 - Task Oriented Practice-Trouble Clearing > 254-341-000 - DMERT > 254-341-220 - MTCE Management and Diagnostics > > API: > 234-100-020 - APS Description and Theory > 254-201-003 - API Frame Theory > 254-280-114 - APS Software Description > 254-281-030 - API Growth > > Misc: > 234-153-055 - File Store Relocation > > 3B Schematics: > SD-4C050-01 - 3B Control Frame, DMA Unit, CC Unit, MASM 0, MASM 1, Power Unit, Cooling Unit, Filter Unit > SD-4C059-02 - 3B Peripheral Control Frame > SD-4C052-01 - IOP Basic > SD-4C049-01 - IOP Growth > SD-4C051-01 - DFC Unit > SD-4C056-01 - Disk Inverter Frame > SD-4C058-01 - Tape Transport Frame > SD-4C057-01 - Inverter Unit > SD-4C070-02 - Inverter Control Unit > SD-4C054-01 - Duct & Cabling > SD-4C053-01 - Power Distributing, AC Requirements > SD-4C069-01 - Micro Level Test Set > SD-4C065-01 - Port Switch > > API Schematics: > SD-5A056-01 - API Frame > SD-5A055-01 - API Unit > > There are then references to a few "Handbooks": > > HB 309 - 3B Proc > HB 263(A) - API ITS (Mainly mentions tests and diagnostic stuff) > HB 264(A) - API Generic (Not sure what ITS and Generic distinguish here) > > Finally, a bit of fun. Towards the back of the pamphlet is a small crossword puzzle concerning the API: https://i.imgur.com/0pLcK9p.jpg > > Answers to come when I do a scan of this. If anyone has any particular information they want me to try and find in it sooner rather than later just let me know, it's 88 pages but small form. > > - Matt G. >From a similar thread a year ago, here are the top level BSP numbers for various 3B systems: 254- 3B20D 555- 3B2 585- 3B5 From arnold at skeeve.com Thu Sep 14 07:28:55 2023 From: arnold at skeeve.com (arnold at skeeve.com) Date: Wed, 13 Sep 2023 14:28:55 -0700 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? Message-ID: <202309132128.38DLSvEj026719@freefriends.org> Hello All. For whoever's interested, the csv code has been merged into the master branch of the Git repo. Have fun! Arnold > From: arnold at skeeve.com (arnold at skeeve.com) > Date: Sun, 10 Sep 2023 13:41:34 -0600 > Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? > > Hi. > > markus schnalke wrote: > > > Hoi, > > > > I just discovered that one of my favorite computer books about my > > best liked programming language (besides C) releases in a second > > edition. Does anyone know what the differences of 1st and 2nd > > edition are? > > > > As the original book is almost perfect, the only rework and > > extension direction I can think of is towards different > > implementations like gawk, mawk, portability and such things. > > > > Does anyone know more about it? Maybe some inside information? ;-) > > > > meillo > > Inside information? As it happens, yes, I do have some. :-) > (I was a reviewer.) > > [In the below, "awk" means Brian Kernighan's awk.] > > In the 36 (!) years since the first edition was published, awk > has undergone, shall we say, a large number of small changes. These > are listed in the FIXES file currently in the master branch of > https://github.com/onetrueawk/awk. > > In addition, Brian Kernighan decided to add support for UTF-8 input, > which is what awk now expects, and support for CSV input files when > invoked with the --csv option. Furthermore, there is a new \u escape > sequence which must be followed by 1-8 hexadecimal digits for specifying > Unicode code points. > > The book itself has been carefully revised. The large second chapter > which was a reference to the full language was moved to an appendix. > Many of the example programs from the first edition were retained > and updated, but there is also quite of lot of pleasing new material. > > There is mention of, and occasional comparison with, gawk, mawk and > Ben Hoyt's GoAwk, but by and large the focus is on the authors' version. > > The new code is currently in the "csv" branch of the above Github > repo. The maintainer is in the process of tidying up the repo (dealing > with issues and pull requests) and will merge the csv branch into > master sometime in the very near future. > > I'm told that the printed books with get to the publisher's warehouse > towards the end of September. The book is available now on O'Reilly's > Safari learning site (safari.oreilly.com) for anyone who has a > subscription. > > Matching code (--csv and \u) are in gawk's master branch now. I will > make a release this fall, after the new code has moved into master > in BWK's awk. > > I heartily recommend the book; it is totally up to Brian Kernighan's > usual very high standard. > > Enjoy, > > Arnold > From mah at mhorton.net Fri Sep 15 11:40:27 2023 From: mah at mhorton.net (Mary Ann Horton) Date: Thu, 14 Sep 2023 18:40:27 -0700 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <202309132128.38DLSvEj026719@freefriends.org> References: <202309132128.38DLSvEj026719@freefriends.org> Message-ID: <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> Which Git repo?  The CSV code is a game changer for awk, otherwise I need a hokey python script. Is it being merged into the Gnu awk that will wind up in Red Hat awk? Please please? Thanks, /Mary Ann Horton/ (she/her/ma'am) maryannhorton.com “This is a great book about an amazing journey of a woman who went through hell to become the person she is today.” * - Monica Helms, creator of the transgender flag* "Brave and Important - Don’t miss this wonderful book!" * - Laura L. Engel, Intl. Memoir Writers Assn.*       Available on Amazon and bn.com. Audiobook on Google Play. On 9/13/23 14:28, arnold at skeeve.com wrote: > Hello All. > > For whoever's interested, the csv code has been merged into the master > branch of the Git repo. Have fun! > > Arnold > >> From: arnold at skeeve.com (arnold at skeeve.com) >> Date: Sun, 10 Sep 2023 13:41:34 -0600 >> Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? >> >> Hi. >> >> markus schnalke wrote: >> >>> Hoi, >>> >>> I just discovered that one of my favorite computer books about my >>> best liked programming language (besides C) releases in a second >>> edition. Does anyone know what the differences of 1st and 2nd >>> edition are? >>> >>> As the original book is almost perfect, the only rework and >>> extension direction I can think of is towards different >>> implementations like gawk, mawk, portability and such things. >>> >>> Does anyone know more about it? Maybe some inside information? ;-) >>> >>> meillo >> Inside information? As it happens, yes, I do have some. :-) >> (I was a reviewer.) >> >> [In the below, "awk" means Brian Kernighan's awk.] >> >> In the 36 (!) years since the first edition was published, awk >> has undergone, shall we say, a large number of small changes. These >> are listed in the FIXES file currently in the master branch of >> https://github.com/onetrueawk/awk. >> >> In addition, Brian Kernighan decided to add support for UTF-8 input, >> which is what awk now expects, and support for CSV input files when >> invoked with the --csv option. Furthermore, there is a new \u escape >> sequence which must be followed by 1-8 hexadecimal digits for specifying >> Unicode code points. >> >> The book itself has been carefully revised. The large second chapter >> which was a reference to the full language was moved to an appendix. >> Many of the example programs from the first edition were retained >> and updated, but there is also quite of lot of pleasing new material. >> >> There is mention of, and occasional comparison with, gawk, mawk and >> Ben Hoyt's GoAwk, but by and large the focus is on the authors' version. >> >> The new code is currently in the "csv" branch of the above Github >> repo. The maintainer is in the process of tidying up the repo (dealing >> with issues and pull requests) and will merge the csv branch into >> master sometime in the very near future. >> >> I'm told that the printed books with get to the publisher's warehouse >> towards the end of September. The book is available now on O'Reilly's >> Safari learning site (safari.oreilly.com) for anyone who has a >> subscription. >> >> Matching code (--csv and \u) are in gawk's master branch now. I will >> make a release this fall, after the new code has moved into master >> in BWK's awk. >> >> I heartily recommend the book; it is totally up to Brian Kernighan's >> usual very high standard. >> >> Enjoy, >> >> Arnold >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Fri Sep 15 13:48:57 2023 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 14 Sep 2023 21:48:57 -0600 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> Message-ID: <202309150348.38F3mvxD015003@freefriends.org> Hi Mary Anne. Mary Ann Horton wrote: > Which Git repo?  The CSV code is a game changer for awk, otherwise I > need a hokey python script. https://github.com/onetrueawk/awk. This was mentioned in the note you quoted... > Is it being merged into the Gnu awk ... Also mentioned is that the code is already in gawk's master branch and that I will make a release in the fall, I hope. > ... that will wind up in Red Hat awk? > Please please? I have no control over that. Sorry. However, gawk is easy to build and install from tarball, or if you're desperate, directly from Git. Be in touch privately if you want more info. Thanks, Arnold > Thanks, > > /Mary Ann Horton/ (she/her/ma'am) > maryannhorton.com > > “This is a great book about an amazing journey of a woman > who went through hell to become the person she is today.” > * - Monica Helms, creator of the transgender flag* > > "Brave and Important - Don’t miss this wonderful book!" > * - Laura L. Engel, Intl. Memoir Writers Assn.* > >       Available on Amazon and bn.com. Audiobook on Google Play. > > > > > > On 9/13/23 14:28, arnold at skeeve.com wrote: > > Hello All. > > > > For whoever's interested, the csv code has been merged into the master > > branch of the Git repo. Have fun! > > > > Arnold > > > >> From: arnold at skeeve.com (arnold at skeeve.com) > >> Date: Sun, 10 Sep 2023 13:41:34 -0600 > >> Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? > >> > >> Hi. > >> > >> markus schnalke wrote: > >> > >>> Hoi, > >>> > >>> I just discovered that one of my favorite computer books about my > >>> best liked programming language (besides C) releases in a second > >>> edition. Does anyone know what the differences of 1st and 2nd > >>> edition are? > >>> > >>> As the original book is almost perfect, the only rework and > >>> extension direction I can think of is towards different > >>> implementations like gawk, mawk, portability and such things. > >>> > >>> Does anyone know more about it? Maybe some inside information? ;-) > >>> > >>> meillo > >> Inside information? As it happens, yes, I do have some. :-) > >> (I was a reviewer.) > >> > >> [In the below, "awk" means Brian Kernighan's awk.] > >> > >> In the 36 (!) years since the first edition was published, awk > >> has undergone, shall we say, a large number of small changes. These > >> are listed in the FIXES file currently in the master branch of > >> https://github.com/onetrueawk/awk. > >> > >> In addition, Brian Kernighan decided to add support for UTF-8 input, > >> which is what awk now expects, and support for CSV input files when > >> invoked with the --csv option. Furthermore, there is a new \u escape > >> sequence which must be followed by 1-8 hexadecimal digits for specifying > >> Unicode code points. > >> > >> The book itself has been carefully revised. The large second chapter > >> which was a reference to the full language was moved to an appendix. > >> Many of the example programs from the first edition were retained > >> and updated, but there is also quite of lot of pleasing new material. > >> > >> There is mention of, and occasional comparison with, gawk, mawk and > >> Ben Hoyt's GoAwk, but by and large the focus is on the authors' version. > >> > >> The new code is currently in the "csv" branch of the above Github > >> repo. The maintainer is in the process of tidying up the repo (dealing > >> with issues and pull requests) and will merge the csv branch into > >> master sometime in the very near future. > >> > >> I'm told that the printed books with get to the publisher's warehouse > >> towards the end of September. The book is available now on O'Reilly's > >> Safari learning site (safari.oreilly.com) for anyone who has a > >> subscription. > >> > >> Matching code (--csv and \u) are in gawk's master branch now. I will > >> make a release this fall, after the new code has moved into master > >> in BWK's awk. > >> > >> I heartily recommend the book; it is totally up to Brian Kernighan's > >> usual very high standard. > >> > >> Enjoy, > >> > >> Arnold > >> From ajayshah at mayin.org Fri Sep 15 13:49:30 2023 From: ajayshah at mayin.org (Ajay Shah) Date: Fri, 15 Sep 2023 09:19:30 +0530 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> Message-ID: On Fri, 15 Sept 2023 at 07:10, Mary Ann Horton wrote: > Which Git repo? The CSV code is a game changer for awk, otherwise I need > a hokey python script. > Me too! > Is it being merged into the Gnu awk that will wind up in Red Hat awk? > Please please? > Debian is an important distribution in this regard, which may go into many other distros like Ubuntu. I just looked in Debian stable -- $ apt-cache show original-awk Package: original-awk Version: 2022-09-12-1 Installed-Size: 186 Maintainer: Santiago Vila Architecture: amd64 Provides: awk Pre-Depends: libc6 (>= 2.34) Description-en: The original awk described in "The AWK Programming Language" This is the version of awk described in "The AWK Programming Language", by Al Aho, Brian Kernighan, and Peter Weinberger (Addison-Wesley, 1988, ISBN 0-201-07981-X). Description-md5: 7a3c565b081bc0f03d9e79a6fd87fe27 Multi-Arch: foreign Homepage: https://github.com/onetrueawk/awk Tag: devel::interpreter, implemented-in::c, interface::commandline, role::program, scope::utility, use::filtering, use::scanning, works-with::text Section: interpreters Priority: optional Filename: pool/main/o/original-awk/original-awk_2022-09-12-1_amd64.deb Size: 86548 MD5sum: f5a128040710433c276d866038703051 SHA256: f1835f1e5aca2f69e0b997b285b33b45aaa1abf4071edfd7481149076ca80a63 The key URLs are https://packages.debian.org/sid/original-awk https://github.com/onetrueawk/awk sanvila at debian.org -- Ajay Shah ajayshah at mayin.org http://www.mayin.org/ajayshah -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Fri Sep 15 13:54:25 2023 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 14 Sep 2023 21:54:25 -0600 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> Message-ID: <202309150354.38F3sPjK015761@freefriends.org> This is getting off-topic, folks... Thanks, Arnold Ajay Shah wrote: > On Fri, 15 Sept 2023 at 07:10, Mary Ann Horton wrote: > > > Which Git repo? The CSV code is a game changer for awk, otherwise I need > > a hokey python script. > > > Me too! > > > Is it being merged into the Gnu awk that will wind up in Red Hat awk? > > Please please? > > > Debian is an important distribution in this regard, which may go into many > other distros like Ubuntu. I just looked in Debian stable -- > > $ apt-cache show original-awk > Package: original-awk > Version: 2022-09-12-1 > Installed-Size: 186 > Maintainer: Santiago Vila > Architecture: amd64 > Provides: awk > Pre-Depends: libc6 (>= 2.34) > Description-en: The original awk described in "The AWK Programming Language" > This is the version of awk described in "The AWK Programming Language", > by Al Aho, Brian Kernighan, and Peter Weinberger > (Addison-Wesley, 1988, ISBN 0-201-07981-X). > Description-md5: 7a3c565b081bc0f03d9e79a6fd87fe27 > Multi-Arch: foreign > Homepage: https://github.com/onetrueawk/awk > Tag: devel::interpreter, implemented-in::c, interface::commandline, > role::program, scope::utility, use::filtering, use::scanning, > works-with::text > Section: interpreters > Priority: optional > Filename: pool/main/o/original-awk/original-awk_2022-09-12-1_amd64.deb > Size: 86548 > MD5sum: f5a128040710433c276d866038703051 > SHA256: f1835f1e5aca2f69e0b997b285b33b45aaa1abf4071edfd7481149076ca80a63 > > The key URLs are > https://packages.debian.org/sid/original-awk > https://github.com/onetrueawk/awk > sanvila at debian.org > > -- > Ajay Shah > ajayshah at mayin.org > http://www.mayin.org/ajayshah From ken.unix.guy at gmail.com Fri Sep 15 19:09:14 2023 From: ken.unix.guy at gmail.com (KenUnix) Date: Fri, 15 Sep 2023 05:09:14 -0400 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <202309150354.38F3sPjK015761@freefriends.org> References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150354.38F3sPjK015761@freefriends.org> Message-ID: Yes. I normally keep quiet but what does transgender flags have to do with AWK or Unix?? Let's at least stay with what the purpose of this group is. -Ken On Thu, Sep 14, 2023 at 11:54 PM wrote: > This is getting off-topic, folks... > > Thanks, > > Arnold > > Ajay Shah wrote: > > > On Fri, 15 Sept 2023 at 07:10, Mary Ann Horton wrote: > > > > > Which Git repo? The CSV code is a game changer for awk, otherwise I > need > > > a hokey python script. > > > > > Me too! > > > > > Is it being merged into the Gnu awk that will wind up in Red Hat awk? > > > Please please? > > > > > Debian is an important distribution in this regard, which may go into > many > > other distros like Ubuntu. I just looked in Debian stable -- > > > > $ apt-cache show original-awk > > Package: original-awk > > Version: 2022-09-12-1 > > Installed-Size: 186 > > Maintainer: Santiago Vila > > Architecture: amd64 > > Provides: awk > > Pre-Depends: libc6 (>= 2.34) > > Description-en: The original awk described in "The AWK Programming > Language" > > This is the version of awk described in "The AWK Programming Language", > > by Al Aho, Brian Kernighan, and Peter Weinberger > > (Addison-Wesley, 1988, ISBN 0-201-07981-X). > > Description-md5: 7a3c565b081bc0f03d9e79a6fd87fe27 > > Multi-Arch: foreign > > Homepage: https://github.com/onetrueawk/awk > > Tag: devel::interpreter, implemented-in::c, interface::commandline, > > role::program, scope::utility, use::filtering, use::scanning, > > works-with::text > > Section: interpreters > > Priority: optional > > Filename: pool/main/o/original-awk/original-awk_2022-09-12-1_amd64.deb > > Size: 86548 > > MD5sum: f5a128040710433c276d866038703051 > > SHA256: f1835f1e5aca2f69e0b997b285b33b45aaa1abf4071edfd7481149076ca80a63 > > > > The key URLs are > > https://packages.debian.org/sid/original-awk > > https://github.com/onetrueawk/awk > > sanvila at debian.org > > > > -- > > Ajay Shah > > ajayshah at mayin.org > > http://www.mayin.org/ajayshah > -- End of line JOB TERMINATED -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Fri Sep 15 21:24:56 2023 From: crossd at gmail.com (Dan Cross) Date: Fri, 15 Sep 2023 07:24:56 -0400 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150354.38F3sPjK015761@freefriends.org> Message-ID: On Fri, Sep 15, 2023 at 5:09 AM KenUnix wrote: > Yes. I normally keep quiet but what does transgender flags have to do with AWK or Unix?? Several years ago, on this list, someone wrote a lengthy post about women in software. It was not complimentary; the person was some kind of "Men's Rights" activist. I stayed quiet at the time, and I regret that; I'm not going to stay quiet now. To directly answer your question: Mary Ann is an important figure in the history of Unix. Mary Ann is transgender. Mary Ann has written a book about her pioneering experience as a transgender woman in corporate America at a time when that was rare (or, rather, not openly discussed). Mary Ann's .sig contains information about her book. That appears to be what you are responding to: her .sig; not the content of her message, just the signature. > Let's at least stay with what the purpose of this group is. Indeed. Let's. It's not spoon-fooding tech support to people who can't be bothered to figure it out for themselves, or do even a modicum of work researching things on their own. Nor is it writing transphobic non-sequiturs in response to .sigs. You didn't need to respond, at all: you could have just moved on. But you chose not to. Perhaps you should go elsewhere. - Dan C. > On Thu, Sep 14, 2023 at 11:54 PM wrote: >> >> This is getting off-topic, folks... >> >> Thanks, >> >> Arnold >> >> Ajay Shah wrote: >> >> > On Fri, 15 Sept 2023 at 07:10, Mary Ann Horton wrote: >> > >> > > Which Git repo? The CSV code is a game changer for awk, otherwise I need >> > > a hokey python script. >> > > >> > Me too! >> > >> > > Is it being merged into the Gnu awk that will wind up in Red Hat awk? >> > > Please please? >> > > >> > Debian is an important distribution in this regard, which may go into many >> > other distros like Ubuntu. I just looked in Debian stable -- >> > >> > $ apt-cache show original-awk >> > Package: original-awk >> > Version: 2022-09-12-1 >> > Installed-Size: 186 >> > Maintainer: Santiago Vila >> > Architecture: amd64 >> > Provides: awk >> > Pre-Depends: libc6 (>= 2.34) >> > Description-en: The original awk described in "The AWK Programming Language" >> > This is the version of awk described in "The AWK Programming Language", >> > by Al Aho, Brian Kernighan, and Peter Weinberger >> > (Addison-Wesley, 1988, ISBN 0-201-07981-X). >> > Description-md5: 7a3c565b081bc0f03d9e79a6fd87fe27 >> > Multi-Arch: foreign >> > Homepage: https://github.com/onetrueawk/awk >> > Tag: devel::interpreter, implemented-in::c, interface::commandline, >> > role::program, scope::utility, use::filtering, use::scanning, >> > works-with::text >> > Section: interpreters >> > Priority: optional >> > Filename: pool/main/o/original-awk/original-awk_2022-09-12-1_amd64.deb >> > Size: 86548 >> > MD5sum: f5a128040710433c276d866038703051 >> > SHA256: f1835f1e5aca2f69e0b997b285b33b45aaa1abf4071edfd7481149076ca80a63 >> > >> > The key URLs are >> > https://packages.debian.org/sid/original-awk >> > https://github.com/onetrueawk/awk >> > sanvila at debian.org >> > >> > -- >> > Ajay Shah >> > ajayshah at mayin.org >> > http://www.mayin.org/ajayshah > > > > -- > End of line > JOB TERMINATED > > From dave at horsfall.org Sat Sep 16 06:04:40 2023 From: dave at horsfall.org (Dave Horsfall) Date: Sat, 16 Sep 2023 06:04:40 +1000 (EST) Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150354.38F3sPjK015761@freefriends.org> Message-ID: On Fri, 15 Sep 2023, KenUnix wrote: > Yes. I normally keep quiet but what does transgender flags have to do > with AWK or Unix?? Are you really commenting upon the contents of someone's signature block? > Let's at least stay with what the purpose of this group is. Perhaps you could try following your own advice... -- Dave From cowan at ccil.org Sat Sep 16 06:09:20 2023 From: cowan at ccil.org (John Cowan) Date: Fri, 15 Sep 2023 16:09:20 -0400 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150354.38F3sPjK015761@freefriends.org> Message-ID: On Fri, Sep 15, 2023 at 4:04 PM Dave Horsfall wrote: > Are you really commenting upon the contents of someone's signature block? > My view has always been that it's okay to comment on them (within reason) but not okay to complain about them. In a .sig block, people can talk about whatever they want, as long as the block itself is not BUAFed or otherwise warlord-worthy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Sat Sep 16 06:17:16 2023 From: tuhs at tuhs.org (Warren Toomey via TUHS) Date: Sat, 16 Sep 2023 06:17:16 +1000 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150354.38F3sPjK015761@freefriends.org> Message-ID: On Fri, Sep 15, 2023 at 05:09:14AM -0400, KenUnix wrote: > Yes. I normally keep quiet but what does transgender flags have to do > with AWK or Unix?? > Let's at least stay with what the purpose of this group is. > -Ken I rarely weigh in on things, but I think this one is important. The history of Unix is not just of the technology, but also of the people involved, their successes and travails, and the communities that they built. Mary Ann referenced a book about her life and journey in her e-mail's .sig. She is a very important figure in the history of Unix and I think her .sig is entirely relevant to TUHS. Cheers, Warren From will.senn at gmail.com Sat Sep 16 06:59:36 2023 From: will.senn at gmail.com (Will Senn) Date: Fri, 15 Sep 2023 15:59:36 -0500 Subject: [TUHS] TUHS Message-ID: Hi Warren, I don't care to weigh in on the transgender flag issue, but I also don't want to hear about it. Please remove me from the list. All due respect to you and Mary Ann. Monica Helms is not a Unix hero. Thanks, Will -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Sat Sep 16 07:08:16 2023 From: imp at bsdimp.com (Warner Losh) Date: Fri, 15 Sep 2023 22:08:16 +0100 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150354.38F3sPjK015761@freefriends.org> Message-ID: On Fri, Sep 15, 2023, 10:09 AM KenUnix wrote: > > Yes. I normally keep quiet but what does transgender flags have to do with > AWK or Unix?? > Why do you care? It's none of your business. We have all kinds of other signs. Why complain about someone writing a book about her life. She exists. She has an interesting story. Don't like it, leave rather than posting homophobic slurs. Warner Warner > Let's at least stay with what the purpose of this group is. > > -Ken > > > On Thu, Sep 14, 2023 at 11:54 PM wrote: > >> This is getting off-topic, folks... >> >> Thanks, >> >> Arnold >> >> Ajay Shah wrote: >> >> > On Fri, 15 Sept 2023 at 07:10, Mary Ann Horton wrote: >> > >> > > Which Git repo? The CSV code is a game changer for awk, otherwise I >> need >> > > a hokey python script. >> > > >> > Me too! >> > >> > > Is it being merged into the Gnu awk that will wind up in Red Hat awk? >> > > Please please? >> > > >> > Debian is an important distribution in this regard, which may go into >> many >> > other distros like Ubuntu. I just looked in Debian stable -- >> > >> > $ apt-cache show original-awk >> > Package: original-awk >> > Version: 2022-09-12-1 >> > Installed-Size: 186 >> > Maintainer: Santiago Vila >> > Architecture: amd64 >> > Provides: awk >> > Pre-Depends: libc6 (>= 2.34) >> > Description-en: The original awk described in "The AWK Programming >> Language" >> > This is the version of awk described in "The AWK Programming Language", >> > by Al Aho, Brian Kernighan, and Peter Weinberger >> > (Addison-Wesley, 1988, ISBN 0-201-07981-X). >> > Description-md5: 7a3c565b081bc0f03d9e79a6fd87fe27 >> > Multi-Arch: foreign >> > Homepage: https://github.com/onetrueawk/awk >> > Tag: devel::interpreter, implemented-in::c, interface::commandline, >> > role::program, scope::utility, use::filtering, use::scanning, >> > works-with::text >> > Section: interpreters >> > Priority: optional >> > Filename: pool/main/o/original-awk/original-awk_2022-09-12-1_amd64.deb >> > Size: 86548 >> > MD5sum: f5a128040710433c276d866038703051 >> > SHA256: f1835f1e5aca2f69e0b997b285b33b45aaa1abf4071edfd7481149076ca80a63 >> > >> > The key URLs are >> > https://packages.debian.org/sid/original-awk >> > https://github.com/onetrueawk/awk >> > sanvila at debian.org >> > >> > -- >> > Ajay Shah >> > ajayshah at mayin.org >> > http://www.mayin.org/ajayshah >> > > > -- > End of line > JOB TERMINATED > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norman at oclsc.org Sat Sep 16 11:20:52 2023 From: norman at oclsc.org (Norman Wilson) Date: Fri, 15 Sep 2023 21:20:52 -0400 (EDT) Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? Message-ID: To the Attention of Warren Toomey (and all who stay in his hotel): I don't care enough to weigh in on any issues that don't interest me, but I am the most important person in the world, and whatever I say goes, so you better listen or you'll be sorry. It is my very important opinion that only things I want to hear about should be discussed on this mailing list. I want to hear about Unix and awk, but not about perl. No one must talk about any shell except Ken's original. The sun scares me, forcing me to hack all night and sleep all day (never mind the malicious stories that I press wild flowers); therefore there must never be any mention of Sun Microsystems or Solaris. I am also worried that a comet will fall on my house and damage my Twinkie stockpile, so no discussion of the VAX-11/750 is allowed, nor of work done by the Bell Labs Computing Science Research Center during the 1980s when much of their work was done on systems of that model, which were even named (ewwwww!!!) after comets. Any mention of non-nerd-approved(TM) subjects is also forbidden, including Agricola, ferrets, mimes (which are even scarier than comets!), Lions (and Tigers and Bears), lurgi, csv files, and gannets (they wet their nests). Not to mention Bazonka. I hereby direct the moderators of this list, who must obey my every command, to terminate with extreme prejudice anyone who dares even to think of violating these rules. Viva la revolution! Yeliz Gardinovich Bimmler (Mrs) Port Morton, Alabama, CUS From mah at mhorton.net Sun Sep 17 03:32:24 2023 From: mah at mhorton.net (Mary Ann Horton) Date: Sat, 16 Sep 2023 10:32:24 -0700 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150354.38F3sPjK015761@freefriends.org> Message-ID: <28cabcca-ee11-bfde-f333-6eca05a1a91f@mhorton.net> Thank you all for the kind words about me and my sig. I'll add there are parts of this book that focus on UNIX history, although in my next memoir (not started yet) the tech will be the primary focus. Thanks, /Mary Ann Horton/ (she/her/ma'am) maryannhorton.com “This is a great book about an amazing journey of a woman who went through hell to become the person she is today.” * - Monica Helms, creator of the transgender flag* "Brave and Important - Don’t miss this wonderful book!" * - Laura L. Engel, Intl. Memoir Writers Assn.*       Available on Amazon and bn.com. Audiobook on Google Play. On 9/15/23 13:17, Warren Toomey via TUHS wrote: > On Fri, Sep 15, 2023 at 05:09:14AM -0400, KenUnix wrote: >> Yes. I normally keep quiet but what does transgender flags have to do >> with AWK or Unix?? >> Let's at least stay with what the purpose of this group is. >> -Ken > I rarely weigh in on things, but I think this one is important. > > The history of Unix is not just of the technology, but also of the > people involved, their successes and travails, and the communities > that they built. Mary Ann referenced a book about her life and > journey in her e-mail's .sig. She is a very important figure in the > history of Unix and I think her .sig is entirely relevant to TUHS. > > Cheers, Warren -------------- next part -------------- An HTML attachment was scrubbed... URL: From mah at mhorton.net Sun Sep 17 03:52:38 2023 From: mah at mhorton.net (Mary Ann Horton) Date: Sat, 16 Sep 2023 10:52:38 -0700 Subject: [TUHS] The AWK Programming Language, 2nd Ed.: What's new? In-Reply-To: <202309150348.38F3mvxD015003@freefriends.org> References: <202309132128.38DLSvEj026719@freefriends.org> <3963d4b9-a866-2e2f-d289-c4f261e1ca7e@mhorton.net> <202309150348.38F3mvxD015003@freefriends.org> Message-ID: <0e243243-2437-7f25-76f3-676ef83913b1@mhorton.net> On 9/14/23 20:48, arnold at skeeve.com wrote: > > Mary Ann Horton wrote: > > Is it being merged into the Gnu awk ... > Also mentioned is that the code is already in gawk's master > branch and that I will make a release in the fall, I hope. > >> ... that will wind up in Red Hat awk? >> Please please? > I have no control over that. Sorry. However, gawk is easy to > build and install from tarball, or if you're desperate, directly > from Git. Be in touch privately if you want more info. This is wonderful news. Red Hat uses GNU awk, so it seems likely it will (eventually) appear there. I'm aware I could compile it myself, but this is for a professional use on the systems running a power grid, and even if I were to take the leap to install an open source version instead of the official RHEL awk, there are government regulations making that a bad idea. Thanks,     Mary Ann From will.senn at gmail.com Mon Sep 18 10:43:41 2023 From: will.senn at gmail.com (Will Senn) Date: Sun, 17 Sep 2023 19:43:41 -0500 Subject: [TUHS] TUHS Message-ID: All, My apologies to you for sending a note to the entire list that was meant to be sent only to Warren. I definitely had not intended to send it to the entire list (reply list... ugh.). If I were to get a mulligan, I would make it clear that I do not take issue with anyone's signature, including Mary Ann's. Signatures, in my view, are a matter of free speech. I would simply state my preference that we keep the discussion to a more technical level and consider moving cultural topics to another list altogether. Thanks, Will -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Tue Sep 19 04:07:08 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Mon, 18 Sep 2023 18:07:08 +0000 Subject: [TUHS] UNIX Influence on Teletype and Vice Versa Message-ID: Good morning folks, something I've been pondering on this morning is the potential close connection between UNIX development and Teletype interfacing designs as the 70s and 80s marched on. Seeing as Teletype was a part of the Bell System (albeit a little less obviously in marketing than it's kin), was there any sort of official rapport between folks working on UNIX and those designing subsequent Teletypes, Dataspeed terminals, etc? For instance, would there have been any influence an up-and-coming Teletype design would've had on developments in the UNIX tty drivers, or would particulars of UNIX tty drivers "rub off" on specifics of terminals being developed? Or were those units so distant from one another that there would've been little cross-talk between teams? Granted, an argument could be made for specifically avoiding any significant knowledge of internals, that way UNIX tty driver folks don't potentially paint into a Teletype corner and vice versa. Still, with the tightly integrated nature of Bell System R&D, manufacturing, supply, etc. it would be very "Bell System" for there to be some sort of interplay between Teletype and UNIX. Anyone got the scoop on whether Teletype hardware enjoyed a special place in UNIX tty-interfacing considerations or vice versa, or if they were just two relatively insular developments from one another in the same general field from the same big umbrella organization? - Matt G. From ron at ronnatalie.com Tue Sep 19 04:21:13 2023 From: ron at ronnatalie.com (Ronald Natalie) Date: Mon, 18 Sep 2023 18:21:13 +0000 Subject: [TUHS] UNIX Influence on Teletype and Vice Versa In-Reply-To: References: Message-ID: The thing most people know in the (computer) world as a teletype is a model 33 which really UNIX was unaffected by (other than dealing with the limited character set) and UNIX didn’t influence. For those with a model 37 (I had one in my house for a while picked up surplus from Rocky Flats), you knew of its influence on UNIX. This was a unit that the line terminator was indeed NEWLINE (^J) rather than carriage return. It dealt with all the ESC 8 and 9 and the like that nroff put out by default and th SI/SO to shift from the regular to the greek character box. The mechanical teletypes largely predated UNIX having the ability to change anything. That doesn’t mean that UNIX didn’t affect the TELETYPE brand. The 5620 DMD bore the Teletype brand but came straight from BellLabs Blit (aka Jerry) work. From paul.winalski at gmail.com Tue Sep 19 04:27:01 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Mon, 18 Sep 2023 14:27:01 -0400 Subject: [TUHS] UNIX Influence on Teletype and Vice Versa In-Reply-To: References: Message-ID: On 9/18/23, Ronald Natalie wrote: > The thing most people know in the (computer) world as a teletype is a > model 33 which really UNIX was unaffected by (other than dealing with > the limited character set) and UNIX didn’t influence. For those with a > model 37 (I had one in my house for a while picked up surplus from Rocky > Flats), you knew of its influence on UNIX. This was a unit that the > line terminator was indeed NEWLINE (^J) rather than carriage return. It > dealt with all the ESC 8 and 9 and the like that nroff put out by > default and th SI/SO to shift from the regular to the greek character > box. Most importantly for Unix, I would think, the Teletype model 37 had both uppercase and lowercase capability. Essential for a case-sensitive environment such as Unix. The model 33 was uppercase-only. -Paul W. From douglas.mcilroy at dartmouth.edu Tue Sep 19 06:02:20 2023 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Mon, 18 Sep 2023 16:02:20 -0400 Subject: [TUHS] UNIX Influence on Teletype and Vice Versa Message-ID: Joe Ossanna worked diligently to see that WECo's ASCII teletype really would come to market and would meet the needs of Unix. He famously estimated that Bell Labs alone was a sizable market: 777 machines. He also leaned on WECo to make the return key issue a single newline. And he specified what non-ascii characters would fill out the 128 positions on the Bell Labs type boxes. This requires encoding 7-bit ASCII in 8 bits. I don't know whether WECo had been planning to do so anyway. Doug From pnr at planet.nl Tue Sep 19 08:31:30 2023 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 19 Sep 2023 00:31:30 +0200 Subject: [TUHS] x/y/zmodem on Unix Message-ID: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Last december Matt brought up xmodem and recently I needed it for an almost identical use case. Studied it a bit and in the context of Unix, it is an interesting piece of software history. Although xmodem originated on CP/M in 1977, it seems to have arrived on Unix soon thereafter. The first Unix implementation seems to have gone by the name of “rbsb” and must have originated when V7 was prevalent: it uses alarm() to simulate non-blocking I/O. Over the course of the 80’s ymodem and zmodem were added and the package became lrzsz; the source continued to have a very V7-ish feel to it at least to the early 90’s. In the mid-90’s it was converted to ansi-C and had some other modernization, but it looks like it would still have run on SysIII (apart from the ansi).The last update to the upstream source package appears to be from 1998, i.e. 25 years ago. It is still packaged today for FreeBSD and major Linux distros. On Unix it must be one the oldest code bases still in regular use, with the 1980 source still recognizable in its current incarnation. Any other contenders come to mind? I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular for file exchange in the Unix scene as well, along with several other similar tools. Also, the ymodem approach to file metadata is very unix oriented, suggesting it originated on Unix or at least that Unix users were an important user demographic. Yet, I could find little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top. Maybe these programs were commonly pulled from a bulletin board and hence not on distribution media. Not sure how that would be bootstrapped, although the 2.11BSD files for x/y/zmodem include “minirb.c”, a 175 line implementation to receive files using the ymodem protocol. It is possible that this was transferred as plain text as a first step, or even retyped. Any recollections? From pnr at planet.nl Tue Sep 19 09:02:21 2023 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 19 Sep 2023 01:02:21 +0200 Subject: [TUHS] CRC calculation in the 1980s Message-ID: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> When looking at old xmodem code I noticed that it calculated its CRC bit-by-bit, switching to byte-wise using a table in the late 80’s. It never seems to have used the byte-wise, “on-the-fly” algorithm. This seems to match a pattern: I often come across bit-wise and table implementations, but rarely on-the-fly implementations if any. The on-the-fly algorithm was known at least since 1983, following a paper by Perez: http://www.bitsavers.org/components/fairchild/_appNotes/Byte-wise_CRC_Jun83.pdf The paper was noted, for example it is on the citation list of RFC1134, describing the PPP protocol. Today, a wikipedia page gives implementations for various polynomials: https://en.wikipedia.org/wiki/Computation_of_cyclic_redundancy_checks#Parallel_computation_without_table Now, it would seem to me that on memory constrained personal computers and PDP-11’s the “on-the-fly” algorithm would have been a good choice, being just a few lines of code and maybe 30-50% slower than table lookup. The tables aren’t big, but a kilobyte is a lot when you only have 64. Any suggestions as to why the on-the-fly algorithm did not catch on more in the 1980’s? Maybe it was simply less well known than I think? From crossd at gmail.com Tue Sep 19 09:07:25 2023 From: crossd at gmail.com (Dan Cross) Date: Mon, 18 Sep 2023 19:07:25 -0400 Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: On Mon, Sep 18, 2023 at 6:31 PM Paul Ruizendaal wrote: > Last december Matt brought up xmodem and recently I needed it for an almost identical use case. Studied it a bit and in the context of Unix, it is an interesting piece of software history. > > Although xmodem originated on CP/M in 1977, it seems to have arrived on Unix soon thereafter. The first Unix implementation seems to have gone by the name of “rbsb” and must have originated when V7 was prevalent: it uses alarm() to simulate non-blocking I/O. Over the course of the 80’s ymodem and zmodem were added and the package became lrzsz; the source continued to have a very V7-ish feel to it at least to the early 90’s. In the mid-90’s it was converted to ansi-C and had some other modernization, but it looks like it would still have run on SysIII (apart from the ansi).The last update to the upstream source package appears to be from 1998, i.e. 25 years ago. > > It is still packaged today for FreeBSD and major Linux distros. On Unix it must be one the oldest code bases still in regular use, with the 1980 source still recognizable in its current incarnation. Any other contenders come to mind? > > I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular for file exchange in the Unix scene as well, along with several other similar tools. Also, the ymodem approach to file metadata is very unix oriented, suggesting it originated on Unix or at least that Unix users were an important user demographic. Yet, I could find little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top. > > Maybe these programs were commonly pulled from a bulletin board and hence not on distribution media. Not sure how that would be bootstrapped, although the 2.11BSD files for x/y/zmodem include “minirb.c”, a 175 line implementation to receive files using the ymodem protocol. It is possible that this was transferred as plain text as a first step, or even retyped. > > Any recollections? xmodem was an outgrowth of Christensen's MODEM.ASM for CP/M, and is almost criminally simple: a small block of data coupled with a little bit of header information, wait for an acknowledgement, and repeat. Metadata was small (a start byte, a block number, inverse block number, and a single checksum byte), but there was no total byte count, so it was assumed that the final block would be padded with a throwaway character. Chuck Forsberg did YMODEM and ZMODEM; YMODEM is sort of a super-XMODEM: it adds a 16-bit CRC, increases the block size, and adds a "0 block" with some metadata (total file size and file name); thus, it no longer needed a padding byte removed from the final packet. Otherwise, it retains most of the overall structure of XMODEM. ZMODEM was, as I understood it, designed for transfers across telenet, which was pretty reliable; instead of the highly synchronous send/wait-for-ack cycle of xmodem and ymodem, zmodem relies on error detection and correction and is basically a streaming protocol: a packet in a sliding window could be NAK'ed, thus rewinding the transfer, but otherwise it basically just sends data until done. lrzsz came kind of later; I remember Forsberg's rzsz package on Unix back in the day, and then there was a GNU reimplementation. For a while, the Omen Technologies BBS was listed in the BSD phone numbers files. xmodem lives on in a lot of embedded applications because of its overall simplicity. We used it to bootstrap kernels onto Oxide computers, for example, while we were doing active development. - Dan C. From robert at timetraveller.org Tue Sep 19 09:34:14 2023 From: robert at timetraveller.org (Robert Brockway) Date: Tue, 19 Sep 2023 09:34:14 +1000 (AEST) Subject: [TUHS] CRC calculation in the 1980s In-Reply-To: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> References: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> Message-ID: On Tue, 19 Sep 2023, Paul Ruizendaal wrote: > Any suggestions as to why the on-the-fly algorithm did not catch on more > in the 1980’s? Maybe it was simply less well known than I think? Could it have been the per-cpu-second billing that was (fairly?) common at the time. I was only getting in to Unix in the early 90s but saw the tail end of that. Cheers, Rob From tuhs at tuhs.org Tue Sep 19 10:02:50 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Tue, 19 Sep 2023 00:02:50 +0000 Subject: [TUHS] CRC calculation in the 1980s In-Reply-To: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> References: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> Message-ID: > Any suggestions as to why the on-the-fly algorithm did not catch on more in the 1980’s? Maybe it was simply less well known than I think? The CRC algorithm I'm familiar with shows up in Dragon Quest for the Famicom in 1986[1], written in 6502 assembly. Admittedly though I only recognized it due to the EOR with 0x1021 on lines 318-323. That I then only know from a quick and dirty CRC I threw in an XMODEM-CRC client [2] I did to accommodate for a bug in the JH7100 RISC-V board's recovery ROM implementation. Not sure if this is along the lines of the approach you're talking about, admittedly these two instances are literally all I know about CRC, just enough to be dangerous :) Still, a concrete example of ChunSoft's 6502 CRC calculation in the mid 80s, if that helps with the assessment of the time period/flow of ideas in the industry. - Matt G. [1] - https://gitlab.com/segaloco/dq1disasm/-/blob/master/src/chr3/start_pw.s [2] - https://gitlab.com/segaloco/riscv-bits/-/blob/master/util/sxj.c From tytso at mit.edu Tue Sep 19 12:04:12 2023 From: tytso at mit.edu (Theodore Ts'o) Date: Mon, 18 Sep 2023 22:04:12 -0400 Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: On Mon, Sep 18, 2023 at 07:07:25PM -0400, Dan Cross wrote: > > xmodem lives on in a lot of embedded applications because of its > overall simplicity. We used it to bootstrap kernels onto Oxide > computers, for example, while we were doing active development. Today, KDE's Konsole terminal window has a "ZModem upload" function, which I've used for sending a file up to some system where the administrator on the remote machine has disabled scp and sftp for random security $REASONS. I could compress the file and use uuencode/uudecode, but for larger files where cut and paste isn't terribly convenient, using zmodem can be one of the simpler and more effective. - Ted From jsg at jsg.id.au Tue Sep 19 12:46:44 2023 From: jsg at jsg.id.au (Jonathan Gray) Date: Tue, 19 Sep 2023 12:46:44 +1000 Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: On Tue, Sep 19, 2023 at 12:31:30AM +0200, Paul Ruizendaal wrote: > I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular for file exchange in the Unix scene as well, along with several other similar tools. Also, the ymodem approach to file metadata is very unix oriented, suggesting it originated on Unix or at least that Unix users were an important user demographic. Yet, I could find little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top. have a look at tuhs/Applications/Shoppa_Tapes/usenix878889.tar.gz usenix87/Comm/ usenix89/Comm/ also appears in 386bsd/othersrc/public/zmodem-3.03/ where the license was changed to prohibit commercial use after the RLE changes in April 1989 https://www.ohse.de/uwe/software/lrzsz.html is derived from an earlier version with the license changed to GPLv2 From sburjak at systech.com.au Tue Sep 19 13:42:39 2023 From: sburjak at systech.com.au (Serge Burjak) Date: Tue, 19 Sep 2023 13:42:39 +1000 Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: Chuck Forsberg wrote Zmodem as a sliding Window transfer protocol alternative to Kermit and superkermit. It was very high performance over slow links. He provided source code support for the protocol driver on the server and client end, in various versions. His client programs were Yam (yet another modem) and ProYam. You could shell out of the client, run other programs. Other client platforms supported parts of the protocol. Available as binaries and source when licenced. It had a scripting language and out of the box had the zmodem protocol which has the ability to send commands, resume transfers, traverse non 8 bit or unreliable links, remove or rename source file, deal with conflicting file names, synchronise folders and lots of other things. I have written many scripts to help dial up users into a unix box and download financial research data. Scripts still work across the internet. He made the code very portable, clean and I believe it is open source now. Chuck is no longer with us. SecureCRT ssh client from VanDyke.com still has native support. To send a file from the unix host, typically sz filename and it just arrives. Sz- u and it will remove the source file after successful transfer, plus lots of other options. The binary was typically linked so if you type sx it would try to do a transfer with xmodem, even though it's the same binary. Hope this helps. Serge On Tue, 19 Sept 2023 at 12:47, Jonathan Gray wrote: > > On Tue, Sep 19, 2023 at 12:31:30AM +0200, Paul Ruizendaal wrote: > > I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular for file exchange in the Unix scene as well, along with several other similar tools. Also, the ymodem approach to file metadata is very unix oriented, suggesting it originated on Unix or at least that Unix users were an important user demographic. Yet, I could find little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top. > > have a look at > tuhs/Applications/Shoppa_Tapes/usenix878889.tar.gz > usenix87/Comm/ > usenix89/Comm/ > > also appears in > 386bsd/othersrc/public/zmodem-3.03/ > where the license was changed to prohibit commercial use after > the RLE changes in April 1989 > > https://www.ohse.de/uwe/software/lrzsz.html > is derived from an earlier version with the license changed to GPLv2 From cowan at ccil.org Tue Sep 19 13:58:42 2023 From: cowan at ccil.org (John Cowan) Date: Mon, 18 Sep 2023 23:58:42 -0400 Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: On Mon, Sep 18, 2023 at 7:08 PM Dan Cross wrote: but there was no total byte > count, so it was assumed that the final block would be padded with a > throwaway character. > CP/M, like RT-11 and OS/8 before it, did not track the sizes of files in bytes or words, only in blocks. Text files ended in ^Z and were then padded with either NULs or more ^Zs if necessary; binary files were usually padded with NULs. ZMODEM was, as I understood it, designed for transfers across telenet, > which was pretty reliable; instead of the highly synchronous > send/wait-for-ack cycle of xmodem and ymodem, zmodem relies on error > detection and correction and is basically a streaming protocol: a > packet in a sliding window could be NAK'ed, thus rewinding the > transfer, but otherwise it basically just sends data until done. > It's an analogue of TCP/IP, in fact. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sburjak at systech.com.au Tue Sep 19 16:10:38 2023 From: sburjak at systech.com.au (Serge Burjak) Date: Tue, 19 Sep 2023 16:10:38 +1000 Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: If anyone wants more zmodem/Chuck Forsberg history, a prolific usenet poster, here are news archives going back to 1994. It gives a good indication of his personality. I had been dealing with him for about 6 years by then... Still have the licence keys. https://groups.google.com/g/comp.protocols.misc/c/kjQ3HWqR2Ck/m/QVrdXdbsz1EJ Kermit discussion and comparisons, https://groups.google.com/g/comp.dcom.modems/c/GuEzWARpVQY/m/nZX5QdmY7xwJ On Tue, 19 Sept 2023 at 13:42, Serge Burjak wrote: > > Chuck Forsberg wrote Zmodem as a sliding Window transfer protocol > alternative to Kermit and superkermit. It was very high performance > over slow links. He provided source code support for the protocol > driver on the server and client end, in various versions. His client > programs were Yam (yet another modem) and ProYam. You could shell out > of the client, run other programs. Other client platforms supported > parts of the protocol. Available as binaries and source when licenced. > It had a scripting language and out of the box had the zmodem protocol > which has the ability to send commands, resume transfers, traverse non > 8 bit or unreliable links, remove or rename source file, deal with > conflicting file names, synchronise folders and lots of other things. > I have written many scripts to help dial up users into a unix box and > download financial research data. Scripts still work across the > internet. He made the code very portable, clean and I believe it is > open source now. Chuck is no longer with us. > > SecureCRT ssh client from VanDyke.com still has native support. To > send a file from the unix host, typically sz filename and it just > arrives. Sz- u and it will remove the source file after successful > transfer, plus lots of other options. The binary was typically linked > so if you type sx it would try to do a transfer with xmodem, even > though it's the same binary. > > Hope this helps. > > Serge > > > On Tue, 19 Sept 2023 at 12:47, Jonathan Gray wrote: > > > > On Tue, Sep 19, 2023 at 12:31:30AM +0200, Paul Ruizendaal wrote: > > > I had always associated x/y/zmodem with CP/M and MSDOS, not so much with Unix. Last December Clem already pointed out that it was popular for file exchange in the Unix scene as well, along with several other similar tools. Also, the ymodem approach to file metadata is very unix oriented, suggesting it originated on Unix or at least that Unix users were an important user demographic. Yet, I could find little trace of x/y/zmodem in the TUHS Unix Tree. The search tool finds it in 2.11BSD, in Minix 1.5 and 2.0 and in V10. Kermit is in those as well, and in 4.3BSD and 4.4BSD on top. > > > > have a look at > > tuhs/Applications/Shoppa_Tapes/usenix878889.tar.gz > > usenix87/Comm/ > > usenix89/Comm/ > > > > also appears in > > 386bsd/othersrc/public/zmodem-3.03/ > > where the license was changed to prohibit commercial use after > > the RLE changes in April 1989 > > > > https://www.ohse.de/uwe/software/lrzsz.html > > is derived from an earlier version with the license changed to GPLv2 From dave at horsfall.org Tue Sep 19 16:37:53 2023 From: dave at horsfall.org (Dave Horsfall) Date: Tue, 19 Sep 2023 16:37:53 +1000 (EST) Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: On Tue, 19 Sep 2023, Serge Burjak wrote: > If anyone wants more zmodem/Chuck Forsberg history, a prolific usenet > poster, here are news archives going back to 1994. It gives a good > indication of his personality. I had been dealing with him for about 6 > years by then... Still have the licence keys. > > https://groups.google.com/g/comp.protocols.misc/c/kjQ3HWqR2Ck/m/QVrdXdbsz1EJ I remember him well; thanks! > Kermit discussion and comparisons, > https://groups.google.com/g/comp.dcom.modems/c/GuEzWARpVQY/m/nZX5QdmY7xwJ I was wondering when someone was going to mention Kermit... -- Dave, with a 1200/75 software modem at the time From tuhs at tuhs.org Tue Sep 19 19:47:42 2023 From: tuhs at tuhs.org (Michael Stiller via TUHS) Date: Tue, 19 Sep 2023 11:47:42 +0200 Subject: [TUHS] x/y/zmodem on Unix In-Reply-To: References: <81B4F221-C3EF-4543-84CD-78E52ABF0E67@planet.nl> Message-ID: I just wanted to mention, that you can also get zmodem support for iTerm2 if you are on mac. In fact i use it a lot. See here: https://github.com/robberphex/iTerm2-zmodem Best regards, Michael > On 19. Sep 2023, at 04:04, Theodore Ts'o wrote: > > On Mon, Sep 18, 2023 at 07:07:25PM -0400, Dan Cross wrote: >> >> xmodem lives on in a lot of embedded applications because of its >> overall simplicity. We used it to bootstrap kernels onto Oxide >> computers, for example, while we were doing active development. > > Today, KDE's Konsole terminal window has a "ZModem upload" function, > which I've used for sending a file up to some system where the > administrator on the remote machine has disabled scp and sftp for > random security $REASONS. > > I could compress the file and use uuencode/uudecode, but for larger > files where cut and paste isn't terribly convenient, using zmodem can > be one of the simpler and more effective. > > - Ted From jnc at mercury.lcs.mit.edu Tue Sep 19 21:42:29 2023 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Tue, 19 Sep 2023 07:42:29 -0400 (EDT) Subject: [TUHS] CRC calculation in the 1980s Message-ID: <20230919114229.55D3418C084@mercury.lcs.mit.edu> > From: Paul Ruizendaal > Any suggestions as to why the on-the-fly algorithm did not catch on > more in the 1980's? Maybe it was simply less well known than I think? I can't answer that directly, but I will point you at IEN-56, "CRC Checksum Calculation", by Dave Reed (11-Sep-78): https://www.rfc-editor.org/ien/ien56.pdf Dave wanted the INWG to use a more powerful (in terms of detecting errors) CRC, instead of the simple summation eventually adopted, in TCP and IP. So he produced code to implement a particular CRC, to show people that it would not be particularly expensive (whether in time, or space, I don't alas recall definitively; speed would have been an important consideration, when competing with the summation, though). This would have been close to the leading edge of our knowledge at the time; Dave liked playing around with math, and at about that time he did a very fast DES implementation. Noel From lars at nocrew.org Tue Sep 19 22:54:58 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Tue, 19 Sep 2023 12:54:58 +0000 Subject: [TUHS] CRC calculation in the 1980s In-Reply-To: <20230919114229.55D3418C084@mercury.lcs.mit.edu> (Noel Chiappa's message of "Tue, 19 Sep 2023 07:42:29 -0400 (EDT)") References: <20230919114229.55D3418C084@mercury.lcs.mit.edu> Message-ID: <7w4jjqmifx.fsf@junk.nocrew.org> Noel Chiappa writes: > I can't answer that directly, but I will point you at IEN-56, "CRC > Checksum Calculation", by Dave Reed (11-Sep-78): > > https://www.rfc-editor.org/ien/ien56.pdf I can't help but notice the first program is PDP-10 code for running on ITS. ObTUHS: the other is a PDP-11 asssembly language program for Unix. Machine readable versions for both can be procured. From pnr at planet.nl Wed Sep 20 00:10:13 2023 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 19 Sep 2023 16:10:13 +0200 Subject: [TUHS] CRC calculation in the 1980s Message-ID: <3735B8B0-427E-42C6-8343-50B5EE964952@planet.nl> >> Any suggestions as to why the on-the-fly algorithm did not catch on more in the 1980’s? Maybe it was simply less well known than I think? > > Could it have been the per-cpu-second billing that was (fairly?) common at the time. I was only getting in to Unix in the early 90s but saw the tail end of that. Good point, but wasn’t per-cpu-second billing mostly used for big iron? For machines without memory constraint the table method makes the most sense, also if billing was not a factor. >> Any suggestions as to why the on-the-fly algorithm did not catch on more in the 1980’s? Maybe it was simply less well known than I think? > > The CRC algorithm I'm familiar with shows up in Dragon Quest for the Famicom in 1986[1], written in 6502 assembly. Admittedly though I only recognized it due to the EOR with 0x1021 on lines 318-323. That I then only know from a quick and dirty CRC I threw in an XMODEM-CRC client [2] I did to accommodate for a bug in the JH7100 RISC-V board's recovery ROM implementation. Not sure if this is along the lines of the approach you're talking about ... > > [1] - https://gitlab.com/segaloco/dq1disasm/-/blob/master/src/chr3/start_pw.s > [2] - https://gitlab.com/segaloco/riscv-bits/-/blob/master/util/sxj.c Both of these are what I call the bit-wise method: a loop iterating over bytes, with an inner loop iterating over bits. An example of the table method is here: https://github.com/u-boot/u-boot/blob/master/lib/crc16-ccitt.c and an example of the on-the-fly method is here: https://github.com/tio/tio/blob/master/src/xymodem.c#L44-L54 Note how the latter also only has one loop iterating over the bytes, but effectively calculates the table entry ‘on the fly’ for each byte. That is only a handful of instructions more than doing the table lookup. Maybe it is a “stuck in the middle” solution that was quickly forgotten. From paul.winalski at gmail.com Wed Sep 20 01:34:00 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 19 Sep 2023 11:34:00 -0400 Subject: [TUHS] CRC calculation in the 1980s In-Reply-To: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> References: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> Message-ID: For what it's worth, the VAX had a table-driven CRC instruction. It wasn't very popular because on most VAX models it was actually slower than by-hand coding. It is one of the instructions dropped from the microVAX instruction set used on all later VAXen, where it was emulated by the operating system. -Paul W. From tuhs at tuhs.org Wed Sep 20 01:41:23 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Tue, 19 Sep 2023 15:41:23 +0000 Subject: [TUHS] UNIX 4.0 Typesetter Docs on eBay Message-ID: Howdy all, just passing along this auction I spotted on eBay: https://www.ebay.com/itm/256216372208 This is a subset of papers from the Documents for UNIX 4.0 set compiled as a handy volume (and likely a forerunner to the packaged volumes released with System V.)  Missing is the Typing Documents with MM pamphlet which is listed in the TOC of these issues. I've already got one of these or I'd scoop it up myself. Enjoy! - Matt G. P.S. I have seen this and its companion document (Programming Starter Package) with the AT&T deathstar in the upper right rather than Bell Labs and the Bell logo. What I don't know is if it's an exact repackaging just with a change in logo or if those versions have revisions to any of the papers. Most notably as far as version association, the Programming Starter Package includes the Documentation Roadmap, which in my Bell Labs-branded copy, indicates it is for use with UNIX 4.0. I'd be curious if the AT&T variant retains the UNIX 4.0 reference. I'll continue to keep my eye out for an AT&T-branded set, it might reveal some differences. From crossd at gmail.com Wed Sep 20 01:50:19 2023 From: crossd at gmail.com (Dan Cross) Date: Tue, 19 Sep 2023 11:50:19 -0400 Subject: [TUHS] CRC calculation in the 1980s In-Reply-To: References: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> Message-ID: On Tue, Sep 19, 2023 at 11:34 AM Paul Winalski wrote: > For what it's worth, the VAX had a table-driven CRC instruction. It > wasn't very popular because on most VAX models it was actually slower > than by-hand coding. It is one of the instructions dropped from the > microVAX instruction set used on all later VAXen, where it was > emulated by the operating system. I thought that was a general polynomial root finder instruction that used Horner's Method? Obviously it could be used in CRC calculations, but I remember it being more general. I also remember reading about it in the VAX architecture manual and thinking, "wow, that's...something." - Dan C. From pnr at planet.nl Wed Sep 20 02:00:40 2023 From: pnr at planet.nl (Paul Ruizendaal) Date: Tue, 19 Sep 2023 18:00:40 +0200 Subject: [TUHS] CRC calculation in the 1980s In-Reply-To: <20230919114229.55D3418C084@mercury.lcs.mit.edu> References: <20230919114229.55D3418C084@mercury.lcs.mit.edu> Message-ID: <789D4FAC-B6FD-4019-AF7B-C3EAA02E49FC@planet.nl> > On 19 Sep 2023, at 13:42, Noel Chiappa wrote: > >> From: Paul Ruizendaal > >> Any suggestions as to why the on-the-fly algorithm did not catch on >> more in the 1980's? Maybe it was simply less well known than I think? > > I can't answer that directly, but I will point you at IEN-56, "CRC Checksum > Calculation", by Dave Reed (11-Sep-78): > > https://www.rfc-editor.org/ien/ien56.pdf > > Dave wanted the INWG to use a more powerful (in terms of detecting errors) > CRC, instead of the simple summation eventually adopted, in TCP and IP. So he > produced code to implement a particular CRC, to show people that it would not > be particularly expensive (whether in time, or space, I don't alas recall > definitively; speed would have been an important consideration, when > competing with the summation, though). > > This would have been close to the leading edge of our knowledge at the time; > Dave liked playing around with math, and at about that time he did a very > fast DES implementation. > > Noel That is a very interesting paper and it uses the same polynomial. The algorithm seems different from the one that Perez published in 1983, but in the same general direction. So it would seem that this “on-the-fly” method was not (well) known prior to 1983. By the looks of it, it was not well known afterwards either. You were part of the group that invented it and I’m telling you stuff you know better than I do, but the adopted summation in TCP/IP had a lot of advantages: it could be efficiently calculated on 36, 32 and 16 bit machines, it did not care about endianness and was efficient on both two’s and one’s complement machines. I don’t think CRC had all those properties. I don’t think in 1978 the IEN group was aiming for the LAN use case, but as it turned out a few years later the simple summation consumed 25% of a VAX-750 CPU to saturate 3 Mbps ethernet. Even a 30% slowdown in checksumming would have been costly -- but you already pointed out that speed was an important consideration. From paul.winalski at gmail.com Wed Sep 20 02:22:33 2023 From: paul.winalski at gmail.com (Paul Winalski) Date: Tue, 19 Sep 2023 12:22:33 -0400 Subject: [TUHS] CRC calculation in the 1980s In-Reply-To: References: <87246EC9-0AC8-404E-A8DD-508AB046018B@planet.nl> Message-ID: On 9/19/23, Dan Cross wrote: (regarding the CRC instruction on VAX) > > I thought that was a general polynomial root finder instruction that > used Horner's Method? Obviously it could be used in CRC calculations, > but I remember it being more general. I also remember reading about it > in the VAX architecture manual and thinking, "wow, > that's...something." VAX had both a CRC instruction and a polynomial instruction. CRC operated on byte strings and did fixed-point calculation. The polynomial instructions (POLYF and POLYD) operated on single-precision (F) or double-precision (D) data. Unfortunately the algorithm used by POLYF and POLYD wasn't very accurate--there were other algorithms that did a better job preserving precision. Anyone doing serious calculations in floating point avoided POLYF and POLYD. These were two more of the instructions that were dropped from the microVAX architecture and emulated by the OS. -Paul W. From ori at eigenstate.org Wed Sep 20 02:56:40 2023 From: ori at eigenstate.org (Ori Bernstein) Date: Tue, 19 Sep 2023 12:56:40 -0400 Subject: [TUHS] shell escapes in utilities In-Reply-To: References: Message-ID: <20230919125640.b06007d7abef554577987ef7@eigenstate.org> On Tue, 1 Aug 2023 08:30:41 -0700, ron minnich wrote: > I'm ok with things like ed, I'm more thinking of situations where > people would (e.g.) use xdvi to view a file, and Bad Things Happened. > I don't think ed counts, unless we're that worried about scripts. > well, it's a problem when things (*cough*patch*cough*) shell out to ed... -- Ori Bernstein From rminnich at gmail.com Wed Sep 20 03:04:54 2023 From: rminnich at gmail.com (ron minnich) Date: Tue, 19 Sep 2023 10:04:54 -0700 Subject: [TUHS] shell escapes in utilities In-Reply-To: <20230919125640.b06007d7abef554577987ef7@eigenstate.org> References: <20230919125640.b06007d7abef554577987ef7@eigenstate.org> Message-ID: yeah, good point. On Tue, Sep 19, 2023 at 9:56 AM Ori Bernstein wrote: > > On Tue, 1 Aug 2023 08:30:41 -0700, ron minnich wrote: > > > I'm ok with things like ed, I'm more thinking of situations where > > people would (e.g.) use xdvi to view a file, and Bad Things Happened. > > I don't think ed counts, unless we're that worried about scripts. > > > > well, it's a problem when things (*cough*patch*cough*) shell out > to ed... > > -- > Ori Bernstein From tuhs at tuhs.org Wed Sep 20 06:32:15 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Tue, 19 Sep 2023 20:32:15 +0000 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition Message-ID: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> I haven't known when or how to bring up this project idea, but figure I might as well start putting feelers out since my Dragon Quest project is starting to slow down and I might focus back on UNIX manual stuff. So something painfully missing from my and I'm sure plenty of other folks' libraries is a nice, modern paper UNIX manual that takes the past few decades into consideration. The GNU project, BSDs, etc. ship manpages of course, and there's the POSIX manpages, but I'm a sucker for a good print manual. Something I'm thinking of producing as a "deliverable" of sorts from my documentation research is a new-age UNIX manual, derived as closely as possible from the formal UNIX documentation lineages (so Research, SysV, and BSD pages), but: 1. Including subsequent POSIX requirements 2. Including an informational section in each page with a little history and some notes about current implementations, if applicable. This would include notes about "dead on the vine" stuff like things plucked from the CB-UNIX, MERT/PG, and PWB lines. The history part could even be a separate book, that way the manual itself could stay tight and focused. This would also be a good place for luminaries to provide reflections on their involvement in given pieces. One of the main questions that I have in mind is what the legal landscape of producing such a thing would entail. At the very least, to actually call it a UNIX Programmer's Manual, it would probably need to pass some sort of compliance with the materials The Open Group publishes. That said, the ownership of the IP as opposed to the trademarks is a little less certain, so I would be a bit curious who all would be involved in specifically getting copyright approval to publish anything that happened the commercial line after the early 80s, so like new text produced after 1982. I presume anything covered by the Caldera license at least could be published at-cost, but not for a profit (which I'm not looking for anyway.) Additionally, if possible, I'd love to run down some authorship information and make sure folks who wrote stuff up over time are properly credited, if not on each page ala OWNER at least in a Acknowledgements section in the front. As far as production, I personally would want to do a run with a couple of different cover styles, comb bound, maybe one echoing the original Bell Laboratories UNIX User's Manual-style cover complete with Bell logo, another using the original USENIX Beastie cover, etc. but that also then calls into question more copyrights to coordinate, especially with the way the Bell logo is currently owned, that could get complicated. Anywho, anyone know of any such efforts like this? If I actually got such a project going in earnest, would folks find themselves interested in such a publication? In any case I do intend to start on a typesetter sources version of this project sometime in the next year or so, but ideally I would want it to blossom into something that could result in some physical media. This idea isn't even half-baked yet by the way, so just know I don't have a roadmap in place, it's just something I see being a cool potential project over the coming years. - Matt G. From lm at mcvoy.com Wed Sep 20 09:39:25 2023 From: lm at mcvoy.com (Larry McVoy) Date: Tue, 19 Sep 2023 16:39:25 -0700 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> Message-ID: <20230919233925.GB28844@mcvoy.com> One of the projects I thought I'd do in my retirement, but haven't done, was to provide man page / paper as in "a paper", not tree paper, versions of all the GNU info stuff. I could not be less thrilled with info, yeah there are ways to deal, but it just isn't as good (to me) as how Unix did docs. It's like they want to force emacs on us to read docs. I'd start with groff. So I'm a little off topic but if people wanted to work on that, I'd be up for that project. It's not as big as what you are saying but it's pretty big, I think we just start with something, see if we can get debian/ubuntu to pick it up, lather, rinse repeat. In fact if we just get the groff project to pick up our stuff, all the distros will get that eventually. The one drawback I see is people might want to provide info and man docs. My personal preference is that the info stuff goes away but I have learned I don't get what I want. So there may be a period of time where both need to be maintained. On Tue, Sep 19, 2023 at 08:32:15PM +0000, segaloco via TUHS wrote: > I haven't known when or how to bring up this project idea, but figure I might as well start putting feelers out since my Dragon Quest project is starting to slow down and I might focus back on UNIX manual stuff. > > So something painfully missing from my and I'm sure plenty of other folks' libraries is a nice, modern paper UNIX manual that takes the past few decades into consideration. The GNU project, BSDs, etc. ship manpages of course, and there's the POSIX manpages, but I'm a sucker for a good print manual. Something I'm thinking of producing as a "deliverable" of sorts from my documentation research is a new-age UNIX manual, derived as closely as possible from the formal UNIX documentation lineages (so Research, SysV, and BSD pages), but: > > 1. Including subsequent POSIX requirements > 2. Including an informational section in each page with a little history and some notes about current implementations, if applicable. This would include notes about "dead on the vine" stuff like things plucked from the CB-UNIX, MERT/PG, and PWB lines. The history part could even be a separate book, that way the manual itself could stay tight and focused. This would also be a good place for luminaries to provide reflections on their involvement in given pieces. > > One of the main questions that I have in mind is what the legal landscape of producing such a thing would entail. At the very least, to actually call it a UNIX Programmer's Manual, it would probably need to pass some sort of compliance with the materials The Open Group publishes. That said, the ownership of the IP as opposed to the trademarks is a little less certain, so I would be a bit curious who all would be involved in specifically getting copyright approval to publish anything that happened the commercial line after the early 80s, so like new text produced after 1982. I presume anything covered by the Caldera license at least could be published at-cost, but not for a profit (which I'm not looking for anyway.) > > Additionally, if possible, I'd love to run down some authorship information and make sure folks who wrote stuff up over time are properly credited, if not on each page ala OWNER at least in a Acknowledgements section in the front. > > As far as production, I personally would want to do a run with a couple of different cover styles, comb bound, maybe one echoing the original Bell Laboratories UNIX User's Manual-style cover complete with Bell logo, another using the original USENIX Beastie cover, etc. but that also then calls into question more copyrights to coordinate, especially with the way the Bell logo is currently owned, that could get complicated. > > Anywho, anyone know of any such efforts like this? If I actually got such a project going in earnest, would folks find themselves interested in such a publication? In any case I do intend to start on a typesetter sources version of this project sometime in the next year or so, but ideally I would want it to blossom into something that could result in some physical media. This idea isn't even half-baked yet by the way, so just know I don't have a roadmap in place, it's just something I see being a cool potential project over the coming years. > > - Matt G. -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From athornton at gmail.com Wed Sep 20 09:43:16 2023 From: athornton at gmail.com (Adam Thornton) Date: Tue, 19 Sep 2023 16:43:16 -0700 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> Message-ID: I would be very interested in this and haven't heard of anyone trying it. I actually just came to post a question-which-is-really-a-gripe about man pages, because this seems like the group that might be able to answer it, and given the existence of this topic, it seems like the sort of thing that should end up in the Definitive Unix Programmer's Manual. Why (and when) did GNU drop the HISTORY section from its man pages? It actually was really useful to me today to find out that sed appears in v7, but cut didn't come along until AT&T System III. I had to use a Mac's man pages, because the ones on my Linux systems don't have HISTORY sections. (this is in the context of: is there a better way to get the org/reponame information from a git repository than git ls-remote --get-url origin | sed -e "s|^origin$|$(git config --get user.name)/$(basename $(pwd))" -e 's/\.git$//' | rev | cut -d / -f 1-2 | cut -d : -f 1 | rev The more elegant and complicated regex sed expressions that my better-at-regex-than-I coworkers suggested to do away with rev-cut-rev do not manage to work on both (currentish) Mac and Linux, which are my target audience, Rubin Observatory development being split about 75/25; stripping the possible trailing .git is about all I can count on working in both environments' sed, but I can count on having *some* sed and coreutils. If anyone here has a better answer for getting that info out I'd appreciate it) Adam On Tue, Sep 19, 2023 at 1:32 PM segaloco via TUHS wrote: > I haven't known when or how to bring up this project idea, but figure I > might as well start putting feelers out since my Dragon Quest project is > starting to slow down and I might focus back on UNIX manual stuff. > > So something painfully missing from my and I'm sure plenty of other folks' > libraries is a nice, modern paper UNIX manual that takes the past few > decades into consideration. The GNU project, BSDs, etc. ship manpages of > course, and there's the POSIX manpages, but I'm a sucker for a good print > manual. Something I'm thinking of producing as a "deliverable" of sorts > from my documentation research is a new-age UNIX manual, derived as closely > as possible from the formal UNIX documentation lineages (so Research, SysV, > and BSD pages), but: > > 1. Including subsequent POSIX requirements > 2. Including an informational section in each page with a little > history and some notes about current implementations, if applicable. This > would include notes about "dead on the vine" stuff like things plucked from > the CB-UNIX, MERT/PG, and PWB lines. The history part could even be a > separate book, that way the manual itself could stay tight and focused. > This would also be a good place for luminaries to provide reflections on > their involvement in given pieces. > > One of the main questions that I have in mind is what the legal landscape > of producing such a thing would entail. At the very least, to actually > call it a UNIX Programmer's Manual, it would probably need to pass some > sort of compliance with the materials The Open Group publishes. That said, > the ownership of the IP as opposed to the trademarks is a little less > certain, so I would be a bit curious who all would be involved in > specifically getting copyright approval to publish anything that happened > the commercial line after the early 80s, so like new text produced after > 1982. I presume anything covered by the Caldera license at least could be > published at-cost, but not for a profit (which I'm not looking for anyway.) > > Additionally, if possible, I'd love to run down some authorship > information and make sure folks who wrote stuff up over time are properly > credited, if not on each page ala OWNER at least in a Acknowledgements > section in the front. > > As far as production, I personally would want to do a run with a couple of > different cover styles, comb bound, maybe one echoing the original Bell > Laboratories UNIX User's Manual-style cover complete with Bell logo, > another using the original USENIX Beastie cover, etc. but that also then > calls into question more copyrights to coordinate, especially with the way > the Bell logo is currently owned, that could get complicated. > > Anywho, anyone know of any such efforts like this? If I actually got such > a project going in earnest, would folks find themselves interested in such > a publication? In any case I do intend to start on a typesetter sources > version of this project sometime in the next year or so, but ideally I > would want it to blossom into something that could result in some physical > media. This idea isn't even half-baked yet by the way, so just know I > don't have a roadmap in place, it's just something I see being a cool > potential project over the coming years. > > - Matt G. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ken.unix.guy at gmail.com Wed Sep 20 09:47:42 2023 From: ken.unix.guy at gmail.com (KenUnix) Date: Tue, 19 Sep 2023 19:47:42 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> Message-ID: Matt, There was a book printed by Newnes titled UNIX pocket book. It covers System-V Xenix, BSD 4.3, C-shell, plus the usual commands. Also has sections dedicated to system administration, vi, rebuilding the kernel, Bourne Shell, C-Shell, Korn Shell, etc... It was written by Steve Heath 1998 "ISBN 0 7506 410 88" 340 pages. A good read. Only problem, for my eyes it is physically too small. -Ken On Tue, Sep 19, 2023 at 4:32 PM segaloco via TUHS wrote: > I haven't known when or how to bring up this project idea, but figure I > might as well start putting feelers out since my Dragon Quest project is > starting to slow down and I might focus back on UNIX manual stuff. > > So something painfully missing from my and I'm sure plenty of other folks' > libraries is a nice, modern paper UNIX manual that takes the past few > decades into consideration. The GNU project, BSDs, etc. ship manpages of > course, and there's the POSIX manpages, but I'm a sucker for a good print > manual. Something I'm thinking of producing as a "deliverable" of sorts > from my documentation research is a new-age UNIX manual, derived as closely > as possible from the formal UNIX documentation lineages (so Research, SysV, > and BSD pages), but: > > 1. Including subsequent POSIX requirements > 2. Including an informational section in each page with a little > history and some notes about current implementations, if applicable. This > would include notes about "dead on the vine" stuff like things plucked from > the CB-UNIX, MERT/PG, and PWB lines. The history part could even be a > separate book, that way the manual itself could stay tight and focused. > This would also be a good place for luminaries to provide reflections on > their involvement in given pieces. > > One of the main questions that I have in mind is what the legal landscape > of producing such a thing would entail. At the very least, to actually > call it a UNIX Programmer's Manual, it would probably need to pass some > sort of compliance with the materials The Open Group publishes. That said, > the ownership of the IP as opposed to the trademarks is a little less > certain, so I would be a bit curious who all would be involved in > specifically getting copyright approval to publish anything that happened > the commercial line after the early 80s, so like new text produced after > 1982. I presume anything covered by the Caldera license at least could be > published at-cost, but not for a profit (which I'm not looking for anyway.) > > Additionally, if possible, I'd love to run down some authorship > information and make sure folks who wrote stuff up over time are properly > credited, if not on each page ala OWNER at least in a Acknowledgements > section in the front. > > As far as production, I personally would want to do a run with a couple of > different cover styles, comb bound, maybe one echoing the original Bell > Laboratories UNIX User's Manual-style cover complete with Bell logo, > another using the original USENIX Beastie cover, etc. but that also then > calls into question more copyrights to coordinate, especially with the way > the Bell logo is currently owned, that could get complicated. > > Anywho, anyone know of any such efforts like this? If I actually got such > a project going in earnest, would folks find themselves interested in such > a publication? In any case I do intend to start on a typesetter sources > version of this project sometime in the next year or so, but ideally I > would want it to blossom into something that could result in some physical > media. This idea isn't even half-baked yet by the way, so just know I > don't have a roadmap in place, it's just something I see being a cool > potential project over the coming years. > > - Matt G. > -- End of line JOB TERMINATED -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Wed Sep 20 11:26:47 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Wed, 20 Sep 2023 01:26:47 +0000 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <20230919233925.GB28844@mcvoy.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: > I'd start with groff. > > So I'm a little off topic but if people wanted to work on that, I'd be > up for that project. It's not as big as what you are saying but it's > pretty big, I think we just start with something, see if we can get > debian/ubuntu to pick it up, lather, rinse repeat. In fact if we > just get the groff project to pick up our stuff, all the distros will > get that eventually. > > -- > --- > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat That's an excellent point, the beauty of UNIX being a granular system is that such an effort wouldn't need to be a "start at page 1 and finish at page whatever", but could be done piecemeal. Groff would also be a great candidate due to the preponderance of supporting secondary papers, like the NROFF/TROFF manual, different macro definitions, etc. That does then get into the prospect of the secondary papers too, likewise excellent references to this day on a number of subjects that I personally would love to have modernized versions of. Well if anyone catches wind of such a project kicking off in some way elsewhere, know that I'm certainly interested in what I can contribute. What my work towards this eventual goal will probably continue to look like for now though is just doing my diff analysis of manual versions, as one of my principle goals there was to identify the apparent last common ancestor of Research, PWB, and BSD lineages, at least as far as documentation is concerned. Common sense would just say research V7 but there are little tidbits here and there between V6 and V7 that don't show up in other places, just tiny little nuanced things for the most part. I haven't done this part of the analysis at all but a causal glance at a 32V manual diffed with a V7 manual reveals some changes that don't appear to be related to the portability work. But I'm not going to comment on that further without analysis to back it up, just some anecdotal observations at present. > Why (and when) did GNU drop the HISTORY section from its man pages? > > Adam Did GNU ever have a HISTORY section? I just plucked a couple books off the shelf, I don't see HISTORY in the V10, 4.4BSD, or SVR4 books, so probably a later invention in the BSD line that didn't get picked up by other UNIX-likes? Looking at a few illumos manpages, they also don't appear to have a HISTORY section. They appear to be there on macOS, probably as a result of the FreeBSD origins of macOS user space. That said, I also appreciate the HISTORY section, it's tipped me off to things to study that I didn't know on a few occasions. - Matt G. From tuhs at tuhs.org Wed Sep 20 11:30:37 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Wed, 20 Sep 2023 01:30:37 +0000 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: <5RzxjSGC9j4riL9Kr7cVp4UwTY2SVNxyNxQNG5RCZLh874BzbGP9ZS5U7IErArwrvMm6Ii40GhHK7n009tpnToX78D-3MQSt9xx2XqDnKuc=@protonmail.com> > > I'd start with groff. > > > > So I'm a little off topic but if people wanted to work on that, I'd be > > up for that project. It's not as big as what you are saying but it's > > pretty big, I think we just start with something, see if we can get > > debian/ubuntu to pick it up, lather, rinse repeat. In fact if we > > just get the groff project to pick up our stuff, all the distros will > > get that eventually. > > > > -- > > --- > > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat > > > That's an excellent point, the beauty of UNIX being a granular system is that such an effort wouldn't need to be a "start at page 1 and finish at page whatever", but could be done piecemeal. Groff would also be a great candidate due to the preponderance of supporting secondary papers, like the NROFF/TROFF manual, different macro definitions, etc. That does then get into the prospect of the secondary papers too, likewise excellent references to this day on a number of subjects that I personally would love to have modernized versions of. > > Well if anyone catches wind of such a project kicking off in some way elsewhere, know that I'm certainly interested in what I can contribute. What my work towards this eventual goal will probably continue to look like for now though is just doing my diff analysis of manual versions, as one of my principle goals there was to identify the apparent last common ancestor of Research, PWB, and BSD lineages, at least as far as documentation is concerned. Common sense would just say research V7 but there are little tidbits here and there between V6 and V7 that don't show up in other places, just tiny little nuanced things for the most part. I haven't done this part of the analysis at all but a causal glance at a 32V manual diffed with a V7 manual reveals some changes that don't appear to be related to the portability work. But I'm not going to comment on that further without analysis to back it up, just some anecdotal observations at present. > > > Why (and when) did GNU drop the HISTORY section from its man pages? > > > > Adam > > > Did GNU ever have a HISTORY section? I just plucked a couple books off the shelf, I don't see HISTORY in the V10, 4.4BSD, or SVR4 books, so probably a later invention in the BSD line that didn't get picked up by other UNIX-likes? Looking at a few illumos manpages, they also don't appear to have a HISTORY section. They appear to be there on macOS, probably as a result of the FreeBSD origins of macOS user space. That said, I also appreciate the HISTORY section, it's tipped me off to things to study that I didn't know on a few occasions. > > - Matt G. Sorry for the double bump, don't want to lie, just found a few pages in the 4.4BSD manual with a HISTORY section. Checked the same pages in V10, SVR4, and 4.3BSD, no dice, so maybe 4.4BSD at the earliest? Of course I could just grep this but where's the fun in that (and I'm not at a computer I have a UNIX tree on right now...) - Matt G. From clemc at ccc.com Wed Sep 20 12:09:51 2023 From: clemc at ccc.com (Clem Cole) Date: Tue, 19 Sep 2023 21:09:51 -0500 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <20230919233925.GB28844@mcvoy.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: +1 ᐧ On Tue, Sep 19, 2023 at 6:39 PM Larry McVoy wrote: > One of the projects I thought I'd do in my retirement, but haven't done, > was to provide man page / paper as in "a paper", not tree paper, versions > of all the GNU info stuff. I could not be less thrilled with info, yeah > there are ways to deal, but it just isn't as good (to me) as how Unix did > docs. It's like they want to force emacs on us to read docs. > > I'd start with groff. > > So I'm a little off topic but if people wanted to work on that, I'd be > up for that project. It's not as big as what you are saying but it's > pretty big, I think we just start with something, see if we can get > debian/ubuntu to pick it up, lather, rinse repeat. In fact if we > just get the groff project to pick up our stuff, all the distros will > get that eventually. > > The one drawback I see is people might want to provide info and man > docs. My personal preference is that the info stuff goes away but I > have learned I don't get what I want. So there may be a period of > time where both need to be maintained. > > On Tue, Sep 19, 2023 at 08:32:15PM +0000, segaloco via TUHS wrote: > > I haven't known when or how to bring up this project idea, but figure I > might as well start putting feelers out since my Dragon Quest project is > starting to slow down and I might focus back on UNIX manual stuff. > > > > So something painfully missing from my and I'm sure plenty of other > folks' libraries is a nice, modern paper UNIX manual that takes the past > few decades into consideration. The GNU project, BSDs, etc. ship manpages > of course, and there's the POSIX manpages, but I'm a sucker for a good > print manual. Something I'm thinking of producing as a "deliverable" of > sorts from my documentation research is a new-age UNIX manual, derived as > closely as possible from the formal UNIX documentation lineages (so > Research, SysV, and BSD pages), but: > > > > 1. Including subsequent POSIX requirements > > 2. Including an informational section in each page with a little > history and some notes about current implementations, if applicable. This > would include notes about "dead on the vine" stuff like things plucked from > the CB-UNIX, MERT/PG, and PWB lines. The history part could even be a > separate book, that way the manual itself could stay tight and focused. > This would also be a good place for luminaries to provide reflections on > their involvement in given pieces. > > > > One of the main questions that I have in mind is what the legal > landscape of producing such a thing would entail. At the very least, to > actually call it a UNIX Programmer's Manual, it would probably need to pass > some sort of compliance with the materials The Open Group publishes. That > said, the ownership of the IP as opposed to the trademarks is a little less > certain, so I would be a bit curious who all would be involved in > specifically getting copyright approval to publish anything that happened > the commercial line after the early 80s, so like new text produced after > 1982. I presume anything covered by the Caldera license at least could be > published at-cost, but not for a profit (which I'm not looking for anyway.) > > > > Additionally, if possible, I'd love to run down some authorship > information and make sure folks who wrote stuff up over time are properly > credited, if not on each page ala OWNER at least in a Acknowledgements > section in the front. > > > > As far as production, I personally would want to do a run with a couple > of different cover styles, comb bound, maybe one echoing the original Bell > Laboratories UNIX User's Manual-style cover complete with Bell logo, > another using the original USENIX Beastie cover, etc. but that also then > calls into question more copyrights to coordinate, especially with the way > the Bell logo is currently owned, that could get complicated. > > > > Anywho, anyone know of any such efforts like this? If I actually got > such a project going in earnest, would folks find themselves interested in > such a publication? In any case I do intend to start on a typesetter > sources version of this project sometime in the next year or so, but > ideally I would want it to blossom into something that could result in some > physical media. This idea isn't even half-baked yet by the way, so just > know I don't have a roadmap in place, it's just something I see being a > cool potential project over the coming years. > > > > - Matt G. > > -- > --- > Larry McVoy Retired to fishing > http://www.mcvoy.com/lm/boat > -------------- next part -------------- An HTML attachment was scrubbed... URL: From athornton at gmail.com Wed Sep 20 12:25:13 2023 From: athornton at gmail.com (Adam Thornton) Date: Tue, 19 Sep 2023 19:25:13 -0700 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: Yeah, I'm less angry at GNU now--I didn't search as hard, but when I found out 4.3BSD didn't have HISTORY (and neither does 2.11BSD, which is still actively-ish maintained) then I figured it wasn't something classical that GNU dropped, just never imported. I feel like it existed on SunOS and Solaris but I might be wrong about that? Was it really FreeBSD that introduced it? Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Wed Sep 20 12:50:41 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Wed, 20 Sep 2023 02:50:41 +0000 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: > Yeah, I'm less angry at GNU now--I didn't search as hard, but when I found out 4.3BSD didn't have HISTORY (and neither does 2.11BSD, which is still actively-ish maintained) then I figured it wasn't something classical that GNU dropped, just never imported. I feel like it existed on SunOS and Solaris but I might be wrong about that? Was it really FreeBSD that introduced it? > > Adam As per my silly bump (really I hate double messages, I'm probably being harder on myself than any of you would be :P) it seems there are some HISTORY sections in the print 4.4BSD set I have on my shelf. The pages I did spot the section in do not have it in the corresponding 4.3BSD volumes, although I'm basing this on a casual flip through paper books at present, not looking down in /usr/man on any given distro. None of the Bell-adjacent stuff I've flipped through has any HISTORY sections though. - Matt G. From reed at reedmedia.net Wed Sep 20 16:16:12 2023 From: reed at reedmedia.net (Jeremy C. Reed) Date: Wed, 20 Sep 2023 06:16:12 +0000 (UTC) Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> Message-ID: > Additionally, if possible, I'd love to run down some authorship > information and make sure folks who wrote stuff up over time are > properly credited, if not on each page ala OWNER at least in a > Acknowledgements section in the front. My NetBSD section 8 was two long printed book volumes. 68 printed pages 1461 through 1529 had 109 different licences (no duplicate verbiage) and corresponding copyrights and required acknowledgement statements. The permuted index was also 68 pages long. The books will be several volumes long and likely ten thousand of pages. (Not very many sold, it was large learning project.) From meillo at marmaro.de Wed Sep 20 17:19:25 2023 From: meillo at marmaro.de (markus schnalke) Date: Wed, 20 Sep 2023 09:19:25 +0200 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> Message-ID: <1qirUb-7WS-00@marmaro.de> Hoi. [2023-09-20 06:16] "Jeremy C. Reed" > > Additionally, if possible, I'd love to run down some authorship > > information and make sure folks who wrote stuff up over time are > > properly credited, if not on each page ala OWNER at least in a > > Acknowledgements section in the front. > > My NetBSD section 8 was two long printed book volumes. > > 68 printed pages 1461 through 1529 had 109 different licences (no > duplicate verbiage) and corresponding copyrights and required > acknowledgement statements. > > The permuted index was also 68 pages long. > > The books will be several volumes long and likely ten thousand of > pages. > > (Not very many sold, it was large learning project.) Good point. Original manpages reprinted and collected would not interest me much. That stuff is available online and to me more a reference than something I take to bed reading in the evening. This kind of content also leads to the copyright annoyance described above. Much more am I interested in something like Doug's Unix Reader, i.e. commented history and folklore around the tools, collected and arranged. Of course, this is much more work, but it's more original work and to me of greater uniqueness and value. Thus, let's make the manpages *only* contain the History section! ... plus Changes, Bugs, Quirks, Versions, Differences, ... Of course, that would be the Unix Historian's Manual then. ;-) meillo P.S. Adding good manpages to info-only or manpage-stub programs would be a relief, too. From chet.ramey at case.edu Wed Sep 20 23:23:00 2023 From: chet.ramey at case.edu (Chet Ramey) Date: Wed, 20 Sep 2023 09:23:00 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> Message-ID: <2c971548-14c1-420b-ba7b-5c2272da89b2@case.edu> On 9/19/23 7:43 PM, Adam Thornton wrote: > Why (and when) did GNU drop the HISTORY section from its man pages? We never had it. It originated with 4.4 BSD, and none of us picked it up. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ From alx.manpages at gmail.com Thu Sep 21 00:12:26 2023 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Wed, 20 Sep 2023 16:12:26 +0200 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <2c971548-14c1-420b-ba7b-5c2272da89b2@case.edu> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <2c971548-14c1-420b-ba7b-5c2272da89b2@case.edu> Message-ID: On Wed, Sep 20, 2023, 15:23 Chet Ramey wrote: > On 9/19/23 7:43 PM, Adam Thornton wrote: > > > Why (and when) did GNU drop the HISTORY section from its man pages? > > We never had it. It originated with 4.4 BSD, and none of us picked it up. > I recently added a HISTORY section to the Linux man-pages (chapters 2 and 3), and we moved there info that was scattered across various sections. If your plan includes man2 and man3, you could maybe just contribute some extra details to those pages' HISTORY section and you're done with that part. We also produce a PDF book (this was done by Deri James from gropdf): . Cheers, > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Thu Sep 21 00:57:01 2023 From: imp at bsdimp.com (Warner Losh) Date: Wed, 20 Sep 2023 15:57:01 +0100 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <5RzxjSGC9j4riL9Kr7cVp4UwTY2SVNxyNxQNG5RCZLh874BzbGP9ZS5U7IErArwrvMm6Ii40GhHK7n009tpnToX78D-3MQSt9xx2XqDnKuc=@protonmail.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <5RzxjSGC9j4riL9Kr7cVp4UwTY2SVNxyNxQNG5RCZLh874BzbGP9ZS5U7IErArwrvMm6Ii40GhHK7n009tpnToX78D-3MQSt9xx2XqDnKuc=@protonmail.com> Message-ID: On Wed, Sep 20, 2023, 2:30 AM segaloco via TUHS wrote: > > > I'd start with groff. > > > > > > So I'm a little off topic but if people wanted to work on that, I'd be > > > up for that project. It's not as big as what you are saying but it's > > > pretty big, I think we just start with something, see if we can get > > > debian/ubuntu to pick it up, lather, rinse repeat. In fact if we > > > just get the groff project to pick up our stuff, all the distros will > > > get that eventually. > > > > > > -- > > > --- > > > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat > > > > > > That's an excellent point, the beauty of UNIX being a granular system is > that such an effort wouldn't need to be a "start at page 1 and finish at > page whatever", but could be done piecemeal. Groff would also be a great > candidate due to the preponderance of supporting secondary papers, like the > NROFF/TROFF manual, different macro definitions, etc. That does then get > into the prospect of the secondary papers too, likewise excellent > references to this day on a number of subjects that I personally would love > to have modernized versions of. > > > > Well if anyone catches wind of such a project kicking off in some way > elsewhere, know that I'm certainly interested in what I can contribute. > What my work towards this eventual goal will probably continue to look like > for now though is just doing my diff analysis of manual versions, as one of > my principle goals there was to identify the apparent last common ancestor > of Research, PWB, and BSD lineages, at least as far as documentation is > concerned. Common sense would just say research V7 but there are little > tidbits here and there between V6 and V7 that don't show up in other > places, just tiny little nuanced things for the most part. I haven't done > this part of the analysis at all but a causal glance at a 32V manual diffed > with a V7 manual reveals some changes that don't appear to be related to > the portability work. But I'm not going to comment on that further without > analysis to back it up, just some anecdotal observations at present. > > > > > Why (and when) did GNU drop the HISTORY section from its man pages? > > > > > > Adam > > > > > > Did GNU ever have a HISTORY section? I just plucked a couple books off > the shelf, I don't see HISTORY in the V10, 4.4BSD, or SVR4 books, so > probably a later invention in the BSD line that didn't get picked up by > other UNIX-likes? Looking at a few illumos manpages, they also don't appear > to have a HISTORY section. They appear to be there on macOS, probably as a > result of the FreeBSD origins of macOS user space. That said, I also > appreciate the HISTORY section, it's tipped me off to things to study that > I didn't know on a few occasions. > > > > - Matt G. > > Sorry for the double bump, don't want to lie, just found a few pages in > the 4.4BSD manual with a HISTORY section. Checked the same pages in V10, > SVR4, and 4.3BSD, no dice, so maybe 4.4BSD at the earliest? Of course I > could just grep this but where's the fun in that (and I'm not at a computer > I have a UNIX tree on right now...) > 4.4BSD almost certainly had some history. All the current BSDs have a HISTORY section for many of their pages. And we are busy borrowing each other's primary research for them... though it a man page, not a treatis on the evolution of signals since V7. Nor do the vast majority of command line flags have mention. Those that do are either 4BSD vs System V or XXXBSD vs Linux (and maybe a few FooBSD vs BarBSD, but those are rare). Warner > -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Thu Sep 21 01:40:50 2023 From: crossd at gmail.com (Dan Cross) Date: Wed, 20 Sep 2023 11:40:50 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <5RzxjSGC9j4riL9Kr7cVp4UwTY2SVNxyNxQNG5RCZLh874BzbGP9ZS5U7IErArwrvMm6Ii40GhHK7n009tpnToX78D-3MQSt9xx2XqDnKuc=@protonmail.com> Message-ID: On Wed, Sep 20, 2023 at 10:57 AM Warner Losh wrote: > [snip] > 4.4BSD almost certainly had some history. All the current BSDs have a HISTORY section for many of their pages. And we are busy borrowing each other's primary research for them... though it a man page, not a treatis on the evolution of signals since V7. Nor do the vast majority of command line flags have mention. Those that do are either 4BSD vs System V or XXXBSD vs Linux (and maybe a few FooBSD vs BarBSD, but those are rare). My recollection is that the HISTORY stuff started showing up in BSD man pages around the time of Net/2? - Dan C. From lm at mcvoy.com Thu Sep 21 05:56:35 2023 From: lm at mcvoy.com (Larry McVoy) Date: Wed, 20 Sep 2023 12:56:35 -0700 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: <20230920195635.GG28844@mcvoy.com> On Tue, Sep 19, 2023 at 07:25:13PM -0700, Adam Thornton wrote: > Yeah, I'm less angry at GNU now--I didn't search as hard, but when I found > out 4.3BSD didn't have HISTORY (and neither does 2.11BSD, which is still > actively-ish maintained) then I figured it wasn't something classical that > GNU dropped, just never imported. I feel like it existed on SunOS and > Solaris but I might be wrong about that? Was it really FreeBSD that > introduced it? I don't think SunOS had it. The SunOS man pages were pretty terse, the powers that be wanted just the facts. I remember get beat up for putting examples in some of the man pages I wrote, they got taken out "because examples are for user guides, man pages are not user guides, they are a reference guide" or something like that. I like examples in man pages, I think it jump starts you into using the program more easily but maybe that's just me. -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From tuhs at tuhs.org Thu Sep 21 06:52:23 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Wed, 20 Sep 2023 20:52:23 +0000 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <20230920195635.GG28844@mcvoy.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: > > Yeah, I'm less angry at GNU now--I didn't search as hard, but when I found > > out 4.3BSD didn't have HISTORY (and neither does 2.11BSD, which is still > > actively-ish maintained) then I figured it wasn't something classical that > > GNU dropped, just never imported. I feel like it existed on SunOS and > > Solaris but I might be wrong about that? Was it really FreeBSD that > > introduced it? > > > I don't think SunOS had it. The SunOS man pages were pretty terse, the > powers that be wanted just the facts. I remember get beat up for putting > examples in some of the man pages I wrote, they got taken out "because > examples are for user guides, man pages are not user guides, they are a > reference guide" or something like that. > > I like examples in man pages, I think it jump starts you into using the > program more easily but maybe that's just me. > -- > --- > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat I'm glad I'm not the only one, do I find the dearth of examples on many reference manuals to be quite annoying. The only compelling (to me) reason I've ever heard for flat out avoiding examples is to not imply best practices. For instance, in my own work, there are times I purposefully have to not provide use-case examples of some generic bit I put together because then I find it gets used very narrowly, as if the example given is "how it should be used" rather than "a way it can be used". Granted, that is a product of perception, not intention, so outright avoiding guidance altogether or not, you're always going to have people who read something and accept that as lore rather than just a helpful suggestion or narrow demonstration. It's kinda like providing a reference implementation of some general interface or service. If it works well and is general, folks are just going to use that reference implementation rather than customizing their own logic to drive the interface, defeating the purpose of trying to standardize an interface instead of just providing a distinct product. Kinda like X these days, how many *new* X servers or clients have popped up lately? Is everything X.org now? I don't know enough to speculate on that but might serve as a good example. Another way I think about that caveat is, say one is providing an example of fseek. A narrow example of seeking to the end of a file would be an effective example, a perfect use-case for fseek. However, presenting instead a bulkier example that demonstrates determining filesize *using* the fact you can fseek to the end and then ftell the position, while also demonstrating use of fseek perfectly well, might embed itself in someone's mind as suggesting *the* way to do this sort of thing in C. Granted, again, that's up to user perception, but those are the kinds of scenarios that do give me pause when drafting examples for documentation: The concern that someone will take an example as an absolute in every angle that it touches. I think in a full UNIX manual, constituting both volumes (or whatever other permutations of manpages and doc papers), it does make a bit more sense to keep examples terse because for many materials needing detail, one could simply cite the paper that is also considered a part of that standard documentation package. Sadly with the fragmentary nature of documentation these days, that sort of packaged manual doesn't really exist anymore, and all of what would have gone into that arrangement of documents is out leading multiple lives as info, HTML pages, PDFs, etc probably from TeX sources. /usr(/share)/doc is a thing of the past in its old way of being. I guess these days what with the internet and all, it's less important for each and every scrap of information being succinctly accessible by a common mechanism on the system itself...but hoo boy would it be nice...I guess that's what they wanted info to be but...yeah that needs no further comment from me. - Matt G. From alan.coopersmith at oracle.com Thu Sep 21 07:15:09 2023 From: alan.coopersmith at oracle.com (Alan Coopersmith) Date: Wed, 20 Sep 2023 14:15:09 -0700 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: <94c444c6-7067-404f-a440-c960076cf3a8@oracle.com> On 9/19/23 19:25, Adam Thornton wrote: > Yeah, I'm less angry at GNU now--I didn't search as hard, but when I found out > 4.3BSD didn't have HISTORY (and neither does 2.11BSD, which is still > actively-ish maintained) then I figured it wasn't something classical that GNU > dropped, just never imported.  I feel like it existed on SunOS and Solaris but I > might be wrong about that?  Was it really FreeBSD that introduced it? I just checked the nroff sources for the man pages from SunOS 3.0, 4.0, 4.1.4, and 5.0 (aka Solaris 2.0), and none of them had a "HISTORY" section, except that the SunOS 4 man page for sail(6) has a "THE HISTORY OF SAIL" section. I have been adding HISTORY sections to the Solaris 11.4 man pages over time, but don't remember seeing any existing sections in our man pages that predate this effort. (And I'm only documenting the history in Solaris, leaving it to others to document history in other OS'es.) -- -Alan Coopersmith- alan.coopersmith at oracle.com Oracle Solaris Engineering - https://blogs.oracle.com/solaris From dave at horsfall.org Thu Sep 21 08:31:32 2023 From: dave at horsfall.org (Dave Horsfall) Date: Thu, 21 Sep 2023 08:31:32 +1000 (EST) Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <20230920195635.GG28844@mcvoy.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: On Wed, 20 Sep 2023, Larry McVoy wrote: [...] > I like examples in man pages, I think it jump starts you into using the > program more easily but maybe that's just me. No, it's not just you :-) -- Dave From marc.donner at gmail.com Thu Sep 21 10:26:16 2023 From: marc.donner at gmail.com (Marc Donner) Date: Wed, 20 Sep 2023 20:26:16 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <20230920195635.GG28844@mcvoy.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: Maybe it’s just you, but it’s just me too. On Wed, Sep 20, 2023 at 3:57 PM Larry McVoy wrote: > On Tue, Sep 19, 2023 at 07:25:13PM -0700, Adam Thornton wrote: > > Yeah, I'm less angry at GNU now--I didn't search as hard, but when I > found > > out 4.3BSD didn't have HISTORY (and neither does 2.11BSD, which is still > > actively-ish maintained) then I figured it wasn't something classical > that > > GNU dropped, just never imported. I feel like it existed on SunOS and > > Solaris but I might be wrong about that? Was it really FreeBSD that > > introduced it? > > I don't think SunOS had it. The SunOS man pages were pretty terse, the > powers that be wanted just the facts. I remember get beat up for putting > examples in some of the man pages I wrote, they got taken out "because > examples are for user guides, man pages are not user guides, they are a > reference guide" or something like that. > > I like examples in man pages, I think it jump starts you into using the > program more easily but maybe that's just me. > -- > --- > Larry McVoy Retired to fishing > http://www.mcvoy.com/lm/boat > -------------- next part -------------- An HTML attachment was scrubbed... URL: From athornton at gmail.com Thu Sep 21 13:06:27 2023 From: athornton at gmail.com (Adam Thornton) Date: Wed, 20 Sep 2023 20:06:27 -0700 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: That's it, isn't it? Man pages are great if you a) already know the name of the program you want (c'mon, apropos/man -k have never really worked that well), and b) have at least a vague idea of how to use it. If you want to look up what a flag does, or whether there's a flag that does what you need, they can't be beat. But they aren't great for either exploration or discoverability. Now, this is less of a problem when a command takes its input on stdin, writes its output to stdout, any errors to stderr, and it has, like, five or fewer different options. For the use case of programs-which-are-really-filter-stages designed to be connected via pipes, and when there aren't all that many of them, it works fine. Once something else is on the table, yeah, the man page is at best incomplete and at worst basically useless. Say what you will about VMS, its HELP functionality makes it much easier to go from "I want to do X" to "And here's a sequence of commands that will do X" when you don't already have a good mental model of what's on the system. Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.branden.robinson at gmail.com Thu Sep 21 13:30:02 2023 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Wed, 20 Sep 2023 22:30:02 -0500 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: <20230921033002.xtbbvpnjszv3h7sl@illithid> At 2023-09-20T20:06:27-0700, Adam Thornton wrote: > That's it, isn't it? Man pages are great if you a) already know the > name of the program you want (c'mon, apropos/man -k have never really > worked that well), I think that's less a technology problem than a human problem. People need to write the summary descriptions of a man page--that would be .SH Name this part \- right here in a way that helps people find it. Sure, you could use AI to scan the full texts of pages to try to infer keywords, but I don't see that putting natural intelligence out of business quite yet. Man page authors labor under all kinds of crazy delusions about what can go into the "Name" section. Some people seem to think there's a length limit. Some seem to think that the summary description has to fit on one _input line_, as if filling is disabled when it's read. > But they aren't great for either exploration or discoverability. I got positive feedback on the groff development list regarding precisely these points when (1) revising its ~60 man pages to give them more helpful summary descriptions and (2) compiling them into a navigable PDF. https://www.gnu.org/software/groff/manual/groff-man-pages.pdf > Say what you will about VMS, its HELP functionality makes it much > easier to go from "I want to do X" to "And here's a sequence of > commands that will do X" when you don't already have a good mental > model of what's on the system. My experience with VMS is far too limited (and 30 years out of date) to offer any comparison here, but I do agree that man pages are frequently unhelpful when it comes to synthesis of the components they document. Taking the "Examples" section more seriously might help with this. At the same time I get that it's really tempting to punt. "If it's not documenting this discrete element of the system," people want to say, "it doesn't belong in the man page". I sympathize with that perspective only insofar as the people who say this then proceed to offer some alternative form of documentation that does. If they don't, then I suspect this professed piety about modular design to be a mask for unwillingness to do work that needs doing. Regards, Branden -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From phil at ultimate.com Thu Sep 21 13:48:19 2023 From: phil at ultimate.com (Phil Budne) Date: Wed, 20 Sep 2023 23:48:19 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: <202309210348.38L3mJDD097031@ultimate.com> Adam Thornton: > Say what you will about VMS, its HELP functionality makes it much easier to > go from "I want to do X" to "And here's a sequence of commands that will do > X" when you don't already have a good mental model of what's on the system. I'm reminded of two fourty-odd year old diatribes(*) against Unix documentation, yet somehow the Unix model has persisted, while other systems have long since been taken out with yesterdays ashbin of history. My beef, coming from TOPS-20 with VMS "HELP" was that (in the days on having a single dumb terminal on your desk) was if you couldn't remember a switch midway through typing a command you had to erase it, wade through multiple levels of HELP screens, cancel out of that, and then type the switch, until you realized you needed another switch. On TOPS-20 you just typed ? Yes, I suppose Unix-like systems could have section of pages with overviews of functional areas (tho to be fair, v6/v7 era Unix didn't have any facilities complex enough to warrant such things). And it depends on which Unix-like system: "man 4 inet" on FreeBSD is a more useful intro to TCP/IP socket programming than anything I can find on my Ubuntu 22.04 system. However, the FreeBSD man page (unhelpfully) references two 4.3 BSD Interprocess Communication Tutorials with no obvious information on how to find them. (*) For diatribe #1 search for: "Last night I dreamed that the Real World had adopted the Unix Philosophy". For #2 search for "it's literally a five-foot shelf of documentation" From g.branden.robinson at gmail.com Thu Sep 21 13:49:10 2023 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Wed, 20 Sep 2023 22:49:10 -0500 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <20230919233925.GB28844@mcvoy.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> Message-ID: <20230921034910.te2ryw2fu2ypx2kv@illithid> Hi Larry, At 2023-09-19T16:39:25-0700, Larry McVoy wrote: > One of the projects I thought I'd do in my retirement, but haven't > done, was to provide man page / paper as in "a paper", not tree paper, > versions of all the GNU info stuff. I could not be less thrilled with > info, yeah there are ways to deal, but it just isn't as good (to me) > as how Unix did docs. It's like they want to force emacs on us to > read docs. > > I'd start with groff. > > So I'm a little off topic but if people wanted to work on that, I'd be > up for that project. It's not as big as what you are saying but it's > pretty big, I think we just start with something, see if we can get > debian/ubuntu to pick it up, lather, rinse repeat. You have raised this issue before (multiple times) so I will remind you that I'm working on this. $ COLUMNS=72 git diff --stat 1.22.4 1.23.0 man/{roff,groff{,_font,_out}}.*.man man/groff.7.man | 9853 ++++++++++++++++++++++++++++-------------- man/groff_font.5.man | 1269 ++++-- man/groff_out.5.man | 1035 +++-- man/roff.7.man | 2973 +++++++++---- 4 files changed, 9991 insertions(+), 5139 deletions(-) Check out the compiled groff man pages PDF. I'm open to your critiques but I'd prefer you made then when I could be confident that had recently familiarized yourself with the problem. https://www.gnu.org/software/groff/manual/groff-man-pages.pdf > The one drawback I see is people might want to provide info and man > docs. My personal preference is that the info stuff goes away but I > have learned I don't get what I want. So there may be a period of > time where both need to be maintained. I have no particular love for Texinfo, but it really is trying to do a somewhat different thing than a man page does. As Trent Fisher and Werner Lemberg left it, groff's Texinfo manual felt (to me) influenced by the O'Reilly Nutshell book approach. And that's not a bad thing. It converses with and guides the reader in a way that classically laconic Unix man pages don't. Since Bertrand released groff 1.23.0 in July, I've further whacked on the early chapters of the groff Texinfo manual (among other things). You can find a bleeding edge version here. https://www.dropbox.com/sh/17ftu3z31couf07/AAC_9kq0ZA-Ra2ZhmZFWlLuva?dl=0 But if someone simply refuses to read the Texinfo manual, I've tried to make roff(7) an introduction to the formatting language and further developed groff(1) as a general introduction from the command-line perspective. I think we're deep into the "maintain both" period: $ grep -n '@c.*parallel' doc/groff.texi 584:@c BEGIN Keep parallel with groff(1), section "Description" (after the 602:@c END Keep parallel with groff(1), section "Description" (after the 605:@c BEGIN Keep parallel with groff(1), section "Usage" (introduction). 616:@c END Keep parallel with groff(1), section "Usage" (introduction). 1591:@c BEGIN Keep parallel with groff(1), subsection "Paper format". 1626:@c END Keep parallel with groff(1), subsection "Paper format". 1631:@c BEGIN Keep parallel with groff(1), section "Examples". 1696:@c END Keep parallel with groff(1), section "Examples". 5036:@c BEGIN Keep roughly parallel with roff(7) section "Concepts". 5342:@c END Keep roughly parallel with roff(7) section "Concepts". 5561:@c TODO: Consider parallelizing with groff_tmac(5) "Description". 5934:@c BEGIN Keep (roughly) parallel with section "Measurements" of 6052:@c END Keep (roughly) parallel with section "Measurements" of groff(7). ... 16963:@c BEGIN Keep parallel with section "Warnings" of troff(1). 17138:@c END Keep parallel with section "Warnings" of troff(1). 17568:@c BEGIN TODO: Make parallel with groff_out(5). 18448:@c END TODO: Make parallel with groff_out(5). 18453:@c BEGIN Keep parallel with groff_font(5). 18940:@c END Keep parallel with groff_font(5). Yes, these are screeching examples of DRY violation. But such violations are warranted if doing so means that users have a fighting chance to obtain competence with the system being documented. Regards, Branden -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From wobblygong at gmail.com Thu Sep 21 14:09:59 2023 From: wobblygong at gmail.com (Wesley Parish) Date: Thu, 21 Sep 2023 16:09:59 +1200 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: On 21/09/23 10:31, Dave Horsfall wrote: > On Wed, 20 Sep 2023, Larry McVoy wrote: > > [...] > >> I like examples in man pages, I think it jump starts you into using the >> program more easily but maybe that's just me. > No, it's not just you :-) > > -- Dave Seconded. IDK when I was learning Linux, how many times I would've been unable to follow the reference in the Linux man pages without a reference example or more to show me how I could do it myself. Wesley Parish From jpl.jpl at gmail.com Thu Sep 21 23:08:19 2023 From: jpl.jpl at gmail.com (John P. Linderman) Date: Thu, 21 Sep 2023 09:08:19 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: One of the things I liked about Mark Rochkind's original "Advanced UNIX Programming" book was that he not only provided examples of what to do, but also examples of what NOT to do, like failing to close the write end of a pipe in a child that expects to read input. I'm not sure manual pages are the right place for this sort of thing, but it certainly is valuable. On Thu, Sep 21, 2023 at 12:10 AM Wesley Parish wrote: > On 21/09/23 10:31, Dave Horsfall wrote: > > On Wed, 20 Sep 2023, Larry McVoy wrote: > > > > [...] > > > >> I like examples in man pages, I think it jump starts you into using the > >> program more easily but maybe that's just me. > > No, it's not just you :-) > > > > -- Dave > > Seconded. IDK when I was learning Linux, how many times I would've been > unable to follow the reference in the Linux man pages without a > reference example or more to show me how I could do it myself. > > Wesley Parish > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tytso at mit.edu Thu Sep 21 23:37:00 2023 From: tytso at mit.edu (Theodore Ts'o) Date: Thu, 21 Sep 2023 09:37:00 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <20230920195635.GG28844@mcvoy.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> <20230919233925.GB28844@mcvoy.com> <20230920195635.GG28844@mcvoy.com> Message-ID: On Wed, Sep 20, 2023 at 12:56:35PM -0700, Larry McVoy wrote: > On Tue, Sep 19, 2023 at 07:25:13PM -0700, Adam Thornton wrote: > > Yeah, I'm less angry at GNU now--I didn't search as hard, but when I found > > out 4.3BSD didn't have HISTORY (and neither does 2.11BSD, which is still > > actively-ish maintained) then I figured it wasn't something classical that > > GNU dropped, just never imported. I feel like it existed on SunOS and > > Solaris but I might be wrong about that? Was it really FreeBSD that > > introduced it? > > I don't think SunOS had it. The SunOS man pages were pretty terse, the > powers that be wanted just the facts. I remember get beat up for putting > examples in some of the man pages I wrote, they got taken out "because > examples are for user guides, man pages are not user guides, they are a > reference guide" or something like that. These days, I find the much more useful sections are VERSIONS and STANDARDS, e.g.: VERSIONS openat() was added in Linux 2.6.16; library support was added in glibc 2.4. STANDARDS open(), creat() SVr4, 4.3BSD, POSIX.1‐2001, POSIX.1‐2008. openat(): POSIX.1‐2008. openat2(2) is Linux‐specific. The O_DIRECT, O_NOATIME, O_PATH, and O_TMPFILE flags are Linux‐specific. One must define _GNU_SOURCE to obtain their definitions. The O_CLOEXEC, O_DIRECTORY, and O_NOFOLLOW flags are not specified in POSIX.1‐2001, but are specified in POSIX.1‐2008. Since glibc 2.12, one can obtain their definitions by defining either _POSIX_C_SOURCE with a value greater than or equal to 200809L or _XOPEN_SOURCE with a value greater than or equal to 700. In glibc 2.11 and earlier, one obtains the definitions by defining _GNU_SOURCE. As noted in feature_test_macros(7), feature test macros such as _POSIX_C_SOURCE, _XOPEN_SOURCE, and _GNU_SOURCE must be defined before including any header files. - Ted From kennethgoodwin56 at gmail.com Fri Sep 22 01:53:11 2023 From: kennethgoodwin56 at gmail.com (Kenneth Goodwin) Date: Thu, 21 Sep 2023 11:53:11 -0400 Subject: [TUHS] Project Idea: The UNIX Programmer's Manual: Heritage Edition In-Reply-To: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> References: <4CmySC-mFud1dlrqfAq1itmNKoTWVi8cTuAqCvtUengKvv5CWEoYCFf6-I18dwf5BVSZWAxC-B6BP6Y1e0Gi_mlga344b5cxu5TlUCLXHeg=@protonmail.com> Message-ID: On the copyright issue My humble opinion is you would have less trouble with getting authorization if you do the following 1. Maintain all the original authorship/copyright information for the documents. 2. Charge no more than cost plus shipping so it is nonprofit 3. Optionally add an uptick charge to the costs for charitable contributions and send those donations to a charity of your/group choice. I would recommend picking one or more of the computer museums to use as the charity. The donations therefore going to support the preservation of the history of computing worldwide. This should be something of collective interest that most people and authors/copyright holders could support and endorse. On Tue, Sep 19, 2023, 4:32 PM segaloco via TUHS wrote: > I haven't known when or how to bring up this project idea, but figure I > might as well start putting feelers out since my Dragon Quest project is > starting to slow down and I might focus back on UNIX manual stuff. > > So something painfully missing from my and I'm sure plenty of other folks' > libraries is a nice, modern paper UNIX manual that takes the past few > decades into consideration. The GNU project, BSDs, etc. ship manpages of > course, and there's the POSIX manpages, but I'm a sucker for a good print > manual. Something I'm thinking of producing as a "deliverable" of sorts > from my documentation research is a new-age UNIX manual, derived as closely > as possible from the formal UNIX documentation lineages (so Research, SysV, > and BSD pages), but: > > 1. Including subsequent POSIX requirements > 2. Including an informational section in each page with a little > history and some notes about current implementations, if applicable. This > would include notes about "dead on the vine" stuff like things plucked from > the CB-UNIX, MERT/PG, and PWB lines. The history part could even be a > separate book, that way the manual itself could stay tight and focused. > This would also be a good place for luminaries to provide reflections on > their involvement in given pieces. > > One of the main questions that I have in mind is what the legal landscape > of producing such a thing would entail. At the very least, to actually > call it a UNIX Programmer's Manual, it would probably need to pass some > sort of compliance with the materials The Open Group publishes. That said, > the ownership of the IP as opposed to the trademarks is a little less > certain, so I would be a bit curious who all would be involved in > specifically getting copyright approval to publish anything that happened > the commercial line after the early 80s, so like new text produced after > 1982. I presume anything covered by the Caldera license at least could be > published at-cost, but not for a profit (which I'm not looking for anyway.) > > Additionally, if possible, I'd love to run down some authorship > information and make sure folks who wrote stuff up over time are properly > credited, if not on each page ala OWNER at least in a Acknowledgements > section in the front. > > As far as production, I personally would want to do a run with a couple of > different cover styles, comb bound, maybe one echoing the original Bell > Laboratories UNIX User's Manual-style cover complete with Bell logo, > another using the original USENIX Beastie cover, etc. but that also then > calls into question more copyrights to coordinate, especially with the way > the Bell logo is currently owned, that could get complicated. > > Anywho, anyone know of any such efforts like this? If I actually got such > a project going in earnest, would folks find themselves interested in such > a publication? In any case I do intend to start on a typesetter sources > version of this project sometime in the next year or so, but ideally I > would want it to blossom into something that could result in some physical > media. This idea isn't even half-baked yet by the way, so just know I > don't have a roadmap in place, it's just something I see being a cool > potential project over the coming years. > > - Matt G. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From douglas.mcilroy at dartmouth.edu Fri Sep 22 11:08:13 2023 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Thu, 21 Sep 2023 21:08:13 -0400 Subject: [TUHS] UNIX Influence on Teletype and Vice Versa Message-ID: I omitted one crucial fact from my post about Joe Ossanna's influence on the TTY 37. That happened not in connection with Unix, but with Multics. When Unix came on the scene, model 37 was already in production. Doug From lm at mcvoy.com Fri Sep 22 11:14:24 2023 From: lm at mcvoy.com (Larry McVoy) Date: Thu, 21 Sep 2023 18:14:24 -0700 Subject: [TUHS] UNIX Influence on Teletype and Vice Versa In-Reply-To: References: Message-ID: <20230922011424.GO28844@mcvoy.com> This is perhaps silly, but I really wanted to meet Joe Ossanna because of troff. It was not meant to be. My dad told me you aren't dead until they stop talking about you. Good to see that Joe is still with us. On Thu, Sep 21, 2023 at 09:08:13PM -0400, Douglas McIlroy wrote: > I omitted one crucial fact from my post about Joe Ossanna's influence > on the TTY 37. That happened not in connection with Unix, but with > Multics. When Unix came on the scene, model 37 was already in > production. > > Doug -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From dave at horsfall.org Fri Sep 22 14:57:32 2023 From: dave at horsfall.org (Dave Horsfall) Date: Fri, 22 Sep 2023 14:57:32 +1000 (EST) Subject: [TUHS] UNIX Influence on Teletype and Vice Versa In-Reply-To: <20230922011424.GO28844@mcvoy.com> References: <20230922011424.GO28844@mcvoy.com> Message-ID: On Thu, 21 Sep 2023, Larry McVoy wrote: > This is perhaps silly, but I really wanted to meet Joe Ossanna because > of troff. It was not meant to be. My dad told me you aren't dead until > they stop talking about you. Good to see that Joe is still with us. I still remember being blown away when I encountered ROFF on Edition 5; like, I didn't know that you could use a computer for that sort of thing... (Yeah, I quickly discovered that Unix was first and foremost a typesetting system after I read the Bell Labs paper.) -- Dave From tuhs at tuhs.org Tue Sep 26 11:24:51 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Tue, 26 Sep 2023 01:24:51 +0000 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? Message-ID: Hello, my studies lately bring me to the question: Are there any extant examples of telephone switching software, built on UNIX, from the various parts of the Bell System prior to the introduction of the 5ESS and 3B20D? My focus veers earlier as some 5ESS/3B20D/DMERT technology is still in active use, that sleeping dragon can lie. What's gotten me curious is reading about 1ESS in a BSTJ volume I picked up, noting the particulars on how previous concerns of manual and electro-mechanical systems were abstracted into software. Even without surviving examples, were previous systems such as the 1ESS central control ever ported to or considered for porting to UNIX, or was the hardware interface to the telco lines too specific to consider a future swap-out with, say, a PDP11 running arbitrary software? Columbus's SCCS (switching, not source code) also comes to mind, although all I know that survives of that is the CB-UNIX 2.3 manual descriptions of bits and pieces. By the way, it's funny, I have UNIX to thank for my current experiments with telephones and other signalling stuff, what with making me study the Bell System more generally. It's starting to come full circle in that I want to take a crack at reading dialing, at least pulse, into some sort of software abstraction on a SBC that can, among other things, provide a switching service on top of a UNIX-like kernel. I don't know what I'd do with such a thing other than assign work conference call rooms their own phone numbers to dial with a telephone on a serial line...but if I can even get that far I'd call it a success. One less dependency on the mobile... - Matt G. From jon at fourwinds.com Tue Sep 26 11:37:54 2023 From: jon at fourwinds.com (Jon Steinhart) Date: Mon, 25 Sep 2023 18:37:54 -0700 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: References: Message-ID: <202309260137.38Q1btZU325043@darkstar.fourwinds.com> segaloco via TUHS writes: > Hello, my studies lately bring me to the question: Are there any extant > examples of telephone switching software, built on UNIX, from the various > parts of the Bell System prior to the introduction of the 5ESS and 3B20D? > My focus veers earlier as some 5ESS/3B20D/DMERT technology is still in > active use, that sleeping dragon can lie. > > What's gotten me curious is reading about 1ESS in a BSTJ volume I > picked up, noting the particulars on how previous concerns of manual and > electro-mechanical systems were abstracted into software. Even without > surviving examples, were previous systems such as the 1ESS central > control ever ported to or considered for porting to UNIX, or was the > hardware interface to the telco lines too specific to consider a future > swap-out with, say, a PDP11 running arbitrary software? Columbus's SCCS > (switching, not source code) also comes to mind, although all I know that > survives of that is the CB-UNIX 2.3 manual descriptions of bits and pieces. > > By the way, it's funny, I have UNIX to thank for my current experiments > with telephones and other signalling stuff, what with making me study the > Bell System more generally. It's starting to come full circle in that I > want to take a crack at reading dialing, at least pulse, into some sort > of software abstraction on a SBC that can, among other things, provide a > switching service on top of a UNIX-like kernel. I don't know what I'd do > with such a thing other than assign work conference call rooms their own > phone numbers to dial with a telephone on a serial line...but if I can even > get that far I'd call it a success. One less dependency on the mobile... > > - Matt G. Heinz might know something about this. If I remember correctly, one of the projects in his group was SS1, an all-digital exchange. I have some vague memory of him and Carl poring over some gigantic switch statement looking for a bug - the long distance code wasn't sending the ST pulse and as a result all of the key pulse senders at the Berkeley Heights telephone exchange were taken off line and needed a technician to go in and manually reset them. They were not amused. Fortunately, they and BTL were both children of Ma Bell. If my memory serves me correctly, the system had a pair of PDP-11/10s that ran Hal Alles's digital filter code, a PDP11/70 behind the whole thing, Harry Breece's active replacement circuitry for the hybrid transformers, and some huge insanely fast wire-wrapped boards designed by John Sheets that did TDM switching. Jon From jon at fourwinds.com Tue Sep 26 11:39:12 2023 From: jon at fourwinds.com (Jon Steinhart) Date: Mon, 25 Sep 2023 18:39:12 -0700 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: References: Message-ID: <202309260139.38Q1dCCg325287@darkstar.fourwinds.com> Oh yeah, and I think that this is why Heinz wrote MERT, but he should know more than me about it. From andrew at humeweb.com Tue Sep 26 11:50:52 2023 From: andrew at humeweb.com (Andrew Hume) Date: Mon, 25 Sep 2023 18:50:52 -0700 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: References: Message-ID: <92D94A5E-AE0F-446D-8C26-C8EF1E672230@humeweb.com> i’m sure there are standing references to all this. but as i recall, the 5ESS was a more or less local switching system. the guts of the long distance network were embedded in the (roughly) 140 1ESS’s. they ran a completely separate (and substantially older) code that supported things like SS7 (signaling code that was NOT embedded in the voice channel). the 1ESS was a complex piece of equipment. my interaction with this was tangential. throughout the 1980-90s, i had a tight grip on how accounting messages were handled within AT&T and wrote some C code that handled a a transition from one level of the standard software to another level. and that code ran inside the 1ESS. but this was a long time ago. > On Sep 25, 2023, at 6:24 PM, segaloco via TUHS wrote: > > Hello, my studies lately bring me to the question: Are there any extant examples of telephone switching software, built on UNIX, from the various parts of the Bell System prior to the introduction of the 5ESS and 3B20D? My focus veers earlier as some 5ESS/3B20D/DMERT technology is still in active use, that sleeping dragon can lie. > > What's gotten me curious is reading about 1ESS in a BSTJ volume I picked up, noting the particulars on how previous concerns of manual and electro-mechanical systems were abstracted into software. Even without surviving examples, were previous systems such as the 1ESS central control ever ported to or considered for porting to UNIX, or was the hardware interface to the telco lines too specific to consider a future swap-out with, say, a PDP11 running arbitrary software? Columbus's SCCS (switching, not source code) also comes to mind, although all I know that survives of that is the CB-UNIX 2.3 manual descriptions of bits and pieces. > > By the way, it's funny, I have UNIX to thank for my current experiments with telephones and other signalling stuff, what with making me study the Bell System more generally. It's starting to come full circle in that I want to take a crack at reading dialing, at least pulse, into some sort of software abstraction on a SBC that can, among other things, provide a switching service on top of a UNIX-like kernel. I don't know what I'd do with such a thing other than assign work conference call rooms their own phone numbers to dial with a telephone on a serial line...but if I can even get that far I'd call it a success. One less dependency on the mobile... > > - Matt G. From sjenkin at canb.auug.org.au Tue Sep 26 12:13:29 2023 From: sjenkin at canb.auug.org.au (steve jenkin) Date: Tue, 26 Sep 2023 12:13:29 +1000 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: <92D94A5E-AE0F-446D-8C26-C8EF1E672230@humeweb.com> References: <92D94A5E-AE0F-446D-8C26-C8EF1E672230@humeweb.com> Message-ID: <6985E7CD-6C8A-42E5-A58B-2533F635677F@canb.auug.org.au> For those playing along at home, “Gecko” was written up. Virtual Data Warehousing, Data Publishing and Call Detail and a conference paper of the same name from Databases in Telecommunications: International Workshop, Co-located with VLDB-99, Edinburgh, Scotland, UK, September 6th, 1999. Proceedings > On 26 Sep 2023, at 11:50, Andrew Hume wrote: > > i’m sure there are standing references to all this. but as i recall, > the 5ESS was a more or less local switching system. > > the guts of the long distance network were embedded in the (roughly) 140 1ESS’s. they ran > a completely separate (and substantially older) code that supported things like SS7 (signaling code > that was NOT embedded in the voice channel). the 1ESS was a complex piece of equipment. > > my interaction with this was tangential. throughout the 1980-90s, i had a tight grip on how > accounting messages were handled within AT&T and wrote some C code that handled a > a transition from one level of the standard software to another level. and that code ran inside > the 1ESS. > > but this was a long time ago. -- Steve Jenkin, IT Systems and Design 0412 786 915 (+61 412 786 915) PO Box 38, Kippax ACT 2615, AUSTRALIA mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin From kevin.bowling at kev009.com Tue Sep 26 12:20:54 2023 From: kevin.bowling at kev009.com (Kevin Bowling) Date: Mon, 25 Sep 2023 19:20:54 -0700 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: References: Message-ID: On Mon, Sep 25, 2023 at 6:25 PM segaloco via TUHS wrote: > Hello, my studies lately bring me to the question: Are there any extant > examples of telephone switching software, built on UNIX, from the various > parts of the Bell System prior to the introduction of the 5ESS and 3B20D? > My focus veers earlier as some 5ESS/3B20D/DMERT technology is still in > active use, that sleeping dragon can lie. > Your best bet may be to contact Sarah Autumn at the Connections Museum, they have a 1ESS and 3ESS. http://www.telcomhistory.org/connections-museum-seattle-exhibits/electronic-switching/ I don't remember if they have the 1A variant but they should have the BSPs for all of this which would give you a lot of what you are after. > What's gotten me curious is reading about 1ESS in a BSTJ volume I picked > up, noting the particulars on how previous concerns of manual and > electro-mechanical systems were abstracted into software. Even without > surviving examples, were previous systems such as the 1ESS central control > ever ported to or considered for porting to UNIX, or was the hardware > interface to the telco lines too specific to consider a future swap-out > with, say, a PDP11 running arbitrary software? Columbus's SCCS (switching, > not source code) also comes to mind, although all I know that survives of > that is the CB-UNIX 2.3 manual descriptions of bits and pieces. > > By the way, it's funny, I have UNIX to thank for my current experiments > with telephones and other signalling stuff, what with making me study the > Bell System more generally. It's starting to come full circle in that I > want to take a crack at reading dialing, at least pulse, into some sort of > software abstraction on a SBC that can, among other things, provide a > switching service on top of a UNIX-like kernel. I don't know what I'd do > with such a thing other than assign work conference call rooms their own > phone numbers to dial with a telephone on a serial line...but if I can even > get that far I'd call it a success. One less dependency on the mobile... > > - Matt G. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad at anduin.eldar.org Tue Sep 26 21:28:19 2023 From: brad at anduin.eldar.org (Brad Spencer) Date: Tue, 26 Sep 2023 07:28:19 -0400 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: <92D94A5E-AE0F-446D-8C26-C8EF1E672230@humeweb.com> (message from Andrew Hume on Mon, 25 Sep 2023 18:50:52 -0700) Message-ID: Andrew Hume writes: > i’m sure there are standing references to all this. but as i recall, > the 5ESS was a more or less local switching system. > > the guts of the long distance network were embedded in the (roughly) 140 1ESS’s. they ran > a completely separate (and substantially older) code that supported things like SS7 (signaling code > that was NOT embedded in the voice channel). the 1ESS was a complex piece of equipment. > > my interaction with this was tangential. throughout the 1980-90s, i had a tight grip on how > accounting messages were handled within AT&T and wrote some C code that handled a > a transition from one level of the standard software to another level. and that code ran inside > the 1ESS. > > but this was a long time ago. > >> On Sep 25, 2023, at 6:24 PM, segaloco via TUHS wrote: >> >> Hello, my studies lately bring me to the question: Are there any extant examples of telephone switching software, built on UNIX, from the various parts of the Bell System prior to the introduction of the 5ESS and 3B20D? My focus veers earlier as some 5ESS/3B20D/DMERT technology is still in active use, that sleeping dragon can lie. >> >> What's gotten me curious is reading about 1ESS in a BSTJ volume I picked up, noting the particulars on how previous concerns of manual and electro-mechanical systems were abstracted into software. Even without surviving examples, were previous systems such as the 1ESS central control ever ported to or considered for porting to UNIX, or was the hardware interface to the telco lines too specific to consider a future swap-out with, say, a PDP11 running arbitrary software? Columbus's SCCS (switching, not source code) also comes to mind, although all I know that survives of that is the CB-UNIX 2.3 manual descriptions of bits and pieces. >> >> By the way, it's funny, I have UNIX to thank for my current experiments with telephones and other signalling stuff, what with making me study the Bell System more generally. It's starting to come full circle in that I want to take a crack at reading dialing, at least pulse, into some sort of software abstraction on a SBC that can, among other things, provide a switching service on top of a UNIX-like kernel. I don't know what I'd do with such a thing other than assign work conference call rooms their own phone numbers to dial with a telephone on a serial line...but if I can even get that far I'd call it a success. One less dependency on the mobile... >> >> - Matt G. So... I am speaking from a single point of view about stuff I worked on a very long time ago.... On the operations support system I worked on at AT&T/Lucent... we supported all manor of switches, all of those made by AT&T/Lucent and a whole lot made by other companies. This is what I recall... I believe that the 1A had left the building and no longer existed in any installed base in the US when I started, although we supported it still in our software because of the pain of ripping it out. The 1E did exist in some numbers (I may have these backwards, BTW... it might have been the 1A that existed and the 1E had crossed the rainbow bridge). I do recall that either the 1A or 1E had a lot of features, but it may have been more of a case of it just being around a long time and was infected with warts. The work horses, however, were the 4ESS and the 5ESS. The 4E did the long haul work and was often configured in a tandem set up (that was the term used, Tandem, which meant something specific at the time). The 4E was also a very large monster of a device... taking up pretty much an entire floor of a building. The software in it was very old with lots of add on sub-devices to provide new ability (our product used some of those sub-devices). Those sub-devices might have run their own operating systems and could have been computers all by themselves... that is, there were multiple glued together computers to get the 4E effect. The 5E was a lot cleaner and much smaller. It also had some sub-device stuff going on but nothing like what the 4E had. I do seem to recall that the 5E runs some version of Unix, at least on part of the device, but I don't remember if the main switching engine did, however (I suspect not, BTW). I am almost 100% sure that the 4E did not run Unix on the main switching engine, but it would not surprise me that it found its way into it somewhere or other (probably in multiple places given how large the device was). The 5E and 4E may have started out life with particular roles, but this wasn't very strict. That is it was completely ok for a 5E to do long haul work if so configured and it was very possible for a 4E to terminate a call. There was a great deal of flexibility present. AT&T long wire was a bit of a different place with respect to the 4E.. I seem to remember that at the time I was there they had something like 800 4E configured to do long distance service in the US. At least the traffic management part was managed by a set of Amdauls (using Unix as the operating system I believe) running an operation system product that was very much related to the one I worked on... simular internals for example and in my early days we all sat in the same group. The product I worked on also supported the SS7 switches (I say "switches" here, but that probably isn't exactly what they were. I also don't remember their actual names... STM, maybe??). I worked with the SS7 development group on a new operations system interface and during that conversation they mentioned that the SS7 switch ran a more or less unmodified Unix Solaris (one would assume on a Sparc). They also mentioned that they had performed kernel modifications in the past, but the cost of maintaining that sort of thing was very high, so they had moved away from doing that towards a more "push it to userland" way of thinking. -- Brad Spencer - brad at anduin.eldar.org - KC8VKS - http://anduin.eldar.org From douglas.mcilroy at dartmouth.edu Tue Sep 26 23:38:07 2023 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Tue, 26 Sep 2023 09:38:07 -0400 Subject: [TUHS] custom NS and NE man(7) macros Message-ID: > You didn't say, but I reckon this is a survey of man(7) macros that > might be considered extensions? My presentation was too cute for my own good. I pointed out the consistency of xS/xE for various x. I apologized for EX/EE, which varied from that form (as UR/UE did more recently), and I questioned OQ/CQ, which utterly breaks it. The intended point was that one should have a strong rationale for deviating from established custom, and thereby fostering mental overload. Doug From g.branden.robinson at gmail.com Tue Sep 26 23:58:31 2023 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Tue, 26 Sep 2023 08:58:31 -0500 Subject: [TUHS] custom NS and NE man(7) macros In-Reply-To: References: Message-ID: <20230926135831.35ljg4emxie4adl4@illithid> Hi Doug, At 2023-09-26T09:38:07-0400, Douglas McIlroy wrote: > > You didn't say, but I reckon this is a survey of man(7) macros that > > might be considered extensions? > > My presentation was too cute for my own good. I pointed out the > consistency of xS/xE for various x. Oh! > I apologized for EX/EE, which varied from that form It seems then that we are owed an apology from other quarters for `EQ`/`EN`. ;-) > (as UR/UE did more recently), There is also `MT`/`ME`, another groff 1.20 extension. > and I questioned OQ/CQ, which utterly breaks it. I proposed `QO`/`QC`, but yes, the same is true of that letter ordering. > The intended point was that one should have a strong rationale > for deviating from established custom, and thereby fostering > mental overload. Fair. I'm fine with renaming my proposed quotation macros `QS`/`QE`. Something I'm still mulling over is how one would specify to these new macros that they should not perform a break. Or maybe they should perform no break by default. That would seem to be the more common expected case. Regards, Branden -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From f4grx at f4grx.net Wed Sep 27 02:07:34 2023 From: f4grx at f4grx.net (Sebastien F4GRX) Date: Tue, 26 Sep 2023 18:07:34 +0200 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: References: Message-ID: <39a754af-1a38-da96-379f-d5886fcd0fec@f4grx.net> Hello, You beat me to it! I was about to reply that the Connections Museum of Seattle would have more info about this, or know people who do. This video of their channel shows a 3ESS software boot : https://www.youtube.com/watch?v=k865-VjWUk8 Sebastien Le 26/09/2023 à 04:20, Kevin Bowling a écrit : > > > On Mon, Sep 25, 2023 at 6:25 PM segaloco via TUHS wrote: > > Hello, my studies lately bring me to the question: Are there any > extant examples of telephone switching software, built on UNIX, > from the various parts of the Bell System prior to the > introduction of the 5ESS and 3B20D?  My focus veers earlier as > some 5ESS/3B20D/DMERT technology is still in active use, that > sleeping dragon can lie. > > > Your best bet may be to contact Sarah Autumn at the Connections > Museum, they have a 1ESS and 3ESS. > http://www.telcomhistory.org/connections-museum-seattle-exhibits/electronic-switching/ > > I don't remember if they have the 1A variant but they should have the > BSPs for all of this which would give you a lot of what you are after. > > What's gotten me curious is reading about 1ESS in a BSTJ volume I > picked up, noting the particulars on how previous concerns of > manual and electro-mechanical systems were abstracted into > software.  Even without surviving examples, were previous systems > such as the 1ESS central control ever ported to or considered for > porting to UNIX, or was the hardware interface to the telco lines > too specific to consider a future swap-out with, say, a PDP11 > running arbitrary software?  Columbus's SCCS (switching, not > source code) also comes to mind, although all I know that survives > of that is the CB-UNIX 2.3 manual descriptions of bits and pieces. > > By the way, it's funny, I have UNIX to thank for my current > experiments with telephones and other signalling stuff, what with > making me study the Bell System more generally.  It's starting to > come full circle in that I want to take a crack at reading > dialing, at least pulse, into some sort of software abstraction on > a SBC that can, among other things, provide a switching service on > top of a UNIX-like kernel.  I don't know what I'd do with such a > thing other than assign work conference call rooms their own phone > numbers to dial with a telephone on a serial line...but if I can > even get that far I'd call it a success.  One less dependency on > the mobile... > > - Matt G. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Wed Sep 27 04:45:18 2023 From: tuhs at tuhs.org (segaloco via TUHS) Date: Tue, 26 Sep 2023 18:45:18 +0000 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: <39a754af-1a38-da96-379f-d5886fcd0fec@f4grx.net> References: <39a754af-1a38-da96-379f-d5886fcd0fec@f4grx.net> Message-ID: Thanks for all the great information everyone. It sounds like I need to start considering a little trip down to Seattle to check out a few of these different computing/technology history organizations. They're a quick train ride down the coast, and my dreams of pulling together some sort of computing/technology history collective up here have been fruitless thus far, maybe I just need to recognize it as a big city thing and get more comfortable with considering little trips for this kind of stuff. - Matt G. ------- Original Message ------- On Tuesday, September 26th, 2023 at 9:07 AM, Sebastien F4GRX wrote: > Hello, > > You beat me to it! I was about to reply that the Connections Museum of Seattle would have more info about this, or know people who do. > > This video of their channel shows a 3ESS software boot : https://www.youtube.com/watch?v=k865-VjWUk8 > > Sebastien > > Le 26/09/2023 à 04:20, Kevin Bowling a écrit : > >> On Mon, Sep 25, 2023 at 6:25 PM segaloco via TUHS wrote: >> >>> Hello, my studies lately bring me to the question: Are there any extant examples of telephone switching software, built on UNIX, from the various parts of the Bell System prior to the introduction of the 5ESS and 3B20D? My focus veers earlier as some 5ESS/3B20D/DMERT technology is still in active use, that sleeping dragon can lie. >> >> Your best bet may be to contact Sarah Autumn at the Connections Museum, they have a 1ESS and 3ESS. >> http://www.telcomhistory.org/connections-museum-seattle-exhibits/electronic-switching/ >> >> I don't remember if they have the 1A variant but they should have the BSPs for all of this which would give you a lot of what you are after. >> >>> What's gotten me curious is reading about 1ESS in a BSTJ volume I picked up, noting the particulars on how previous concerns of manual and electro-mechanical systems were abstracted into software. Even without surviving examples, were previous systems such as the 1ESS central control ever ported to or considered for porting to UNIX, or was the hardware interface to the telco lines too specific to consider a future swap-out with, say, a PDP11 running arbitrary software? Columbus's SCCS (switching, not source code) also comes to mind, although all I know that survives of that is the CB-UNIX 2.3 manual descriptions of bits and pieces. >>> >>> By the way, it's funny, I have UNIX to thank for my current experiments with telephones and other signalling stuff, what with making me study the Bell System more generally. It's starting to come full circle in that I want to take a crack at reading dialing, at least pulse, into some sort of software abstraction on a SBC that can, among other things, provide a switching service on top of a UNIX-like kernel. I don't know what I'd do with such a thing other than assign work conference call rooms their own phone numbers to dial with a telephone on a serial line...but if I can even get that far I'd call it a success. One less dependency on the mobile... >>> >>> - Matt G. -------------- next part -------------- An HTML attachment was scrubbed... URL: From douglas.mcilroy at dartmouth.edu Wed Sep 27 08:05:45 2023 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Tue, 26 Sep 2023 18:05:45 -0400 Subject: [TUHS] 7th ed local man pages Message-ID: The union table of man pages in "A Research Unix Reader', CSTR 139, marks with "L" local man pages that were not distributed outside of research. Does anyone have a copy of those pages? I'm particularly interested in galloc(3), but would love to get them all, which I unwittingly trashed when I retired. Doug From ggm at algebras.org Wed Sep 27 09:32:23 2023 From: ggm at algebras.org (George Michaelson) Date: Wed, 27 Sep 2023 09:32:23 +1000 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: References: <39a754af-1a38-da96-379f-d5886fcd0fec@f4grx.net> Message-ID: It may be obvious, certainly to anyone who owns the BSTL series (I don't. I read it in the library at uni), but would I be right in thinking the motivation behind "real time UNIX" was switching? It's the closest problem to Bell labs which demands time critical outcomes, the delay tolerance in circuit establishment and end-to-end telephony signaling was there, but it was finite. And, for capacity planning purposes being told a single unit of UNIX logic controlling a telephone switch could handle X call setup and tear-down events per second meant you could do the Erlang sums over customers call volumes and phone handset counts. The british digital switch experience in the System-X series Was GEC/Plessey, and ran CORAL which was the real time system they used to do MILSPEC stuff. It was explicitly fault tolerant design, lots of 2 and triple redundant componentry. -G From heinz at osta.com Wed Sep 27 09:35:29 2023 From: heinz at osta.com (Heinz Lycklama) Date: Tue, 26 Sep 2023 16:35:29 -0700 Subject: [TUHS] 7th ed local man pages In-Reply-To: References: Message-ID: This it for galloc(3)?     https://man.cat-v.org/unix_8th/3/galloc Heinz On 9/26/2023 3:05 PM, Douglas McIlroy wrote: > The union table of man pages in "A Research Unix Reader', CSTR 139, > marks with "L" local man pages that were not distributed outside of > research. Does anyone have a copy of those pages? I'm particularly > interested in galloc(3), but would love to get them all, which I > unwittingly trashed when I retired. > > Doug From heinz at osta.com Wed Sep 27 12:46:41 2023 From: heinz at osta.com (Heinz Lycklama) Date: Tue, 26 Sep 2023 19:46:41 -0700 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: <202309260137.38Q1btZU325043@darkstar.fourwinds.com> References: <202309260137.38Q1btZU325043@darkstar.fourwinds.com> Message-ID: <1eaf63bd-5394-a692-ad42-c8dd5c1a741c@osta.com> To answer Jon's following question (2 minutes later): ------------------------------------------------------------------------ Oh yeah, and I think that this is why Heinz wrote MERT, but he should know more than me about it. ------------------------------------------------------------------------ Yes, my Dept. in MH was involved in the early days of digital switching and the need for real-time response was certainly recognized. But MERT was not developed with a specific telephony project in mind. I was mostly involved in software in support of current projects being done in the Dept. We started the MERT project at the time that DEC announced their PDP-11/45 mini-computer in the early 1970's because it supported 3 separate address spaces - system, supervisor, and user. This enabled us to run operating system environments with different user application program needs, specifically real-time under control of one supervisor and time-sharing applications in another supervisor, to start with. Hence its name - Multi-Environment Real Time (MERT). Once we had MERT up and running on the PDP-11/45 and PDP-11/70 computers, some projects in other Bell Labs locations involved in telephony projects started building their projects on the MERT system. The DMERT system was developed later on by projects at yet another Bell Labs location. Heinz On 9/25/2023 6:37 PM, Jon Steinhart wrote: > segaloco via TUHS writes: >> Hello, my studies lately bring me to the question: Are there any extant >> examples of telephone switching software, built on UNIX, from the various >> parts of the Bell System prior to the introduction of the 5ESS and 3B20D? >> My focus veers earlier as some 5ESS/3B20D/DMERT technology is still in >> active use, that sleeping dragon can lie. >> >> What's gotten me curious is reading about 1ESS in a BSTJ volume I >> picked up, noting the particulars on how previous concerns of manual and >> electro-mechanical systems were abstracted into software. Even without >> surviving examples, were previous systems such as the 1ESS central >> control ever ported to or considered for porting to UNIX, or was the >> hardware interface to the telco lines too specific to consider a future >> swap-out with, say, a PDP11 running arbitrary software? Columbus's SCCS >> (switching, not source code) also comes to mind, although all I know that >> survives of that is the CB-UNIX 2.3 manual descriptions of bits and pieces. >> >> By the way, it's funny, I have UNIX to thank for my current experiments >> with telephones and other signalling stuff, what with making me study the >> Bell System more generally. It's starting to come full circle in that I >> want to take a crack at reading dialing, at least pulse, into some sort >> of software abstraction on a SBC that can, among other things, provide a >> switching service on top of a UNIX-like kernel. I don't know what I'd do >> with such a thing other than assign work conference call rooms their own >> phone numbers to dial with a telephone on a serial line...but if I can even >> get that far I'd call it a success. One less dependency on the mobile... >> >> - Matt G. > Heinz might know something about this. If I remember correctly, one of the > projects in his group was SS1, an all-digital exchange. I have some vague > memory of him and Carl poring over some gigantic switch statement looking > for a bug - the long distance code wasn't sending the ST pulse and as a > result all of the key pulse senders at the Berkeley Heights telephone > exchange were taken off line and needed a technician to go in and manually > reset them. They were not amused. Fortunately, they and BTL were both > children of Ma Bell. > > If my memory serves me correctly, the system had a pair of PDP-11/10s that > ran Hal Alles's digital filter code, a PDP11/70 behind the whole thing, > Harry Breece's active replacement circuitry for the hybrid transformers, > and some huge insanely fast wire-wrapped boards designed by John Sheets > that did TDM switching. > > Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Wed Sep 27 16:15:23 2023 From: lars at nocrew.org (Lars Brinkhoff) Date: Wed, 27 Sep 2023 06:15:23 +0000 Subject: [TUHS] Known Specimens of Pre-5ESS UNIX Telephone Switching Software? In-Reply-To: (segaloco via TUHS's message of "Tue, 26 Sep 2023 18:45:18 +0000") References: <39a754af-1a38-da96-379f-d5886fcd0fec@f4grx.net> Message-ID: <7wedikjg5g.fsf@junk.nocrew.org> Maybe also visit the ICM Museum / SDF Museum. From jnc at mercury.lcs.mit.edu Thu Sep 28 00:30:17 2023 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Wed, 27 Sep 2023 10:30:17 -0400 (EDT) Subject: [TUHS] 7th ed local man pages Message-ID: <20230927143017.1E83518C089@mercury.lcs.mit.edu> > From: Douglas McIlroy > The union table of man pages in "A Research Unix Reader', CSTR 139, > marks with "L" local man pages that were not distributed outside of > research. Does anyone have a copy of those pages? ... would love to get > them all The CB-UNIX manual has a bunch of xL "Local" pages, e.g. the Section 3 ones here: https://www.tuhs.org/Archive/Distributions/USDL/CB_Unix/man/man3/ I know this is CB-UNIX, but perhaps they are the same? Noel From grog at lemis.com Thu Sep 28 14:08:03 2023 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Thu, 28 Sep 2023 14:08:03 +1000 Subject: [TUHS] Lions' commentary updated Message-ID: <20230928040803.GC80098@eureka.lemis.com> No, sorry, there hasn't been a new edition, just corrections. The original version contained a number of scan errors, and thanks to Conway Yee I have now applied a number of corrections. You can find the document in various forms at http://www.lemis.com/grog/Documentation/Lions/. I've scanned through the output, and it seems correct. If you find any further errors, please let me know. Also thanks to Brian Foley, who sent me similar corrections 10 years ago, but which I never got round to incorporating. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA.php -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From christopher.fujino at gmail.com Thu Sep 28 15:01:28 2023 From: christopher.fujino at gmail.com (christopher fujino) Date: Wed, 27 Sep 2023 22:01:28 -0700 Subject: [TUHS] Lions' commentary updated In-Reply-To: <20230928040803.GC80098@eureka.lemis.com> References: <20230928040803.GC80098@eureka.lemis.com> Message-ID: Hey Greg, thanks for sharing this! FYI, I get a code 403 forbidden from the link: http://www.lemis.com/grog/Documentation/Lions/book/ On Wed, Sep 27, 2023 at 9:08 PM Greg 'groggy' Lehey wrote: > No, sorry, there hasn't been a new edition, just corrections. The > original version contained a number of scan errors, and thanks to > Conway Yee I have now applied a number of corrections. You can find > the document in various forms at > http://www.lemis.com/grog/Documentation/Lions/. > > I've scanned through the output, and it seems correct. If you find > any further errors, please let me know. > > Also thanks to Brian Foley, who sent me similar corrections 10 years > ago, but which I never got round to incorporating. > > Greg > -- > Sent from my desktop computer. > Finger grog at lemis.com for PGP public key. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft mail program > reports problems, please read http://lemis.com/broken-MUA.php > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grog at lemis.com Fri Sep 29 14:26:44 2023 From: grog at lemis.com (Greg 'groggy' Lehey) Date: Fri, 29 Sep 2023 14:26:44 +1000 Subject: [TUHS] Lions' commentary updated In-Reply-To: References: <20230928040803.GC80098@eureka.lemis.com> Message-ID: <20230929042644.GA62979@eureka.lemis.com> On Wednesday, 27 September 2023 at 22:01:28 -0700, christopher fujino wrote: > On Wed, Sep 27, 2023 at 9:08 PM Greg 'groggy' Lehey wrote: > >> No, sorry, there hasn't been a new edition, just corrections. The >> original version contained a number of scan errors, and thanks to >> Conway Yee I have now applied a number of corrections. You can find >> the document in various forms at >> http://www.lemis.com/grog/Documentation/Lions/. >> >> I've scanned through the output, and it seems correct. If you find >> any further errors, please let me know. > > Hey Greg, thanks for sharing this! > > FYI, I get a code 403 forbidden from the link: > http://www.lemis.com/grog/Documentation/Lions/book/ Oh. How about that, it's a day 1 bug. My external web site doesn't display directory contents, so it needs a real page to access them. It's there now, but I'm left wondering if this is even needed. When I set it up, data was expensive, and this was an attempt to not transfer more than necessary. Nowadays it's easier just to download the tarball and access the files there. I think I'll remove it. Greg -- Sent from my desktop computer. Finger grog at lemis.com for PGP public key. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA.php -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 163 bytes Desc: not available URL: From rob at atvetsystems.com Fri Sep 29 20:01:33 2023 From: rob at atvetsystems.com (rob at atvetsystems.com) Date: Fri, 29 Sep 2023 11:01:33 +0100 Subject: [TUHS] Lions' commentary updated In-Reply-To: <20230928040803.GC80098@eureka.lemis.com> References: <20230928040803.GC80098@eureka.lemis.com> Message-ID: <7D942784-89DD-4618-B3CD-1953068C6C69@atvetsystems.com> I’ve got the 1996 reprint. I noticed while just scrolling through the PDF that on page 74 is: 5736 fp = getf (u.u_ar0[R0]; In my version (sheet 57) there is a closing bracket in the code. Regards, Rob. > On 28 Sep 2023, at 05:08, Greg 'groggy' Lehey wrote: > > No, sorry, there hasn't been a new edition, just corrections. The > original version contained a number of scan errors, and thanks to > Conway Yee I have now applied a number of corrections. You can find > the document in various forms at > http://www.lemis.com/grog/Documentation/Lions/. > > I've scanned through the output, and it seems correct. If you find > any further errors, please let me know. > > Also thanks to Brian Foley, who sent me similar corrections 10 years > ago, but which I never got round to incorporating. > > Greg > -- > Sent from my desktop computer. > Finger grog at lemis.com for PGP public key. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft mail program > reports problems, please read http://lemis.com/broken-MUA.php From will.senn at gmail.com Fri Sep 29 23:52:20 2023 From: will.senn at gmail.com (Will Senn) Date: Fri, 29 Sep 2023 08:52:20 -0500 Subject: [TUHS] Lions' commentary updated In-Reply-To: <7D942784-89DD-4618-B3CD-1953068C6C69@atvetsystems.com> References: <20230928040803.GC80098@eureka.lemis.com> <7D942784-89DD-4618-B3CD-1953068C6C69@atvetsystems.com> Message-ID: BTW, This is a version of Lions' that I generated from the actual source on the tape images we have on TUHS: https://decuser.github.io/unix/research-unix/v6/2022/06/06/updated-lions-v6-source-w-appendices-and-helps-v1.4.html I'm sure it's got its share of issues (with the helps which haven't been triple/quadruple checked), but the source is verbatim. Will On 9/29/23 05:01, rob at atvetsystems.com wrote: > I’ve got the 1996 reprint. I noticed while just scrolling through the PDF that on page 74 is: > > 5736 fp = getf (u.u_ar0[R0]; > > In my version (sheet 57) there is a closing bracket in the code. > > > Regards, Rob. > > >> On 28 Sep 2023, at 05:08, Greg 'groggy' Lehey wrote: >> >> No, sorry, there hasn't been a new edition, just corrections. The >> original version contained a number of scan errors, and thanks to >> Conway Yee I have now applied a number of corrections. You can find >> the document in various forms at >> http://www.lemis.com/grog/Documentation/Lions/. >> >> I've scanned through the output, and it seems correct. If you find >> any further errors, please let me know. >> >> Also thanks to Brian Foley, who sent me similar corrections 10 years >> ago, but which I never got round to incorporating. >> >> Greg >> -- >> Sent from my desktop computer. >> Finger grog at lemis.com for PGP public key. >> See complete headers for address and phone numbers. >> This message is digitally signed. If your Microsoft mail program >> reports problems, please read http://lemis.com/broken-MUA.php From web at loomcom.com Sat Sep 30 09:57:23 2023 From: web at loomcom.com (Seth Morabito) Date: Fri, 29 Sep 2023 16:57:23 -0700 Subject: [TUHS] SVR3 or SVR4 kernel sources for 3B2/700 Message-ID: <337d1f26-81a4-4c8e-9c30-030188ace3e2@app.fastmail.com> It's been a while since I asked, and I would be extraordinarily surprised if the situation had changed, but I thought I might try my luck one more time... The 3B2 is a dreadful computer, but nevertheless I find myself compelled to try to make the SIMH 3B2 emulation more accurate. The emulation for the 3B2/400 is probably as accurate as it's ever going to be, but the 3B2/700 has very clear and known bugs. One of the things holding back fixing those bugs is documentation in the form of source code. I have the leaked kernel source code for the 3B2/400 ("Version 2") architecture, but I have never seen any kernel source code that targets the 3B2/700 or /1000 ("Version 3") architecture. All I have are the system header files from /usr/include/sys, nothing more. If by some chance you have a /usr/src/uts tree for the 3B2/600, /700, or /1000, I would love to see it. It would refer to the system board using the code name "FALCON", probably with a lot of #ifdef's (at least the system headers do) -Seth -- Seth Morabito * Poulsbo, WA * https://loomcom.com/ From ben at huntsmans.net Sat Sep 30 15:38:15 2023 From: ben at huntsmans.net (Ben Huntsman) Date: Sat, 30 Sep 2023 05:38:15 +0000 Subject: [TUHS] SVR3 or SVR4 kernel sources for 3B2/700 In-Reply-To: <337d1f26-81a4-4c8e-9c30-030188ace3e2@app.fastmail.com> References: <337d1f26-81a4-4c8e-9c30-030188ace3e2@app.fastmail.com> Message-ID: <9DB73B8C-CCE2-4321-B875-240CCCAC0271@huntsmans.net> Speaking of the 3B2, has anyone ever seen DMERT out there or had a chance to run it on their 3B2? Sent from my iPhone > On Sep 29, 2023, at 4:58 PM, Seth Morabito wrote: > > It's been a while since I asked, and I would be extraordinarily surprised if the situation had changed, but I thought I might try my luck one more time... > > The 3B2 is a dreadful computer, but nevertheless I find myself compelled to try to make the SIMH 3B2 emulation more accurate. The emulation for the 3B2/400 is probably as accurate as it's ever going to be, but the 3B2/700 has very clear and known bugs. One of the things holding back fixing those bugs is documentation in the form of source code. I have the leaked kernel source code for the 3B2/400 ("Version 2") architecture, but I have never seen any kernel source code that targets the 3B2/700 or /1000 ("Version 3") architecture. All I have are the system header files from /usr/include/sys, nothing more. > > If by some chance you have a /usr/src/uts tree for the 3B2/600, /700, or /1000, I would love to see it. It would refer to the system board using the code name "FALCON", probably with a lot of #ifdef's (at least the system headers do) > > -Seth > -- > Seth Morabito * Poulsbo, WA * https://loomcom.com/ From marc.donner at gmail.com Sat Sep 30 16:35:06 2023 From: marc.donner at gmail.com (Marc Donner) Date: Sat, 30 Sep 2023 08:35:06 +0200 Subject: [TUHS] SVR3 or SVR4 kernel sources for 3B2/700 In-Reply-To: <337d1f26-81a4-4c8e-9c30-030188ace3e2@app.fastmail.com> References: <337d1f26-81a4-4c8e-9c30-030188ace3e2@app.fastmail.com> Message-ID: I infer from your inquiry that there is no formal statement of the ISA for the machine. Is that correct? On Sat, Sep 30, 2023 at 1:57 AM Seth Morabito wrote: > It's been a while since I asked, and I would be extraordinarily surprised > if the situation had changed, but I thought I might try my luck one more > time... > > The 3B2 is a dreadful computer, but nevertheless I find myself compelled > to try to make the SIMH 3B2 emulation more accurate. The emulation for the > 3B2/400 is probably as accurate as it's ever going to be, but the 3B2/700 > has very clear and known bugs. One of the things holding back fixing those > bugs is documentation in the form of source code. I have the leaked kernel > source code for the 3B2/400 ("Version 2") architecture, but I have never > seen any kernel source code that targets the 3B2/700 or /1000 ("Version 3") > architecture. All I have are the system header files from /usr/include/sys, > nothing more. > > If by some chance you have a /usr/src/uts tree for the 3B2/600, /700, or > /1000, I would love to see it. It would refer to the system board using the > code name "FALCON", probably with a lot of #ifdef's (at least the system > headers do) > > -Seth > -- > Seth Morabito * Poulsbo, WA * https://loomcom.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at quintile.net Sat Sep 30 17:28:26 2023 From: steve at quintile.net (Steve Simon) Date: Sat, 30 Sep 2023 08:28:26 +0100 Subject: [TUHS] lyons commentry Message-ID: hi all, maybe of interest, in the early 1990s i worked at UNSW and met several people in the computer science dept who knew John Lions. i *was* the character on the cover of the reprint, i have a photocopy of the original pamphlet, complete with hand written annotations by someone who attended Lyons course. perhaps everyone on tuhs has their own photocopy, but if anyone wants this one, give me a shout. -Steve From web at loomcom.com Sat Sep 30 23:41:40 2023 From: web at loomcom.com (Seth Morabito) Date: Sat, 30 Sep 2023 06:41:40 -0700 Subject: [TUHS] SVR3 or SVR4 kernel sources for 3B2/700 In-Reply-To: References: <337d1f26-81a4-4c8e-9c30-030188ace3e2@app.fastmail.com> Message-ID: On Fri, Sep 29, 2023, at 11:35 PM, Marc Donner wrote: > I infer from your inquiry that there is no formal statement of the ISA for the machine. Is that correct? The CPU itself is a standard off the shelf WE32200, which is thankfully very well documented, so the ISA is not really an issue. The real thorny question is how everything else is all hooked together. The interrupt controller, the IO buses, the DRAM controller, and so on, are partly documented in the technical reference manual, but only at a very high level, and with at least a few major inaccuracies. Detailed behavior like bus timeouts, error conditions, and so on, are very poorly documented. I was only able to infer how everything was connected on the 3B2/400 by reading the kernel source code to understand what the various drivers expected to see, but the 3B2/700 has a very different memory map and the kernel source code is missing. I have the system board schematics, but there an awful lot of PALs, and no PAL truth tables. -Seth -- Seth Morabito * Poulsbo, WA * https://loomcom.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: