The problem I have with Pip's dependency version handling
Python's Pip package manager hasa system where main programs and packages can specify the generalversions of dependencies that they want. When you install a programthrough pip (either directly into a virtual environment or with aconvenient tool like pipx ), pip resolves thegeneral version specifications to specific versions of the packagesand installs them too. Like many language package managers, pipfollows what I'll call a maximal version selection algorithm; itchooses the highest currently available version of dependenciesthat satisfy all constraints. Unfortunately I
Two stories of how and why simultaneous multithreading works
Simultaneous multithreading hasbeen controversial since Intel introduced hyper-threading . Some people feltthey were getting a nice bonus; others thought they were getting anet negative ( back in 2011 I sort of had this view ). I don't have any well informed viewson whether or not SMT is useful for me (or for us), but what I dohave is two stories I've absorbed over the years over how it worksand could benefit you.
The first story is that SMT works
Why region based memory allocation help with fragmentation
Recently, I read an advocacy article on Go's garbage collection ascompared to Java ( via ,which has important criticisms of the technical content, also ). One ofthe things it mentioned is that Go's region based memory allocation reduces memory fragmentation. This struck me as both correct andnot entirely obvious, so I want to talk about why region allocationhelps.
A classical simple memory allocator maintains a pool of blocks ofunallocated memory of various sizes. When you ask to allocate someamount of memory,
How we use the SLURM job scheduler system on our compute servers
The Slurm Workload Manager , often calledjust SLURM, is what a lot of supercomputers and big compute clustersuse to manage scheduling and executing all of the jobs that peoplealways want to run on them. However you don't have to use it insuch grand environments, and we use it in a much more modest one, with a relatively simple usageas experienced by most people. We've actually had two iterationsof our SLURM environment, one that I described in 2019 and the current one
Computation that needs to be "secure" is everywhere in practice
In a comment on my entry on the security or lack of it ofsimultaneous multithreading on modern x86 CPUs , Verisimilitude said in (small) part:
It's obvious to me that secure computation should happen at ahigher level. [...]
This is a very tempting idea in theory, but I think it has a problemin practice, namely that such secure computation is likely to beeverywhere in the modern world. At least, it's everywhere once youstart including confidentiality in your
I wish systemd logged information about the source of "transactions"
It would be nice if systemd would log a line when it starts atransaction to say what the transaction was and what asked it to dothe transaction. Brought to you by the currently burning question of'what asked systemd to reboot this system?'
Back long ago, the only thing that would reboot Linux machines inan orderly way was the system administrator logging in and running' reboot ' (or an equivalent). If a machine rebooted on you and thelogs said it was an orderly reboot
Why it matters to us if exploits are available for security issues
In theory, a security issue is a security issue whether or notanyone has released a public exploit for it, or exploits for ithave been found in use in the wild. In practical system administration,it often does matter, especially for broad and potentially hard tomitigate issues like whether or not simultaneous multithreadingis secure (enough) on modern x86 CPUs .It's a truism that the more that exploits exist and are circulating,the more likely you are to be exploited, but there's
Why V7 Unix matters so much
When I talk about things involving the history of Unix, I oftenwind up mentioning V7, also known as Seventh Edition of Research Unix from Bell Labs (fora recent example, in my entry on when Unix got stack size limits ). If you're relatively new to the history ofUnix, you might wonder why V7 keeps coming up so often. There area number of reasons that V7 matters so much both for the historyof Unix and for what is what we think of as being
Linux has no fair-share scheduling that really works for compute servers
I was recently contacted by someone who has a small group of computeservers and wanted a simple way to do some sort of fair sharescheduling for them, without the various overheads of an actual joballocation system like SLURM . This personwas drawn to me because of my entry on how we do per-user CPU andmemory resource limits on Ubuntu 18.04 .Unfortunately the real answer to their questions is that you cannotreally do useful resource management and fair-share scheduling ofcompute servers with only standard
Why your Go programs can surprisingly be dynamically linked
Recently I read Julia Evans' Debugging a weird 'file not found'error ,where the root problem was that a Linux Go program that Evansexpected to be statically linked (because Go famously producesstatically linked binaries) was instead dynamically linked andrunning in an environment without its required ELF interpreter . Although Go defaults to producingstatic executables when it can, winding up with a dynamically linkedGo program on Linux is surprisingly common. I gave one version ofthe story in a tweet :
The Go standard library can need to