It's a pity that Mastodon lacks quote tweets, but in addition to the post I boosted yesterday, I just wanted to say my own thanks for #DTrace reaching 20.
I use it all the time, and struggle when working on other systems that do not have it. Only yesterday I wrote a quick script to verify a code path that lead to this fix: https://github.com/NetBSD/pkgsrc/commit/def71d1c1f988e3a2cf3bb86716a2937ce6f48fc.
It has found significant performance wins in #pkgsrc and #pkgin, fixed numerous bugs, and helped me understand systems.
Thanks @bcantrill and @ahl!
As part of my recent #pkgin work I came across another optimisation for "pkg_admin rebuild-tree" using #DTrace that makes it a further 12x faster on my test system.
http://mail-index.netbsd.org/tech-pkg/2023/08/07/msg027952.html
This is in addition to the 13x speedup I committed a few years ago:
https://gist.github.com/jperkin/98550d5bd07f4179ebfeea825fc3ec20
#pkgin #dtrace #pkgsrc #netbsd
#FreeBSD30 timeline - a few things that happened during these 30 #FreeBSD years:
#UNIX #BSD #MULTICS #1BSD #2BSD #42BSD #43BSD #386BSD #ipfw #FreeBSDCon #BSDCON #ports #jails #FreeBSDFoundation #kqueue #EuroBSDCon #CoreTeam #AsiaBSDCon #BSDCAN #pf #OpenBSD #ZFS #DTrace #VNET #Capsicum #Cheri #Poudriere #clang #llvm #subversion #OpenZFS #git
#freebsd30 #freebsd #unix #bsd #multics #1bsd #2bsd #42bsd #43bsd #386bsd #ipfw #freebsdcon #bsdcon #ports #jails #freebsdfoundation #kqueue #eurobsdcon #coreteam #asiabsdcon #bsdcan #pf #openbsd #zfs #dtrace #vnet #capsicum #cheri #poudriere #clang #llvm #subversion #openzfs #git #freebsdday
After some more #DTrace analysis and code reading, found another easy optimisation win in the pattern matching code, for a further 2x speedup.
Next up some database optimisations which will hopefully get us to around 8x combined speedup, as well as helping with future development work.
Today I used #bpftrace on a #Linux system to investigate a performance issue. Really took me back to my days working on #Solaris at Sun Microsystems. I used #dtrace quite a bit at that time, and it's good to see the same good ideas thriving in Linux.
I worked in #AIX development at #IBM for many years, and I was always a little sad that people didn't use #probevue more. Perhaps this is because AIX has a good system trace infrastructure and the culture grew up using that. Hard to say.
#bpftrace #linux #solaris #dtrace #aix #ibm #probevue
After maintaining #pkgin for a few years, there's still a bunch of low-hanging fruit for cleanup and optimisation.
While working on SUPERSEDES support, I was curious as to why the dependency calculation takes so long.
Re-ran it with #DTrace, saw break_depends() taking up 55% of the runtime. What does it do?
Turns out, nothing. Various changes over the years, notably refresh support, have made its calculations obsolete.
Nice easy 2x performance win and much easier-to-understand code path.
the fact that this laptop is trying to fly away because one tiny virtual machine is hitting one virtual CPU at 100% is beyond ridiculous
… and the reason I'm contemplating diving into the #DTrace manuals at 01:55 in the mor…night.
because, like, doesn't this dtrace_kinst(4)
look really easy??
Update: I *finally* located the post that to me best summarizes the philosophy in question: https://queue.acm.org/blogposting.cfm?id=673 (this is the ACM repost, appropriately enough). Cc @bcantrill @ahl
#gdb #gdbserver #dtrace #unix #socat #ssh #eclipse #linux
@lattera Could #bpftrace be an acceptable replacement? (I don't know #dtrace and have only limited experience with bpftrace)
https://www.brendangregg.com/blog/2018-10-08/dtrace-for-linux-2018.html
DTrace is great for instrumenting kernel, drivers, malware samples and vulnerability research. I will use it in one of the next articles.
@sjmulder You might want different graphs for different timing data, e.g. runtime vs CPU time, on #illumos you can get these out of curpsinfo on process exit but while I think curpsinfo itself is available on all OS the members themselves may vary.
As a quick hack for printing the process tree on illumos I'd system() a ptree(1) on the exiting process and then filter on interesting stacks.
Experimenting with #DTrace is fun! Would recommend just playing and see what you come up with.
@sjmulder The answer is #DTrace, start with something like execsnoop but keep a track of parent/child relationships and how long the processes ran for, then use the tools in the #FlameGraph repository to convert the text output to an SVG.
Sometimes performance work is little more than watching #DTrace execsnoop for duplicate lines while builds are running.
Spotted a huge number of chmod processes being spawned, very quickly tracked it down to "find -exec" using "{} \;" instead of "+".
Tiny change, but reduces number of execs by over 25,000 and significantly reduces build and system time.
If you're interested in hacking on #pkgsrc there is plenty of very rewarding, very easy to fix, low-hanging fruit like this!
So things are getting bad, and #cloud is being considered more complex to manage and more costly.
I have to admit that while I really believed in the cloud, the problems that my customers were facing started to break my beliefs.
Ops costs increased, because they still hired the IT guy, but now a #SiteReliabilityEngineer costs way more than a traditional #SystemAdministrator, and often he has an #infra knowledge way inferior, since everything is not in house anymore. Everything is hosted and you don't access to the technology internals anymore.
Also, they are dealing with a level of complexity unheard of.
Before the cloud there were only physical and virtualized layers of complexity.
Now we also have then #containerorchestration and #servicemesh layers.
We moved from putting a stupid #Java class in classpath to run #JBoss with the #APM profiling to running #Prometheus, #Grafana, #Jaeger and #Kiali to obtain the same level of informations.
The only good thing is that now in Linux engineers are using #eBPF to profiling network and software execution.
But this is NOT innovation at all. You could do that already with #DTrace in #Solaris and #FreeBSD with way less complexity, and the same level of profiling since #Java, #Ruby and lot of frameworks included its probes to support advanced tracing.
Apple's #macOS used to provide a good #DTrace frontend called #Instruments that was really cool to work with.
https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47e0
#cloud #SiteReliabilityEngineer #systemadministrator #infra #containerorchestration #servicemesh #java #jboss #apm #prometheus #grafana #jaeger #kiali #ebpf #dtrace #solaris #freebsd #ruby #macos #instruments