@thindil Haha, that was just a gentle hint 😉. But I think it's really important to understand the difference between #Linuxulator and #LinuxKPI, although you're certainly right, *both* were further improved/extended in #FreeBSD 14.

BTW, I'm pretty sure there is *no* GPL-licensed code in the kernel. There is some GPL-licensed code in base, yes, but you can build base without it; the goal is always to provide a full BSD-licensed OS.

In fact, if I'm not mistaken, remains of GPL-licensed files in LinuxKPI are the main reason we still have drm-kmod drivers as a port (and not integrated with the base kernel), where this port includes these small parts of LinuxKPI. The drm drivers themselves are not GPL-licensed.

@thindil @emaste Please understand that a #syscall is something completely different than some other, in-kernel, call. A syscall is the special way userspace can call something in the kernel, which needs a context switch. Some architectures have special CPU instructions for syscalls available, on other architectures, software-interrupts (that the kernel can handle) are used, etc.

Nothing of that is done for in-kernel calls, they're "just" function calls.

#FreeBSD #Linuxulator provides a set of #Linux-compatible syscalls. The #LinuxKPI project on the other hand aims to provide in-kernel Linux compatibility for use by certain drivers. They're completely unrelated.

BTW, you're missing a syllable, it's spelled LinuxULator 😉

Did all these tests, did some fixes, "#Linuxulator userland from source" branch builds fine on #FreeBSD 14-CURRENT/13.2-RELEASE, aarch64/amd64/i386 🥳

Now doing test builds with 15-CURRENT, which already has a fix for the #Linux #xattr issue. Unfortunately, it's still incomplete. Neverending story 😞

JFTR, not blaming dchagin at all. It seems Linux has some very weird design decisions, and semantics of the xattr syscall return codes -- EPERM is considered fatal by GNU/Linux tools, because Linux returns ENOATTR or ENOTSUP when access to e.g. the system namespace is restricted 🤯

Double-checking still in progress. So far, fixed an issues on 13.2/aarch64, another one on 14-CURRENT/amd64 (yep, didn't upgrade my #FreeBSD test builders to 15-CURRENT yet 🙈) and yet another one on 14-CURRENT/i386.

Now, test builds for 13.2/i386 are running. We will see. Once I'm sure the #Linuxulator version of #ffmpeg builds fine everywhere, I'll finally check #MakeMKV 😎

And now, the *real* dependency monster in my #FreeBSD #Linuxulator userland: #ffmpeg 🙈

That's #Linux port #151 I added, it has *almost* everything enabled that's in the default options of the FreeBSD port. I left out a few things that seemed (too) complex like vulkan ... 🙈

I guess now it's time to double-check the branch on other architectures and other FreeBSD versions first. And then, finally, check whether #MakeMKV will work fine with this!

@meka Well, if you can build them, you can (probably) port them to work natively on #FreeBSD. This is "just" about building the userland for #Linuxulator to support closed-source (or, otherwise "unportable") #Linux apps.

I'm still unsure whether it's a good idea. I was thinking about using #MakeMKV as a PoC, which requires #ffmpeg. Only half way (or less) through with the dependencies for a somewhat "full-featured" ffmpeg and I'm already at 135 commits to my branch 🤯 I certainly won't be able to maintain all the ports needed for this userland all by myself 😉

Minor news from my #FreeBSD #Linuxulator userland project: I now succeeded to build a first "dependency monster" (in order to have all features consumers might expect): #cairo. Among other things, required porting all the #Xorg libs first 😉

I'm about to force-push my #Linuxulator userland branch now, removing all hacks to disable xattr usage.

TL;DR is: If you want to test it on #FreeBSD 14 and newer right now, you'll have to apply this patch: https://people.freebsd.org/~dchagin/xattr.patch -- I hope it will be committed to main and stable/14 soon 😎

The (weird) background is: Support for #Linux xattr syscalls was added quite recently, and it correctly maps the Linux syscalls to the FreeBSD ones. So far, so good. BUT: Access to the "system" namespace for extended attributes is typically restricted to root (and, on FreeBSD, also restricted in #jails). Now, FreeBSD returns EPERM on attempts, which IMHO makes perfect sense. But, Linux returns ENOTSUP in these cases instead. And: GNU tools and other Linux software using extended attributes considers EPERM a *fatal* error as a consequence. This means things like "install" from GNU coreutils are now broken in jails and as non-root. 🤯

The patch above fixes this.

@alexr Sorry, but it really doesn't make sense to me to compare anything just regarding #Linux vs #Linuxulator. The kernel(!) as a source of indeterminism is very unlikely, everything else (like #glibc where the allocator is implemented) is the same. I don't see what I would gain from that huge amount of work here.

@alexr Are you suggesting that the same GCC (native #Linux, not cross-target) would produce different code when the Kernel isn't "real" #Linux, but #FreeBSD's #Linuxulator?

I still see no sane way to check this, maybe short of installing this very same version on a Linux system and comparing assembler output... 🤔 but it doesn't seem like a likely thing to me 🤷‍♂️

Did some minor fixes and overall improvements in the new USES=linuxsrc today ... it already grew quite large 🤔
https://github.com/Zirias/zfbsd-ports/blob/linux/Mk/Uses/linuxsrc.mk

BTW, a while ago, I couldn't resist to add some #branding to the #FreeBSD #Linuxulator toolchain, currently in line 119.

I mean, the #vendor field in these target triplets is there for a reason, right? To be used by vendors? 😏

Python issue solved the "redneck repair" way, installing *everything* to /usr/lib64 (on a 64bit system) instead of *just* the architecture-dependent stuff. Now, python works and finds all its modules... 🙄

After porting lots of extra python tools (cause #pip won't do it for proper packaging with staging), it seems I finally have a working #meson in my #FreeBSD #Linuxulator userland 🥳

Nevermind all these extra modules install to /usr/lib. Well, it works, I don't care 😂

Doing a full rebuild of all the #Linux ports now just to be sure (cause I also changed stuff in my USES). If all is fine, branch will be updated 🙏

Hitting the next roadblock in my #FreeBSD #Linuxulator "userland from source" project: #Linux #Python is causing trouble. I got it to build and package, but runtime parts are wrong, *somehow* the "/compat/linux" prefix "sneaks in" 😞 (still didn't understand how exactly...).

You might ask why port python at all, and indeed, I didn't plan to do so initially.

The issue is build systems. With the goal to port shared libraries that might be needed by (closed-source) Linux binaries, not all of them use plain #make (or #GNU #autotools) for building.

#cmake is popular, so I'll need it. To build it using shared libraries itself, one of its prerequisites needs #meson, which, in turn, requires python. 🤯

Ahh, welcome to dependency hell 😜

Userland built from source for #FreeBSD's #Linuxulator is making progress again (and yes, my branch builds and works again!)

☑️ Metaports "linuxsrc_base" for a minimal base userland and "linuxsrc-devtools" for minimal basic tools needed to build #Linux software (just C and C++ and some common tools)
☑️ Automatic selection of the Linux kernel version based on target FreeBSD version, overridable in DEFAULT_VERSIONS (but checked for compatibility of course)
☑️ A new "USES=linuxsrc" to do all the ugly "plumbing" needed for building Linux software using FreeBSD #ports, offering different configurations.

Guess now I could start adding some optional packages and try to get some *real* Linux application working 😎

https://github.com/Zirias/zfbsd-ports/blob/linux/Mk/Uses/linuxsrc.mk

In case anyone tried and wondered why the branch is currently broken 🙈

I'm working on adding a new USES for this #Linuxulator userland. It should hide all the "crap" needed to build it, and also help with creating new ports, either *for* it (e.g. shared libs), or just using it.

So I have to rework each and every port now, including test builds...

Right now, I'm test-building "native" gcc ... for the third time 😂 so, native toolchain almost complete again.

I hope to get the branch into sane shape tomorrow!

#Linux #FreeBSD #Linuxulator

Today's achievement: We now have #OpenSSL (and #GNU coreutils built using it), plus grep, sed, awk, make, groff *and* man-db in #FreeBSD's #Linuxulator userland.

But there's a catch 😞 It doesn't build with #poudriere any more. Can be patched, and I guess I should soon look into getting this fixed...

For details, see here:
https://lists.freebsd.org/archives/freebsd-ports/2023-August/004286.html

Today's progress: #GNU #coreutils and glibc's localedata seem to work fine in #FreeBSD's #Linuxulator, in addition to bash.

Still, coreutils is not complete yet, it *should* used hash functions from #OpenSSL, so, time to port that for #Linux (tomorrow).

Time for cleanup after identifying two "interesting" issues:

1.) #FreeBSD ports come with a Templates/config.site file which is normally used by #GNU #autoconf, containing quite a lot of "cache variables" so autoconf can avoid running the actual tests. Nice, but completely wrong when targeting #Linux. So, add an empty "CONFIG_SITE=" variable to every linux port, and suddenly I can also remove some explicit cache variables again 🙈

2.) I'm building the "native" toolchain for #Linuxulator --with-sysroot=/compat/linux, which makes sure the linker reliably finds startup files and system libs, without stuff from FreeBSD base interfering. BUT: This completely breaks when some explicit lib path to /usr/lib or /usr/lib64 is added (and many build systems will do that), because sysroot doesn't apply here. Therefore, add wrapper scripts around gcc and g++ filtering out any "-L/usr/lib" or "-L/usr/lib64" argument 🙈 with that, Linux #bash builds without any patches! 🥳

Now, waiting for builds.

And now, we have a working #Linux #bash running in #FreeBSD's #linuxulator

Which also finally makes the "ldd" script installed by #glibc work 😎

Ok, enough for today 😉

https://github.com/Zirias/zfbsd-ports/blob/linux/shells/linux-bash/Makefile

Finally, my #Linux-native #GNU toolchain for #FreeBSD's #Linuxulator works for C and C++ on #aarch64, #amd64 and #i386 🥳

I guess that's a good time to actually announce the ongoing project!
https://lists.freebsd.org/archives/freebsd-ports/2023-August/004263.html