Regarding ZFS, at least this implementation exists for macOS. Darwin/XNU will still expect and require APFS for the filesystem of boot drives.

• (Distant stretch goal:

  Revive the glory fays of the earlier Mac OS X v10.5 developer betas and restore system- and kernel-level ZFS filesystem support.

• Even more distant further stretch goal:

  Add boot support for other filesystems.

  Not sure which, if any, could/would be suitable, however.)

(…Only if that could be done in some way that doesn't break macOS support for users running non-APFS boot filesystems, of course.)

I like the thought of moving ahead with Darwin/XNU, but have you considered leaving anything in the code open to also coming back and making userland multi-platform later? Someobody else could take up FreeBSD backend support and making the shims and compatibility components for that at some point, if they wanted to. Some modularity might help. Maybe that's too difficult, though. Additionally, maybe the existing coupling assumptions needed in reproducing macOS behavior are still too tight, at least for now.

Everything will still be there in the branch if we do ever want to come back. I expect this will evolve as I work through getting everything to build and boot on xnu.

Regarding ZFS, at least this implementation exists for macOS. Darwin/XNU will still expect and require APFS for the filesystem of boot drives.

* (Distant stretch goal:
  Revive the glory fays of the earlier Mac OS X v10.5 developer betas and restore system- and kernel-level ZFS filesystem support.

* Even more distant further stretch goal:
  Add boot support for other filesystems.
  Not sure which, if any, could/would be suitable, however.)

(…Only if that could be done in some way that doesn't break macOS support for users running non-APFS boot filesystems, of course.)

I really only used ZFS because it's good, FreeBSD supports it, and it is similar to APFS. Being able to keep it would be great but not critical. And of course, filesystem support can be added to a kernel fairly easily. There are projects that support ext2/3/4fs and NTFS on macOS, for example.

Also ZFS requires a patent-grant to implement and distribute, even for the open-source implementations (which is why one of those open-source grants' implementation is typically used).

APFS is a fair compromise as it provides most of the same capabilities (with minimal loss in performance on SSD (and backups on older drives can still have other more appropriate formats with Darwin)) But the transparent clustering of partitions in a container group is more user-friendly (albeit more tightly coupled) and provides key aspects of MacOS behaviors including hot-patch upgrades via the update volume and sandboxing virtual memory on a separate partition, and APFS like ZFS can handle multi-teribyte drives that are common today. Regarding extfs4 (2 and 3 are ill-suited) the use of it would have issues specifically with how MacOS likes to enforce encryption without bringing in a slew of Linux/GNU stuff that under-performs compared with APFS, so APFS (while not the ZFS) is best aligned with the other previously mentioned goals of MacOS compatibility/similarity. But I digress.

Add boot support for other filesystems.

Not sure which, if any, could/would be suitable, however.)

Interestingly the public headers from MacOS LibSystem (distantly based on FreeBSD btw) where the /usr/sys/*.h stuff on MacOS lives, contain a bit of insight on which could be suitable: the issue is XNU/Darwin are stripped of the implementation for non-Apple intended hardware. So some of the FreeBSD stuff could actually be brought back in, and that's why apple does just that from time-to-time. The difference is Apple is a for-profit company, and thus always driven in part by profit, where this project is not part of a for-profit company, and thus could explore less profitable, but more interesting experiments with hardware. Apple could support more hardware, but would need to take on the additional support overhead, and possibly go against their company mantra "Think Different℠" and think like open-source communities (e.g. Canonical comes to mind), but they don't see the profit in it (at this time? 🤷).

All that to say: So as an idea, we could fill the gaps with expanded HW support. I'm personally very interested in seeing if we could support some of the older Apple hardware in a sort-of 'lite' fashion, given MacOS often pushes the deprication of support for principles like security (intel CPUs and inherent speculative processing vulnerabilities, or bootloader that requires a hardware secure enclave), and performance (required multi-processors, and more memory for graphics, or actual audio DSP for sound, or SSD for APFS) but the older headers could be shimmed instead of depricated at the cost of performance and security (let's face it, the project isn't to compete with apple's MacOS, which as a for-profit OS will always strive to justify itself against open-source (especially free) alternatives)

Longer-term: Linux thrives in buisness because the industry figured out they can make a buisness of just supporting the free OS, (e.g. conviencnce-as-a-service/Support-aaS ) why not plan to have the same; the further you deviate with hardware the harder your setup might be. 🤷

@mszoek I leave these ideas to you.

Regarding drivers: apple, just lets venders define their drivers for stuff by giving them an API (very blackbox in many-ways), and ABI compatibility should give the project many of those. However, mac-ports is probably the best option besides the fact that we'll need to write/patch/shim many drivers if we want to expand beyond supporting Metal and USB-C/thunderbolt, Broadcom, etc. (e.g. CloverOS area of expertise)

How about we can do all three of them, depending on the situation with each driver?

Using the existing kexts would be top priority, then new native drivers for Darwin get made based on the Linux/FreeBSD driver source code and be maintained, then in the long term a compatibility shim can be made for various stuff from other OS' drivers that don't have any code for it available for rewriting.

So from top to lowest priority, from easiest to hardest: First option, second option, third option

We're on the same page :)

For example real MacOS has no graphic drivers for Tiger Lake while ravynOS does.
It is not a simple work to make a graphics driver. No successes in Hackintosh.
I think the better way is to port all FreeBSD sources to Mach-O.

ravynOS is just using the EFI framebuffer to draw unaccelerated. We had accelerated drivers for AMD and Intel chip sets. Nvidia was a work in progress. If I can get similar framebuffer support out of xnu, we should be ok to start with.

Trust me, getting FreeBSD to build as Mach-O is not easy. The libc in particular has a LOT of assumptions about the binary format and capabilities of ELF objects that are no longer true with Mach-O. And even if we did, there's a lot left to do. I'd rather take a shot at porting some framebuffer code or video drivers that we have source for in other OSes. But we'll see.... this might be a dead end.

• Create a compatibility shim like FreeBSD did for Linux wifi drivers. This means implementing enough of the Linux or BSD kernel APIs that drivers use on top of xnu, then doing minor porting effort to bring in verbatim drivers from those other projects. This is the hardest approach but could unlock a wide range of drivers with mostly one-time effort.

IMHO

@mszoek take a look at my Shim for sys/cdefs.h for XNU/Darwin/FreeBSD/Musl libc at reactive-firewall/liberals, which I use in the same way for porting FreeBSD to musl libc (which is FAR more diverged from the BSDs than XNU is)

I think if the goal is maintainability with near-one-time efforts, the compatibility layer makes the most sense, and aligns with the goal of MacOS API compatibility best.

Hope this helps 🙇

Why we can't use different methods for launch simple FreeBSD application or flat-pack?

In theory (because AFAIK the tools have not been publicly implemented ... yet? 🤷 ) there is no limit preventing us from using different methods, if we can figure out how to implement it (possibly from scratch) however, without a way to share common libraries like the libc or libc++ or libobjc via a common dynamic load object (e.g. ELF libc.so and Mach-O libSystem.dylib provide the same ABI logically in different compiled formats) I would assume we are talking static libs for the Mach-O (which absolutly breaks most of macOS ABI compatibility, as Mach-O defaults to assuming the existence of a global (e.g., MH_DYLINKER) dynamic lib loader) which will grow very, very, large. But I digress into my own speculation.

In the second case we will assume it is MachO application calling MachO libraries.

This is where I get lost; how do we even get to the second case? E.g., in C-lang how do we get the compiler to go from the C Language standard entry-point to the theoretical code that will identify either case? The format (e.g., ELF or Mach-O) of the binary needs to be identified before the code to load and run can even be parsed, all before it can be even checked for things like the MACH-O load header or ELF start code (e.g., all the stuff before the developer's main() function is called) which WILL be needed to launch either case. This is why I assumed an emulator (in traditional ELF format, and work something like wine but for Mach-O) or full fledged Virtual Machine would handle loading the flatpack, where we could then assume the second case as you mentioned, and do stuff like intercept calls to shared code with a custom MH_DYLINKER.

I've gone quite a-bit into explaining some of the why, not to say we can't do things, but rather to keep the focus on @mszoek's initial topic question:

We have to get to Mach-O now. Let's not go into why or timing or whatever - I know we need to and why but I don't want to explain it all. We have to. Now, how?

Why we can't use different methods for launch simple FreeBSD application or flat-pack? In the second case we will assume it is MachO application calling MachO libraries.

Let's follow this line of thought. How does your app inside the flatpak communicate with things like Finder, launchd, and system frameworks? Where do those live? Your flatpak support code or host OS would have to implement all of those APIs and services. If you're going to do that, why not just make apps a native format?

Because I want to launch closed-source applications. Else why all this?
May be you are right and we can make Darwin which is able to launch Mach-O application if we have all needed frameworks. Graphics drivers will be only flat framebuffer or may be even software OpenGL from private Apple's library... may be.

Because I want to launch closed-source applications

Flatpak is necessary for Linux distros because Linux distros don't care enough about offering a stable ABI platform for proprietary apps.

As long as RavynOS takes the proper responsibility to ensure backwards compatibility with proprietary apps, tools like Flatpaks would be unnecessary.

I wrote and wiped a big story, to conclude Linux and BSD bringing a macUlator:Darling? is a better idea than Linux/BSD apps on RavynOS/each-other. However I would still love to see the linuxulator ported.
And for true cross-platform, LLVM IR/.Net is the best.
But I would expect Flatpak macOS to just install mac ports and dependencies from mac ports too.

It's looking good so far. We've been able to build both C and Swift Embedded code (i.e. only needing libSystem) on MacOS and run the result in ravynOS. Just simple stuff, but very promising.

Thank you so much for this vote of confidence! I agree with your take here - it is unfortunate to lose the hardware support, but I believe we can gain it back, and what we've gained is already proving to be huge.

I'll shamelessly take this opportunity to let you all know that I am currently working full time on this OS after being laid off a while ago, so if you are in a position to sponsor financially or know someone who is, please do! There are GitHub Sponsors, Patreon, and Paypal options, or get in touch with me directly. And thank you in advance to anyone who can! I would love to keep bashing this out for y'all and not have to get another day job.