AI Developer Desktop Initiative
Status
This proposal is a request for comment. Please suggest changes, and express interest if you would like to work on any of the issues described below.
Introduction
The Fedora "AI Developer Desktop Initiative" aims to build a thriving community around AI technologies with a focus on three key areas: equip developers with the necessary platforms, libraries, and frameworks; ensure users experience painless deployment and use of AI applications; and build communities to foster mutual support with upstream projects and to solve the problems that have limited participation in the past.
Community building and effective outreach are skilled endeavors, often distinct from a developer's core expertise. This initiative will bridge this gap, help developers find an audience for their work and ensure that their users can easily run and engage with their applications.
Outputs of this Initiative are expected to work on all platforms, from ARM, AMD, Intel, and NVIDIA accelerators. We also hope to provide hardware optimized experiences that go beyond the baseline outputs. We want to prioritize and promote Free Software in Fedora, but we also want to demonstrate that interested third parties can build on top of Fedora systems.
In short, the Initiative will focus on developer tools, community building, and platforms and packaging. Its intent is to determine what barriers to participation exist, and to resolve them through demonstration, documentation, fixing bugs, and improving tools.
Current Issues
Fedora's "Four Foundations" suggest a project that actively engages with emerging technologies, but when we look at software development in the AI/ML field, we often see documentation, development pipelines, and software delivery streams (such as container images) that feature competing platforms.
The purpose of this initiative is to demonstrate ways that Fedora can be used by third parties and to identify the ways that Fedora might be hard to use and to smooth out rough edges.
Complex initial setup turns away prospective users
Fedora provides applications and an operating system on which to run them. A great deal of the work we do to package applications is to ensure that they require minimal post-install configuration in order to be usable. This isn't always the case with AI tools, and local setup complexity varies significantly across hardware vendors. The ideal operating system is built and tested before users deploy it on hosts; we would like to minimize the amount of build and configuration that happens at the deployed system, and therefore the amount of expertise required to install, configure, and use AI tools and workflows.
Kernel and modules
The purpose of the stable release process is to allow users to test a new release and upgrade on asynchronous schedules, by continuing to support previous releases.
Reliable systems generally follow a pattern of build, test, deploy, but many users today use software that requires a deploy, build, test pattern (i.e. akmods). Users frequently disrupt the build because it doesn't provide enough feedback, and if an update doesn't work properly there may not be an obvious rollback process.
The post-install build system is also a challenge for Atomic systems, and at odds with the fundamental motivation for those systems. For the security-minded, the status quo supports Secure Boot only through the use of a local signing key, which offers only the illusion of security, since malware can use the officially supported module install infrastructure to install itself.
Packaging
Complex applications can be very difficult to build in Fedora. Some applications include deep integration with their dependencies, and often use private interfaces. The ability to integrate deeply with dependencies is an advantage of Open Source software, and it encourages a development approach in which components are not siloed. But the cost of that integration is that an application may require very specific release streams of its dependencies. As Fedora is structured today, this places a responsibility on the dependency maintainer, for a benefit expected by the application maintainer.
There have been several attempts to solve this problem or similar problems in the past. Modularity, Software Collections, and containers all approach this problem in different ways. In a general sense, they provide some form of isolation in which an application can be built with its dependencies, and that isolation reduces the friction inherent in the attempt to build one coherent collection of components.
Addressing this is essential to building applications like vLLM (a library for LLM inference and serving), and similarly complex applications in the future. I expect this to be the most complex of the problems to solve, and probably the last deliverable goal. Delivering vLLM may be a sign that we have made progress in this area, if we believe that the approach that enabled its delivery will be generally applicable to complex applications in the future.
No specific solution to this problem is presented as a part of this proposal. Above all, this proposal is an invitation to participate in resolving a problem, not a directive to deploy a finished solution.
Logistical
Many upstream projects lack documentation for Fedora, in part because the OS requires configuration that varies from one hardware vendor to another. When an application developer sees the need for documentation that varies across distributions, and further across hardware vendors, they may choose to focus on the systems that require the least variation in configuration before the user is able to install and use their application.
This can become a self-reinforcing loop, where the appearance that Fedora is not supported causes users to select another platform, which leads to a user base that exclusively uses another platform, which reduces the motivation for documenting software use on Fedora.
Legal / Policy
Fedora is dedicated to Free and Open Source Software. We build and distribute software that users are free to modify to suit their needs and to share with others. While Fedora can't deliver non-Free software directly, we can build relationships with developers who want to provide users with complete systems that include their software. Those partnerships can expand the community to include users and developers who rely on low-level APIs that are not Free Software.
Deliverables
This initiative will demonstrate ways that Fedora can be used by third parties and to identify the ways that Fedora might be hard to use and to smooth out rough edges. Producing the following deliverables will help us discover the challenges and measure progress (whereas uptake is a goal that is more difficult to measure and may develop over a longer period of time.)
1. Document and demonstrate the process of building kernels and modules outside of the project for use by Fedora users. For example, build an LTS kernel as a third-party to deliver the benefit of a stable release process consistently, across the release. These builds may be used in a Remix.
2. Build and sign the NVIDIA out-of-tree Open Source kmod, OpenRM (until the Nova driver is ready), as a third party to eliminate the need for signing keys on the device where the keys are used, to provide a standard best practice build-test-deploy sequence, and to enable an Atomic system Remix to support NVIDIA hardware. (Fedora will not sign the certificates or builds involved.)
3. Publish an Atomic system image focused on support for accelerated AI workloads, composed of only Fedora artifacts, as a Fedora Spin.
3.1 Publish a variant of that Atomic system image with third party kernels or modules, as a Fedora Remix.
4. Include user-friendly tools common to various AI back-ends, such as Goose CLI and Podman Desktop.
5. Provide a contribution guide and invite community developers to directly improve the image rather than writing post-installation configuration guides.
6. Document the installation and use of Free Software projects on Fedora systems, either as contributions to the documentation provided by upstream projects, or within Fedora's documentation.
7. Publish optimized Fedora container images for machine learning applications, aimed at AI developers.
8. Improve Fedora's support for complex applications, which often require very specific dependency streams.
Timeframe
- Platform work, Deliverables 1-4 by F45 release
- Community work, Deliverables 5-6 by F46 release
- Developer tools, Deliverables 7-8 by F47 release
Outreach
Fulfillment of this initiative will require outreach both inside Fedora and beyond. Specifically contemplated contacts include:
- Universal Blue: Are there common objectives in our work and if so, could we work together to help both efforts?
- NVIDIA: Is there an interest in a Fedora Remix for their developer community?
- AMD/ARM/Intel: Provide first-class acceleration where there is upstream support
Feedback
We've talked to Fedora's kernel maintainers and others to clarify that during this initiative, we intend to build kernel and module packages outside of Fedora, and we do not expect those binaries and packages to be signed with Fedora certificates. Furthermore, the longterm kernel package is one possible path to deliver out of tree modules, but not the end goal itself. The end goal is a module packaging process that works reliably without akmods/dkms.
Interested People
Máirín Duffy
Tom Rix
Initiative Lead
Gordon Messmer gmessmer@redhat.com
History
May 19 - Narrow the scope of Remix characteristics, clarify the intent and process of kernel and kernel module builds, expand on the work required for packaging
Notes
A preview of the desktop Remix is here: https://quay.io/repository/gordonmessmer/atomic-desktop/silverblue
The ostree config for that remix is here: https://pagure.io/fork/gordonmessmer/workstation-ostree-config/tree/
The longterm kernel with nvidia module used in that remix is here: https://copr.fedorainfracloud.org/coprs/gordonmessmer/kernel-longterm-6.18-plus/
Non-goals:
The system image will not be pre-configured with applications that inspect or monitor how users interact with the system or otherwise place user privacy at risk.
Tools and applications included in the AI Desktop will not be pre-configured to connect to remote AI services.
AI tools will not be added to Fedora's existing system images, Editions, etc, by the AI Desktop initiative.
Planned work is not expected to change the kernel or kernel module policies, binary content policies, allowed licenses, or the AI contributions policy. Any changes within Fedora will follow the standard changes processes
