Purpose - Request for Resources
The purpose of this page is to help track requests for resources and to attach resources with their project leaders. It is primarily intended to cover non-routine requests for significant resources that require ongoing maintenance, e.g. deploying an entirely new service in Fedora Infrastructure. Routine requests, e.g. to add an organization to a forge, do not need to go through this process; you can simply file an informal infrastructure ticket to request those. In general the way this works is someone fills out an RFR ticket (see below). If for some reason the request is denied the requester should take their issue to the Fedora Council. If you work for a company or Red Hat and have a budget to back the request, you are far more likely to be able to get request approved. There's only so much stuff to go around.
Request Resources
First, please read this entire page and collect answers to questions you will be asked. If your resource requires packages, please make sure that you create, get approved and have your package(s) available in Fedora and/or EPEL before you file your RFR ticket. Then fill out a Request for Resources ticket. Include all information about why this is important for Fedora and why we should dedicate resources to test it and ultimately deploy, support and back it up. You may wish to join the Infrastructure room on Matrix to enable real-time discussion of your request.
What to expect
All infrastructure discussion is done in the open as much as possible. This is especially true for new ideas and new requests so be prepared to defend your ideas and make sure that you have a clearly defined goal in mind. Unfortunately our resources are finite and we can't just do everything for everyone whenever they ask for it. Denied requests can be appealed to the Fedora Council, and even if they should say no, there's always other places to turn to to host your project.
Significant requests to e.g. add entire new systems or substantially expand the scope of existing ones in ways that require ongoing management and maintenance are most likely to be approved if they can be implemented as an OpenShift application. Hosting new virtual machines or hardware is significantly more work and requires substantial justification to be approved. You will be asked to work with the infrastructure team to implement deployment and maintenance via Ansible. For OpenShift-based applications, you can provide container images yourself, or work with the infrastructure team to set things up so images can be automatically built from a project repository and added to the internal registry. For VM or hardware-based applications, you will be asked to work with the infrastructure team regularly to keep them updated.
It is the job of those requesting resources to find a sponsor and convince them the project is worth their time. *Not all projects can be approved* If not initially accepted, remember to contact a sponsor and explain why the project should be accepted and how Fedora benefits from it. Contact a sponsor on the FIGs page related to the request. For example, if you're building a website, contact a sponsor of the sysadmin-web or sysadmin-test group.
Project Leaders
In the perfect world your request would get approved and within days your request would be live and people could access it. For complex requests, this takes much longer with a lot more work. These requests require a project leader. The project leader is typically the person who made the RFR and is responsible for seeing the project through to conclusion. They will need to find an infrastructure member with access to the test servers and must work together with the team to make sure the work is getting done.
The infrastructure team doesn't intentionally not do work but often times other things get busy and if a Project Leader isn't around to continue to work with Infrastructure by asking when things will be done and what can be done to help, then that RFR typically gets forgotten about. This is actually a very community way of doing things. In the corporate world the boss says do and it gets done. In the community world if the person who thought the thing up doesn't care to stick around and see it through to conclusion then its questionable if that thing should have been done in the first place.
Responsibilities of an RFR Project Leader
If you have a service that is accepted as an RFR that you plan to eventually have deployed permanently in Fedora you are signing up for maintenance as well as for initial coding and deployment. Keep that in mind :-)
Here's a list of some of the things that we expect of an RFR maintainer. Note that infrastructure is a team so it won't just be on your shoulders to do these things but equally, infrastructure is a team of which you're a part and it's not fair to the rest of the team to bring in new maintenance burden without pulling your own weight.
Bringing in a team of people to do these things is always appreciated so there's not a single point of failure.
- **Recruiting and training other people to work on the service so that you aren't a single point of failure**
- Keeping the service and to any underlying pieces of the application stack updated if necessary.
- Note that there are infra policies on freeze periods around release and updating pieces of the software stack may require you to interact with the teams working on other services deployed in infrastructure.
- Applying hotfixes in an appropriate manner if there's an urgent security issue or bug.
- Keeping up with upstream development if appropriate.
- This includes keeping track of security fixes. Note that for many apps, this task involves much more work than simply following the Fedora package updates.
- Answering questions about whether an update (to your app or to the underlying stack) might break your app.
- Also, testing if you don't immediately know the answer
- Also coding patches to fix things should it become apparent that the app is broken with the update
- Fixing things should an app start throwing errors in production for unknown reasons
- Could include deploying to staging
- Could include coding and diagnosing
- Could include spending long hours staring at log files
- Could include being paged or contacted off hours to get the service back on line.
- Work on deployment problems
- It's too slow, what can we change to speed up this page?
- Testing things in staging before deploying to production
- Maintaining packages of the application (if it's packaged), or container definitions (if it's a container), etc.
People Resources
In addition to a Project Leader, each resource should have a community of maintainers around it. Resources with single leaders or where there is not a healthy turn around of contributors can lead to failure when the leader is busy or unavailable. Resources with a single leader/maintainer may be rejected until a larger community is available. If your resource is popular / desirable, you should have no trouble on-boarding interested maintainers.
Commitment
When submitting a non-routine Request for Resources the RFR Leader should realize that they are committing to producing, testing, deploying and maintaining this resource. If they become unable to continue, it's the responsibility of the RFR project leader to hand off the resource or retire it. Don't expect things to magically continue when there are no interested folks maintaining or driving deployment.
Requirements
All projects must be deployed on working, supported hardware (ideally, in OpenShift; less ideally, as virtual machines; least ideally, directly on bare metal). Hardware that is EOLed or "hobby hardware" will never be accepted into Fedora Infrastructure for a production application that we then support. The bottom line is that we're not going to say "we now support this" if the box might die in the middle of the night and no replacements can be found.
All production resources must be clusterable / load balanceable. Preferably the latter. Containerized deployment goes a long way towards achieving this, and that's one of the many reasons it is the preferred option. For non-containerized requests, remember you're typically asking for _at least_ double what you need to get something up and running. Shared storage or database being the exception to this. Make sure to think hard about what you're asking. Don't forget about backups and storage space. Imagine if your project is wildly popular but suddenly running out of disk space.
Ongoing maintenance
When considering driving a RFR, keep in mind the on-going maint costs for the resource:
- How often does the package need to be updated (for instance for security fixes)?
- Do we sometimes have to program our own code to fix things/auth to fas/etc?
- Is the upstream alive or dead?
- Do we have a relationship with upstream where we can ask them to do things for us?
- Is the upstream branch going to be producing bugfixes (or at least, security fixes) to the service for a long time?
- How easy is updating? (press the rollout button or dnf update && done at one end of the spectrum; we're packaging ourselves, porting our custom addons, run a series of scripts to update the production database, update the config file, finally, take an outage to actually do the update at the other end of the spectrum).
Tips and Hints
What can you do to get your RFR accepted and moved out to users better/faster?
- Have your resource use tools that are already widely deployed in Fedora Infrastructure (python, flask, etc) and not things that are not (jboss, java, ruby on rails, etc).
- Make sure your resource is already deployable as a container and/or packaged up for Fedora/EPEL before filing a RFR. Sometimes this can take a while, and filing an RFR too early will cause people to get discouraged or less excited by your project.
- Make sure you have multiple people involved/willing to help.
- Be available on Matrix for problems or questions about your resource.
- Feel free to nag your sponsor and others to keep driving things forward, but keep in mind that this is a process and we will NOT skip steps or apply a shoddy workaround to get your resource deployed faster.
- Gather information from affected fedora groups before proposing your RFR. For example, if your resource is related to package maintainers, tell them about it and get feedback first. Having a clear mandate from a Fedora group that your resource would help them improves chances for it to be accepted/worked on.
Warnings
- If at any point in time the Project Leader or secondary contacts become unresponsive, the project may be removed.
- Test instances are NOT backed up unless requested. Once something goes live it will be brought into the normal fold of what we do
- Ultimately the Council has final say on what is and is not "Fedora"
