From Fedora Project Wiki
No edit summary
No edit summary
Line 12: Line 12:


* I honestly do not see the advantage of packaging the tests. I doubt that for most project upstream is going to release them as a tarball which means the packagers will have to do that. Then write down the process on how to execute them. Why not doing this with something like ansible from the start? It makes it easy to list the dependencies (just install them in one task). Then you would need to specify how the tests should be run, which can be just as easily done in ansible. It also means that we would have to go through the FPC to get this approved in the packaging guideline, for imho little benefit.
* I honestly do not see the advantage of packaging the tests. I doubt that for most project upstream is going to release them as a tarball which means the packagers will have to do that. Then write down the process on how to execute them. Why not doing this with something like ansible from the start? It makes it easy to list the dependencies (just install them in one task). Then you would need to specify how the tests should be run, which can be just as easily done in ansible. It also means that we would have to go through the FPC to get this approved in the packaging guideline, for imho little benefit.
* I agree with above about not enough benefits with packaged tests. Right now we have already problems with slow dnf, dowloading too much, huge metadata and extra slow dependency solving. This won't be beneficial to our users, yet it would have negative impact on them. Some tool like fedpkg or something will be needed anyway. Same with configs, dependencies, etc... we will need something else, as rpm can't cover everything. So I think it would be good idea to leave rpm out of this and not load another weigh on core infrastructure, processes and user experience.

Revision as of 17:30, 11 April 2017

  1. AliVigni: In invocation why would I want to hardcode absolute paths for test execution, artifacts, logs, This should be a relative path so where ever you run things it is in the local workspace. My machine, Jenkins, taskotron, etc.
    • MartinPitt: I reworked the invocation; it was also impractical for tests that run as non-root, and it would have potentially clobbered the root directory with temporary stuff.


  1. tflink: As I understand it, the proposal requires -test subpackages to either have globally-unique file names or explicit conflicts in the spec file. Why not use a subdirectory matching the name from the specfile e.g. /usr/tests/gzip for the gzip packaged tests? That would make filename conflicts much less likely and would be one less thing for packagers to worry about when including tests.
    • MartinPitt: Excellent point; spec changed to /usr/tests/srcpkgname/ to make use of the already unique name space that source packages (aka. spec file names) give us. Will that be sufficient to map a source package to all of its binary packages that contain tests? I. e. "give me all rpms of the gtk+ source that provide tests?
  1. pingou:
  • Execute all executable files in /usr/tests/*/ directories one at a time.

This is a neatpick but there will be people complaining about it since /usr/tests isn't in the FHS and isn't really a good place for executable, we could suggest using /usr/libexec which is meant for executable and put a /test subfolder there

  • I honestly do not see the advantage of packaging the tests. I doubt that for most project upstream is going to release them as a tarball which means the packagers will have to do that. Then write down the process on how to execute them. Why not doing this with something like ansible from the start? It makes it easy to list the dependencies (just install them in one task). Then you would need to specify how the tests should be run, which can be just as easily done in ansible. It also means that we would have to go through the FPC to get this approved in the packaging guideline, for imho little benefit.
  • I agree with above about not enough benefits with packaged tests. Right now we have already problems with slow dnf, dowloading too much, huge metadata and extra slow dependency solving. This won't be beneficial to our users, yet it would have negative impact on them. Some tool like fedpkg or something will be needed anyway. Same with configs, dependencies, etc... we will need something else, as rpm can't cover everything. So I think it would be good idea to leave rpm out of this and not load another weigh on core infrastructure, processes and user experience.