QA:Testcase Retrace Server CLI

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(make #9 more clear for testers)
(Update for F16)
Line 1: Line 1:
 
{{QA/Test_Case
 
{{QA/Test_Case
 
|description=This test case tests the [[Features/RetraceServer|Retrace Server]]'s command line interface.
 
|description=This test case tests the [[Features/RetraceServer|Retrace Server]]'s command line interface.
 +
* There are two Retrace Server instances running in the Fedora infrastructure at the moment: {{command|retrace.fedoraproject.org}} and {{command|retrace02.fedoraproject.org}}. {{command|abrt-retrace-client}} defaults to the first one, which only supports Fedora 15 and will be soon replaced by the {{command|retrace02}} machine. For testing purposes the {{command|retrace02}} server needs to be used.
 
|setup=
 
|setup=
 
# Make sure you have ABRT crashes caught by CCpp (crashes in C and C++ applications) that need retrace (i.e. you haven't already retrieved the necessary debuginfo packages and generated the full backtrace) available. If you do not have any, run a C or C++ application and crash it using {{command|kill -SIGSEGV (pid)}} or {{command|pkill -SIGSEGV (processname)}}
 
# Make sure you have ABRT crashes caught by CCpp (crashes in C and C++ applications) that need retrace (i.e. you haven't already retrieved the necessary debuginfo packages and generated the full backtrace) available. If you do not have any, run a C or C++ application and crash it using {{command|kill -SIGSEGV (pid)}} or {{command|pkill -SIGSEGV (processname)}}
Line 6: Line 7:
 
|actions=
 
|actions=
 
# Run {{command|abrt-retrace-client create -d /path/ABRT_crash_directory}}
 
# Run {{command|abrt-retrace-client create -d /path/ABRT_crash_directory}}
−
# Run {{command|abrt-retrace-client create -k -d /path/ABRT_crash_directory}}, the {{command|-k}} option disables HTTPS certificate check
+
# Run {{command|abrt-retrace-client create --url retrace02.fedoraproject.org -d /path/ABRT_crash_directory}}
−
# Run {{command|abrt-retrace-client status -k -t task_id -p task_password}} with task_id and task_password obtained from {{command|create}} action
+
# Run {{command|abrt-retrace-client create --url retrace02.fedoraproject.org -k -d /path/ABRT_crash_directory}}, the {{command|-k}} option disables HTTPS certificate check
−
# Run {{command|abrt-retrace-client status -k -t task_id -p task_password}} with a wrong combination of task_id and task_password
+
# Run {{command|abrt-retrace-client status --url retrace02.fedoraproject.org -k -t task_id -p task_password}} with task_id and task_password obtained from {{command|create}} action
−
# Run {{command|abrt-retrace-client log -k -t task_id -p task_password}} while the job is still {{command|PENDING}}
+
# Run {{command|abrt-retrace-client status --url retrace02.fedoraproject.org -k -t task_id -p task_password}} with a wrong combination of task_id and task_password
−
# Run {{command|abrt-retrace-client log -k -t task_id -p task_password}} after the job is finished (by success or failure)
+
# Run {{command|abrt-retrace-client log --url retrace02.fedoraproject.org -k -t task_id -p task_password}} while the job is still {{command|PENDING}}
−
# Run {{command|abrt-retrace-client backtrace -k -t task_id -p task_password}} after the job is finished successfully
+
# Run {{command|abrt-retrace-client log --url retrace02.fedoraproject.org -k -t task_id -p task_password}} after the job is finished (by success or failure)
−
# Run {{command|abrt-retrace-client backtrace -k -t task_id -p task_password}} after the job is finished by failure
+
# Run {{command|abrt-retrace-client backtrace --url retrace02.fedoraproject.org -k -t task_id -p task_password}} after the job is finished successfully
−
# Run {{command|abrt-retrace-client batch -k -d /path/ABRT_crash_directory}}
+
# Run {{command|abrt-retrace-client backtrace --url retrace02.fedoraproject.org -k -t task_id -p task_password}} after the job is finished by failure
−
# Run {{command|abrt-retrace-client create -d /path/ABRT_crash_directory}}, replacing coredump, executable or package file by some random content
+
# Run {{command|abrt-retrace-client batch --url retrace02.fedoraproject.org -k -d /path/ABRT_crash_directory}}
−
# Run {{command|abrt-retrace-client create -d /path/ABRT_crash_directory --url URL}}, with URL different to {{command|retrace01.fedoraproject.org}}
+
# Run {{command|abrt-retrace-client create --url retrace02.fedoraproject.org -k -d /path/ABRT_crash_directory}}, replacing coredump, executable or package file by some random content
 +
# Try to upload a larger crash (eg. kill JVM).
 +
# Try to replace coderump, executable or package file by some special file (device, symlink)
 
|results=
 
|results=
 
{{admon/note|In order|The expected results match the commands - the first expected result is for the first command, the second is for the second command, and so on}}
 
{{admon/note|In order|The expected results match the commands - the first expected result is for the first command, the second is for the second command, and so on}}
−
# The process should fail because of untrusted HTTPS certificate
+
# The process should fail because F16 is not supported by the default server
 +
# The process should fail because of certificate hostname mismatch
 
# The job should be started, task_id and task_password returned on stdout
 
# The job should be started, task_id and task_password returned on stdout
−
# One of the values {{command|PENDING}}, {{command|FINISHED_SUCCESS}}, {{command|FINISHED_FAILURE}} should be returned on stdout
+
# One of the values {{command|PENDING}}, {{command|FINISHED_SUCCESS}}, {{command|FINISHED_FAILURE}} should be returned on stdout together with a more detailed description
 
# HTTP {{command|404 Not Found}} or {{command|403 Forbidden}} error code should be returned depending on whether task with task_id exists or not. Should be the same with {{command|log}} and {{command|backtrace}} actions
 
# HTTP {{command|404 Not Found}} or {{command|403 Forbidden}} error code should be returned depending on whether task with task_id exists or not. Should be the same with {{command|log}} and {{command|backtrace}} actions
 
# HTTP {{command|404 Not Found}} error code should be returned. Should be the same with {{command|backtrace}} action
 
# HTTP {{command|404 Not Found}} error code should be returned. Should be the same with {{command|backtrace}} action
Line 28: Line 32:
 
# All actions should happen at once - create, periodically ask for status, download log (print to stdout) / backtrace (save into crash directory) depending on whether the result was successful or not
 
# All actions should happen at once - create, periodically ask for status, download log (print to stdout) / backtrace (save into crash directory) depending on whether the result was successful or not
 
# The task should finish by failure
 
# The task should finish by failure
−
# The upload should fail
+
# If the upload size is larger than 8 MB, you should be prompted to confirm
 +
# The request should be denied
 
}}
 
}}
 
[[Category:Package_abrt_test_cases]]
 
[[Category:Package_abrt_test_cases]]

Revision as of 20:33, 24 September 2011

Contents

Description

This test case tests the Retrace Server's command line interface.

  • There are two Retrace Server instances running in the Fedora infrastructure at the moment: retrace.fedoraproject.org and retrace02.fedoraproject.org. abrt-retrace-client defaults to the first one, which only supports Fedora 15 and will be soon replaced by the retrace02 machine. For testing purposes the retrace02 server needs to be used.

Setup

  1. Make sure you have ABRT crashes caught by CCpp (crashes in C and C++ applications) that need retrace (i.e. you haven't already retrieved the necessary debuginfo packages and generated the full backtrace) available. If you do not have any, run a C or C++ application and crash it using kill -SIGSEGV (pid) or pkill -SIGSEGV (processname)
  2. The crash reports will be under /var/spool/abrt; you can identify them by the date and time

How to test

  1. Run abrt-retrace-client create -d /path/ABRT_crash_directory
  2. Run abrt-retrace-client create --url retrace02.fedoraproject.org -d /path/ABRT_crash_directory
  3. Run abrt-retrace-client create --url retrace02.fedoraproject.org -k -d /path/ABRT_crash_directory, the -k option disables HTTPS certificate check
  4. Run abrt-retrace-client status --url retrace02.fedoraproject.org -k -t task_id -p task_password with task_id and task_password obtained from create action
  5. Run abrt-retrace-client status --url retrace02.fedoraproject.org -k -t task_id -p task_password with a wrong combination of task_id and task_password
  6. Run abrt-retrace-client log --url retrace02.fedoraproject.org -k -t task_id -p task_password while the job is still PENDING
  7. Run abrt-retrace-client log --url retrace02.fedoraproject.org -k -t task_id -p task_password after the job is finished (by success or failure)
  8. Run abrt-retrace-client backtrace --url retrace02.fedoraproject.org -k -t task_id -p task_password after the job is finished successfully
  9. Run abrt-retrace-client backtrace --url retrace02.fedoraproject.org -k -t task_id -p task_password after the job is finished by failure
  10. Run abrt-retrace-client batch --url retrace02.fedoraproject.org -k -d /path/ABRT_crash_directory
  11. Run abrt-retrace-client create --url retrace02.fedoraproject.org -k -d /path/ABRT_crash_directory, replacing coredump, executable or package file by some random content
  12. Try to upload a larger crash (eg. kill JVM).
  13. Try to replace coderump, executable or package file by some special file (device, symlink)

Expected Results

Note.png
In order
The expected results match the commands - the first expected result is for the first command, the second is for the second command, and so on
  1. The process should fail because F16 is not supported by the default server
  2. The process should fail because of certificate hostname mismatch
  3. The job should be started, task_id and task_password returned on stdout
  4. One of the values PENDING, FINISHED_SUCCESS, FINISHED_FAILURE should be returned on stdout together with a more detailed description
  5. HTTP 404 Not Found or 403 Forbidden error code should be returned depending on whether task with task_id exists or not. Should be the same with log and backtrace actions
  6. HTTP 404 Not Found error code should be returned. Should be the same with backtrace action
  7. Plaintext log should be returned on stdout
  8. Plaintext backtrace should be returned on stdout
  9. HTTP 404 Not Found error code should be returned
  10. All actions should happen at once - create, periodically ask for status, download log (print to stdout) / backtrace (save into crash directory) depending on whether the result was successful or not
  11. The task should finish by failure
  12. If the upload size is larger than 8 MB, you should be prompted to confirm
  13. The request should be denied
Retrieved from "https://fedoraproject.org/w/index.php?title=QA:Testcase_Retrace_Server_CLI&oldid=254565"