How to File a Good Bug Report

Useful Bug Reports

Useful bug reports are ones that get bugs fixed.

A useful bug report is...

• Reproducible

If an engineer can't see it or conclusively prove that it exists, the engineer will probably stamp it "WORKSFORME" or "INVALID", and move on to the next bug.

• Specific

The quicker the engineer can trace down the issue to a specific problem, the more likely it'll be fixed expediently.

Useful Bug Reports

Goal:

• Pinpoint the bug
• Explain it to the developer

Useful Bug Reports

• Reporter's job

Figure out exactly what the problem is.

• Developer's job

Figure out exactly how to fix the problem.

Writing a Good Bug Report

• Summary
• Description
• Steps to Reproduce
• Actual Results
• Expected Results
• Reduced Test Case
• Version, Platform and OS
• Crash data and/or Screenshot

Summary

• Be precise:

Explain the bug in clear and specific terms.
Prepend "[REG]" to the summary for a regression bug.

• Bad:

"Drag Crash"

• Good:

"Drag-selecting any page crashes Mac builds in NSGetFactory"

Description/Overview

• Goal:

Explain the bug to the developer with a more detailed explanation.

• Tips:

State the problem behavior.
Be as specific as possible.

Steps to Reproduce

• Steps:

Identify the simplest steps needed to reproduce the bug.

• Simple case:
1. Load the attached test case in Browser XYZ
• Complex case:
1. Load the attached test case in Browser XYZ
2. Scroll to the bottom of the page
3. Click the link
4. Press tab to navigate links

Results

• Goal:

Show developer what is wrong.

• Actual Results:

Describe what actually happened.

• Expected Results:

Describe what should have happened.

Reduced Test Case

• Goal:

Pinpoint the bug.

• Reducing Test Cases:

Rip out everything in the page that isn't required to reproduce the bug.

• Variant Test Cases:

Try variations on the test case to find related situations that also trigger the bug.

• Attach all files:

Web pages might change or disappear. Attachments do not.

Environment

Version, Platform and OS

Select the software version, the platform and the OS version that this bug occurred on.

Additional Information

Crash data (backtrace), Screenshot, ...

Sample Bug Report

Mozilla Bug #43220:

Summary:
[CASCADE]author !important rules override user !important rules in user.css
Overview:
!important style rules in the author stylesheet override !important style rules in the user stylesheet ([profile]/chrome/user.css); a direct contradiction of the CSS spec (CSS2:6.4)
Steps to Reproduce:
1.) Copy 'Sample user.css' to the 'chrome' subdirectory in your profile directory for Mozilla. Rename it to 'user.css'.
2.) View 'Testcase' in Mozilla.
* Both to be attached shortly
Tested with build id=2000061720 on Windows 2000

Sample Bug Report

W3C Bug #17686:

Summary:
Audio tests use video element
Description:
Below audio tests use <video> element:

approved/audio/event_canplay.html
approved/audio/event_canplaythrough.html
approved/audio/event_loadeddata.html
approved/audio/event_loadedmetadata.html
approved/audio/event_loadstart.html
approved/audio/event_order_canplay_canplaythrough.html
approved/audio/event_order_canplay_playing.html
approved/audio/event_order_loadedmetadata_loadeddata.html
approved/audio/event_order_loadstart_progress.html
approved/audio/event_pause.html
approved/audio/event_play.html
approved/audio/event_playing.html
approved/audio/event_progress.html

Bug Life Cycle

Bugzilla Bug Status

Bug Triage

Bug Triage is a process to:

• Ensure bug report completeness.
• Analyze and move bug to proper component if needed.
• Set Assignee of bug report to proper bug owner.
• Set appropriate bug priority.
• Adjust bug severity properly (initially set by bug reporter).
• Resolve obvious invalid, duplication, wontfix bugs, etc.

Bug Triage Team:

• A small team works on bug triage, which could be experienced developer, distribution engineer or QA.