How to Report a Bug
If a test fails, congratulations, you have just found a bug. It may be a
browser bug (aka an implementation bug), a bug in the specification, or an
issue with the test suite itself.
What is a Useful Bug Report
Useful bug reports are ones that get bugs fixed. A useful bug report is...
- Reproducible - If an engineer can't see it or conclusively prove that it
exists, the engineer will probably stamp it "WORKSFORME" or "INVALID", and
move on to the next bug.
- Specific - The quicker the engineer can trace down the issue to a specific
problem, the more likely it'll be fixed expediently.
So the goals of a bug report are to:
- Pinpoint the bug
- Explain it to the developer
Your job is to figure out exactly what the problem is.
Bug Reporting General Guidelines
Avoid duplicates: Search before you file!
Always test the latest available build.
One bug per report.
State useful facts, not opinions or complaints.
Flag security/privacy vulnerabilities as non-public.
How to Write a Good Bug Report
A good bug report should include the following information:
Summary
The goal of summary is to make the report searchable and uniquely
identifiable.
A bad example: Drag Crash
A good Example: Drag-selecting any page crashes Mac builds in NSGetFactory
Overview/Description
The overview or description of a bug report is to explain the bug to the
developer, including:
Abstracted summary of behavior (e.g. interpretation of test failures)
Justifications of why this is a bug
Any relevant spec links
Interpretation of the spec
Information on other implementations
Steps to Reproduce
The goal of reproducible steps is to teach developer to recreate the bug on
his own system. It may be as simple as Load the attached testcase in Browser
XYZ
. A more complex case may involve multiple steps, such as:
Step 1: Load the attached testcase in Browser XYZ
Step 2: Scroll to the bottom of the page
Step 3: Click the link
Step 4: Press tab to navigate links
Test Results
The test results, including Expected Result and Actual Result, will show
the developer what's wrong. Expected Result describes what should have
happened, and Actual Result describes what actually happened.
Reduced Test Case
The goal of test case is to pinpoint the bug. A Reduced Test Case rips out
everything in the page that is not required to reproduce the bug. Also try
variations on the test case to find related situations that also trigger the
bug.
Environment Setup and Configuration
Please also include the environment setup and configuration information, such
as OS, system build and platform etc.
Any additional Information
Include any additional information such as URL, crash data, regression range
etc.
Attach all relevant files
Where to Report a Browser Bug
You need to go to vendor's bug reporting website to report a bug of browser.
Where to Report a Spec Issue/Bug
Somewhere in the beginning part of any spec, you should be able to find a
link for reporting any problems in the spec. Look for a link with text such
as new issue, new bug, file an issue, or report a bug; or in the
case of some W3C specs, the text If you wish to make comments regarding
this document, please send them to followed by an e-mail address for
sending a message to a mailing list for a particular W3C working group.
Where to Report a Bug of a Test Itself
You need to follow Github's normal workflow and use "issues" to report a bug of a test itself.
Sources