CSS Test Infrastructure

Elva Fu, Intel

Peter Linss, HP, CSS WG Co-Chair

Elika J. Etemad (fantasai), Mozilla Project, W3C CSS WG

Overview

  • Test Metadata
  • CSS Test Repository
    • Version Control - Mercurial
    • Repository Layout
    • View Repository Online
  • Test Suite Manager - Shepherd
  • Build System
  • Test Harness
  • Demo

Test Metadata

  • Information about test
    • What specification section is being tested
    • Description of what the test is doing
    • Author contact
  • Processed by tools
  • Stored inside test when possible

Template


<!DOCTYPE html>
<html>
 <head>
  <title>CSS Test: SCOPE OF TEST</title>
  <link rel="author" title="NAME_OF_AUTHOR" href="mailto:EMAIL OR http://CONTACT_PAGE">
  <link rel="help" href="RELEVANT_SPEC_SECTION">
  <meta name="flags" content="TOKENS">
  <meta name="assert" content="TEST ASSERTION">
  <style>
   CSS FOR TEST
  </style>
 </head>
 <body>
  CONTENT OF TEST
 </body>
</html>

http://wiki.csswg.org/test/format#template-for-new-tests

Test Repository: Version Control - Mercurial

  • hg clone :
    • Download copy of repository.
  • hg commit :

    • Save your changes as a changeset with a checkin comment describing changes.

  • hg push :

    • Share your changeset as a checkin to the master repository.

  • hg pull --update or hg pull --rebase :

    • Download others' updates into your copy. Do this before you commit!

  • hg merge :

    • Simultaneous changesets: need to merge branches.

    Read the details at http://wiki.csswg.org/tools/hg

Test Repository: Repository Layout

Approved tests:

approved/

    {spec}/

    src/

Tests under development:

contributors/

    {name}/

    incoming/

    submitted/

Test Repository: View Repository Online

http://hg.csswg.org/test/

CSS Test Suite Manager: Shepherd

  • General management tool for test repository
    • Integrated with Mercurial
    • Understands Test Metadata
    • Performs automatic basic checking of tests
    • Provides searches on metadata
    • Bug tracker / test approval
    • More to come...
  • http://test.csswg.org/shepherd/

Build System

    Generates final test suites
  • Gathers tests from submitted/ directories
  • Creates specification based test suites
  • Creates human readable index pages
  • Creates machine readable manifests
  • Performs format conversion
    • HTML . XHTML . XHTML-Print
  • Injects vendor prefixes
  • Runs nightly

Test Harness

    Where tests are run and results gathered
  • Presents built test suites
  • Gathers results by user agent
  • Generates implementation reports
  • Presents tests in order where results are needed most
  • Generates live specification annotations

    • http://test.csswg.org/harness

Demo

More information

http://wiki.csswg.org/

public-css-testsuite@w3.org