Tizen Web TCT

Tizen Compliance: Why?

Tizen Compliance Program

• ensures devices and applications work correctly together
• provides a validation mechanism, certification
• benefits to application developers, device implementers, carries, end users

Tizen Compliance: What?

• Tizen Compliance Specification (TCS)
• Tizen Compliance Tests (TCT)
• Tizen Compliance certification

Tizen Profile

A Tizen Profile describes the requirements for a category of Tizen devices that have a common application execution environment.

Tizen Compliance Tests (TCT)

Tizen Compliance Tests (TCT)

• verify conformance to the Tizen Compliance Specification (TCS)
• consist of Web TCT and Native TCT
• Web TCT is a set of tools and test suites to test the web requirements defined in TCS

Web TCT Workflow

Web TCT Architecture

Web TCT Projects

Search "test/tct/web" at review.tizen.org

• test/tct/web/behavior
• test/tct/web/manager
• test/tct/web/shell
• test/tct/web/utilities
• test/tct/web/webapi
• test/tct/web/webuifw
• test/tct/web/wrt

Web TCT Projects - More

testkit related repositories at github.com/testkit

• testkit-lite
• testkit-stub
• tinyweb
• webrunner

Web TCT Projects - Branch

Most projects has 4 branches:

• master: for development or empty
• tizen_2.2: for Tizen 2.2.1
• tizen_2.3: for Tizen 2.3
• tizen_3.0: for Tizen 3.0

Web TCT Tests

• Web Test Suites cover Tizen Web Device API, Tizen Web Runtime, W3C/HTML5 and Supplementary API, Web UI Framework, device capability features, and/or Vehicle API.
• Web TCT Behavior Test tool is used to test behavior of hardware and software features in interactive mode.

Tizen Web Device API Tests

• tct-<spec>-tizen-tests
• tct-namespace-tizen-tests
• tct-privilege-tizen-tests

tct-alarm-tizen-tests/
+-- alarm
    +-- support
        +-- UnitTest-AlarmTest1
+-- resources
+-- webrunner
      

W3C/HTML5 API Tests

• tct-<spec>-css3-tests
• tct-<spec>-html5-tests
• tct-<spec>-w3c-tests

tct-2dtransforms-css3-tests/
+-- 2dtransforms
    +-- csswg
+-- resources
+-- webrunner
      

Supplementary API Tests

• tct-<spec>-nonw3c-tests

tct-webgl-nonw3c-tests/
+-- resources
+-- webgl
    +-- khronos
        +-- resources
    +-- support
    +-- w3c
+-- webgl-py
    +-- khronos
+-- webrunner
      

Vehicle API Tests

• webapi-vehicleinfo-ivi-tests

webapi-vehicleinfo-ivi-tests/
+-- resources
+-- vehicleinfo
    +-- support
+-- webrunner
      

Device Capability Features Tests

• tct-capability-tests
• tct-security-tcs-tests

tct-capability-tests/
+-- capability
    +-- support
+-- resources
+-- webrunner
      

Configure and Get Capability

• tct-testconfig
• tct-getcapabilities

Web TCT Behavior Tests

• tct-behavior-tests

+-- css
    +-- images
+-- js
    +-- thirdparty
+-- res
    +-- css
    +-- font
    +-- images
    +-- js
    +-- media
+-- tests
    +-- 2D3DAnimation
    +-- Accelerometer
    +-- Alarm
    +-- ...
      

Test Framework and Runner

• resources/: w3c/testharness.js
• webrunner/: testkit/webrunner

Tests from Upstream

• csswg/: W3C CSS WG tests
• w3c/: W3C Web platform tests
• w3c/: W3C widgets tests
• khronos/: Khronos WebGL conformance tests
• blink/: Blink layout tests
• webkit/: WebKit layout tests

Crosswalk Test Suite

Crosswalk test suite is

• a centralized test repository for the Crosswalk Project,
• the code base for Tizen 3.0 Compliance Tests for IVI (TCT),
• an upstream project for upcoming Tizen 3.0 Compliance Tests for other profiles, e.g. Mobile, Wearable, TV profiles.

Here only care about these related to TCT.

Crosswalk Tests Used for Web TCT

• behavior
• doc
• tools
• webapi
• wrt

Differentiation

• Based on Crosswalk web runtime rather than Tizen legacy runtime. tools/crosswalk/
• Centralized resources, e.g. testharness.js, webrunner, tests.xml xsl. tools/resources/
• Python based packaging and (un)installing scripts, toots/build/, with suite.json support.
• More tools, e.g. tools/coverage/, tools/xmlsimplifier/, etc.
• Markdown based documentation rather than PDF.
• More web APIs supported with webapi/webapi-* suites.

Web Test Development

Role and Responsibility

• Owner: be accountable for test development and quality, review submission from any contributor, and own bug fixings.
• Contributor: can submit tests/patches for any code which will be reviewed and merged by project owner.

Web Test Development

Documentation

• Follow the Web Test Suite Developer Guide to create new tests and test suites.
• Follow W3C documentation at Test the Web Forward to set up environment, submit and review tests.

Web Test Development

Commit Message

Capitalized, short (50 chars or less) summary

More detailed explanatory text, if necessary.  Wrap it to about 72
characters or so.  In some contexts, the first line is treated as the
subject of an email and the rest of the text as the body.  The blank
line separating the summary from the body is critical (unless you omit
the body entirely); tools like rebase can get confused if you run the
two together.

Impacted tests: new 0, update 5, delete 0
Unit test platform: Tizen Wearable <version>
Unit test result summary: pass 205, fail 0, block 0
      

See details at my post why commit message is so important.

Web TCT Build and Release

• Follow Web Test Suite Packaging Guide to build test suites.
• Follow Tool Packaging Guide to an all-in-one package for TCT released to download.tizen.org/tct/.