Group annotations on this page
-
Most (quality) specifications provide clear instructions using
those magic words SHALL, SHALL NOT, and MAY where those words have
a defined meaning for an implementor. Paragraphs are clearly
identified as either normative or informative. That way an
implementor knows what they must and may implement to claim
conformance against a specification. This approach has been well
established over time as a sensible way for spec writers and
implementors to work -
Most (quality) specifications provide clear instructions using
those magic words SHALL, SHALL NOT, and MAY where those words have
a defined meaning for an implementor. Paragraphs are clearly
identified as either normative or informative. That way an
implementor knows what they must and may implement to claim
conformance against a specification. This approach has been well
established over time as a sensible way for spec writers and
implementors to work
That is the way quality specifications are written. For
example, ISO/IEC's JTC 1
Directives (link to PDF) requires that international standards
designed for interoperability "specify clearly and unambiguously
the conformity requirements that are essential to achieve the
interoperability."
With that clarity, conformance is testable and can provide
confidence of interoperability. A suite of tests may be developed
and applied to an implementation to determine which tests pass,
which fail, and hence arrive at an objective pronouncement on
conformance of an implementation against the entirety of the
specification. -
In a quality specification, it should be feasible to select a
normative paragraph, identify a conformance test for it, and make
a clear statement that this test proves that an implementation
meets (or fails to meet) that requirement. Call it a test plan:
define the tests (test specification), define the expected set of
results, and define what constitutes a "pass" of each test that
establishes conformance. The plan then provides the matrix of test
spec against requirement. Simple. -
Rob Weir of IBM chaired (apology for the misuse of that last
word) the formation list and then simply announced what the
charter would be rather than seeking consensus among the list
participants. As part of this process before that charter was
produced and while I still naively believed that consensus was a
goal, I sat down with ODF 1.1 and did a paragraph-by-paragraph
review for testability. The numbers were quite revealing. I
completely reviewed only the first four major sections and found
very few clear requirements.
The majority were mere statements with no normative language
used to identify what was required or optional. Implementors would
have to make their own interpretation. -
It's ironic that the chair viewed as good news the fact that
there were far fewer testable paragraphs than he had predicted. But
his prediction of 10,000 test cases is probably far closer to how
many testable paragraphs there should be; my counts were actually
bad news. -
All of the above leads to the interesting question of just how
the chair expects to accomplish much that is useful in regard to ODF
conformance testing before the specification is amended to tighten
up the language and add clear requirements. The syntax conformity is
already handled by validation against the schema, but the semantics
are woefully under-specified. -
Summary: ODF 1.1 isn't verifiable as a specification. From a
fairly cursory review of the latest draft, ODF 1.2 will follow the
same path. With OASIS now being more demanding regarding conformance
requirements on every specification and with ISO/IEC taking a closer
interest in liaison with the ODF TC, I find it hard to see how the
ODF TC co-chairs can maintain this view toward verification.
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.