Graham Perrin's List: code 49767 list 002

• 

   

  Again, the centre requirement is the document packaging mechanism, enhanced to allow unpackaging and web service. The following diagram, though almost uselessly general, lists various components that are entirely common to both websites and XML-in-ZIP compound documents: why are we squabbling in such minute detail on solutions to XML-in-ZIP formats that do nothing to address this tectonic rift between documents and websites? Because we allow our expectations to be dictated by menu bars: something that currently has a Save As.. or Open menu item is real...
• Is ODF the new RTF or the new .DOC? Can it be both? Do we need either?
• <!-- RSPEAK_START -->By Rick Jelliffe
  October 21, 2008
• Jon Bosak
• recently wrote an article concerning the position of ODF, Open XML and PDF
• Jon's public writings are rare, well-considered and always of interest.
• Jon has been exemplary
• he has a side, he does not take sides
• a potted history of the document format landscape over last 25 years

• One Thousand Islands

    

  At the start of the 1990s, the document landscape looked a little like this:

    

  In the above diagram, each of the bubbles represents numerous formats

• One Thousand Islands

    

  At the start of the 1990s, the document landscape looked a little like this:

    

  In the above diagram, each of the bubbles represents numerous formats

• Consolidation

    

  By the mid 1990s, this had changed, in a period of massive consolidation as the web, Microsoft and Adobe starved rivals of oxygen

• Consolidation

    

  By the mid 1990s, this had changed, in a period of massive consolidation as the web, Microsoft and Adobe starved rivals of oxygen
• Standard Generalized Markup Language SGML consolidated many divergent capabilities of existing markup languages
• • 
    
    

  One fairly unheralded aspect of this was the rise in the publishing community, including high-end vendors, of an awareness of the benefits of co-operation and standardization of different kinds. SGML was born out of such cooperation at ISO
• • 
    
    

  One fairly unheralded aspect of this was the rise in the publishing community, including high-end vendors, of an awareness of the benefits of co-operation and standardization of different kinds.

• Realignment

    

  By the end of the 1990s, there had been a realignment of the rationales for the formats: in particular, HTML had moved from the dumb end of town into having its own compelling rationale

• Realignment

    

  By the end of the 1990s, there had been a realignment of the rationales for the formats: in particular, HTML had moved from the dumb end of town into having its own compelling rationale

• 

     

  Of these formats, Microsoft controlled .DOC, .RTF, and was actively involved at W3C in XML and HTML development. PDF was controlled by Adobe.

• 

     

  Of these formats, Microsoft controlled .DOC, .RTF, and was actively involved at W3C in XML and HTML development. PDF was controlled by Adobe.

• Digestion/Stagnation

    

  The early part of the 2000s was a period of extraordinary growth for applications of XML, matched by stagnation in the other areas.

• Digestion/Stagnation

    

  The early part of the 2000s was a period of extraordinary growth for applications of XML, matched by stagnation in the other areas.

    

  The dynamism of the open standards was in marked contrast to the stagnation of the corporate-controlled formats.
• Microsoft adopted a policy of containment
• to prevent HTML from taking market share
• notoriously with an unwillingness to support CSS thoroughly
• to prevent HTML from taking market share from .DOC or even .RTF systems:
• notoriously with an unwillingness to support CSS thoroughly
• either import or export to a foreign format, but not both
• provide either import or export to a foreign format, but not both
• remain non-standard.
• animation areas (e.g. Flash)

• Standardization

    

  In the late noughties, a period of standardization broke through, with all the major formats being taken by their stakeholders through industry consortia and on to ISO.

• Standardization

    

  In the late noughties, a period of standardization broke through, with all the major formats being taken by their stakeholders through industry consortia and on to ISO.

• The enormous controversy over the standardization of OOXML comes largely down to, in my opinion, two different views of the emerging landscape.

    

• The enormous controversy over the standardization of OOXML comes largely down to, in my opinion, two different views of the emerging landscape.

    

• Business as Usual

    

  The first landscape (lets call it business as usual) sees ODF as superseding RTF
• In other words, the primary role of RTF is for medium quality
• OOXML distinguishes itself from ODF by being DOC-in-XML-in-ZIP, with an emphasis on fidelity to a particular rich feature set rather than RTF-in-XML-in-ZIP, with an emphasis on interchange between systems with a basic feature set.  

• The Bermuda Triangle

    

  The second landscape (lets call it the Bermuda triangle) sees ODF as replacing .DOC, with RTF becoming swallowed by both ODF and HTML. I think this is the landscape anticipated by people who don't see much value in IS29500 or multiple targeted fornats.  
• This second approach has a lot to commend it: HTML really does supersede RTF as being the interchange format for simple rich text.

• Getting There

    

  My own view is that the Bermuda triangle would be preferable, and it may be where we end up,
• standardize business as usual
• but that the only way there is to first
• hope of moving forward, but only if ODF, HTML and even PDF are significant improved, with proper vendor support:
• HTML needs to be developed further.
• ODF (2.0) needs to be re-developed with a view to better supporting alternatives and graceful degradation.
• PDF has three challengers, none particularly serious at the moment:
• Microsoft has its XPS format (another XML-in-ZIP),
• there is a mooted Chinese push for the OASIS UOML (Unstructured Objects Markup Language: a simple modern PDL) to be standardized.
• Adobe has some similar technology (MARS)
• various efforts to standardize profiles of PDF
• the challenge is for PDF with embedded SVG, and for ODF (2.0) with embedded PDF alternatives, and so on.
• if we are to move from business as usual to the Bermuda triangle?
• So what are common threads required
• Support for a common next-generation document packaging (XML-in-ZIP) format to be assertively supported by W3C, ODF (2.0) and son-of-PDF.
• Ubquitous support for unvarnished SVG Tiny, MathML, and XForms by W3C, ODF (2.0) and son-of-PDF.
• A co-operative carve up of RTF's functionality between HTML and ODF 2.0.
• A real effort to add support for features in IS29500 and UOF etc to ODF 2.0,
• with an emphasis on graceful degradation
• More support for richer XML media and specialist content types.
• down the path towards the Bermuda triangle.
• Carrots and sticks to entice and direct Microsoft
• these are just for word processing

  • !
  Add Sticky Note
• The Real Present and the Real Future
• the looming challenge for document standards is
• in integrating the packaged world of documents with the fragmented world of web resources.

  • This is very thought-provoking.
  Add Sticky Note
• Documents that can be websites.
• Page description files with external resources.
• Arbitrarily nestable documents.
• Web applications that are single files and editable as applications by word processors
• in this vision, Open Office or MS Word (or perhaps MS Powerpoint) should be able to open up and edit
• or their successors.

  • I imagine a gradual increase in awareness of potential benefits of WebArchive &mdash; within and without WebKit.
  Add Sticky Note
• a .WAR web archive file
• almost uselessly general

  • :-)
  Add Sticky Note
• not an outlandish vision: there is piecemeal support for many parts already
• support plurality
• We need to be able to have our cakes and eat them.

  • +1
  Add Sticky Note
• FOSS people often are concerned with pioneering individual efforts
• I don't have high expectations of leadership from many other groups.
• rather than collective efforts.
• the large vendors or open source projects which straddle both documents and the web, such as Sun, IBM, and Microsoft, are still (I think) focussed on the dubious delights of business as usual and the Bermuda triangle

• Towards Web-able Document Formats

    

  So what is there that we can do to nudge us in the direction of this large-scale interoperability?
• Participate in the most congenial standards body to your interests,
• encourage them to support plurality (modularity, alternatives, feature harmonization with each others standards, graceful degradation) and the dissolution of the great website/document divide.
• foster good reasons (carrots and sticks) so that the large developers see value in this kind of enabling standard.
• make browser plugins that locate resources in an unzipped XML-in-ZIP document delivered from a vanilla web server
• enhance the existing XML-in-ZIP formats to cope with web formats (ODF and OOXML with CSS and SVG Tiny, for example) better
• round-trip
• lots to do —unblock the plumbing:

87 more annotations...

• A fabulously written, very readable article with great insight. - Graham Perrin on 2009-07-29
• HIghlights are very disorderly (sorry) so be sure to read them as overlays to the original page. - Graham Perrin on 2009-07-29
• Quoting Gary Edwards, from his group on Diigo:
  
  "The Web is the future … HTML+ (HTML5, CSS3, SVG/Canvas, JavaScript)." - Graham Perrin on 2009-07-29
• Highlights here are not only disorderly, I now realise that many of my highlights are lost :-( - Graham Perrin on 2009-07-29

• Standards-Based Interoperability
• 05 June 09
• Interoperability without Standards

• 

  First, let’s consider how software interoperability works when it is not standards-based.

  Consider the various ways that four applications can share data, as shown in the diagram to the right.  There are six connections between these four applications, and each connection can be traversed in either direction, so there are 12 total types of interoperability involved.

• As the number of applications increases, this complexity grows rapidly.  Double the number of applications to 8 total, and there will be 56 types of interoperability between them:

  
• All complex software has bugs, and some bugs can present significant challenges to interoperability.  Let’s consider the case that 3 of the 4 applications have bugs that affect interoperability, as shown in the diagram to the right.

• Here’s where those workarounds will need to be implemented:

  

  Note the complexity of this diagram.
• In the real world, interoperability is almost never achieved in this way.  Standards-based interoperability is much better approach for everyone involved,
• whether that standard is an open one such as ODF (IS26300)
• or a de-facto standard set by one popular implementation.
• or Open XML (IS29500)
• The core premise of open standards-based interoperability is this:
• each application implements the published standard as written, and this provides a baseline for delivering interoperability.
• the existence of a standard addresses many of the issues involved, and the other issues can be addressed
• through standards maintenance, transparency of implementation details, and collaborative interoperability testing.

  • Issues relating to CalDAV are well addressed in these ways.
  Add Sticky Note

• In the standards-based scenario, the standard itself is the central mechanism for enabling interoperability between implementations:

  

  This diagram is much simpler

• How this all applies to Office 2007 SP2

  I covered last summer the set of guiding principles that we used to guide the work we did to support ODF in Office 2007 SP2.
• applied in a specific order
• I’d like to revisit the top two guiding principles
• Guiding Principle #1: Adhere to the ODF 1.1 Standard
• Guiding Principle #2: Be Predictable
• Being predictable is also known as the principle of least astonishment.

• What about Bugs and Deviations?

  

  Of course, the existence of a published standard doesn’t prevent interoperability bugs from occurring.
• deviations from the requirements
• different interpretations
• Our approach to the transparency issue has been to document the details of our implementation through published implementer notes.

• Interoperability Testing

  The final piece of the puzzle is hands-on testing
• What else would you like to know about how Office approaches document format interoperability?
• a standard (evolved and improved as reality demands) is the proper foundation for resolving interoperabilty
• there is no question that users of other products are massively surprised by
• (1) their spreadsheets having their formulas lost when interchanged with Excel 2007
• (2) not being able to handle the formulase received in Excel 2007's ODF output.
• I am creating my own fantasy about the state of affairs

  • :-)
  Add Sticky Note
• it is far too early to declare it to be unsuccessful
• I cannot fault the Microsoft approach as incorrect
• I was at the year-ago DII meeting where the guiding principles were announced and their application to spreadsheet formulas described.  I applauded the principles and understood the reasoning for formulas.
• How this would impact various groups of users and non-users (who still want to interoperate) of Office 2007 did not surface in my consciousness.
• there is NO published standard for ODF spreadsheet formulas yet.
• Nor is there any de-facto standard that everyone agrees on.
• the “spaghetti diagram" method, with all of the complexity and risk of bugs that entails
• No implementer we know of has attempted that
• In the case of spreadsheet formulas, help is on the way -- OpenFormula is under development for use with ODF 1.2.
• I’d like to keep this thread on-topic
• I appreciate the post, very good
• Visually I would rather frame it in terms of convergence, a spiral.
• I doubt someone would ever find a magic bullet to interoperability
• and user satisfaction.
• New Comments to this post are disabled

  • Hurrah!
  Add Sticky Note
• © 2009 Microsoft Corporation

47 more annotations...

• Diagrams here are eye-catching. - Graham Perrin on 2009-07-28

• Don't forget about HTML
• February 25, 2007
• HTML’s potential and actual suitability for much document interchange
• HTML is the format to consider first
• validated, standards compliant XHTML in particular
• PDF at the other end (full page fidility but read-only)
• HTML at one end (simple WP documents)
• HTML, ODF, OOXML, PDF

• W3C versus ISO

   
• overlap at all brings nothing but misery and bloat.
• Lie adopts an extreme view towards overlap of standards:
• The next dodgy detail is to make blanket comparisons between HTML and ODF/OOXML.
• ODF and OOXML deal with many issues that HTML/CSS simply does not.
• the W3C argument might be to say that every part should have a URL
• a strange theory that MS wants ODF and OOXML to both fail
• being pro-ODF does not mean you have have to be anti-OOXML
• HTML is the format ｏｆ choice for interchange of simple documents
• ODF will evolve to be the format of choice for more complicated documents
• OOXML is the format of choice for full-fidelity dumps from MS Office
• PDF is the format of choice for non-editable page-faithful documents
• all have overlap
• we need to to encourage a rich library of standard technologies,
• free,
• widely deployed,
• explicit,
• awareness of when each is appropriate
• unencumbered,
• using ISO Schematron
• an adequate set of profiles and profile validators
• Plurality

28 more annotations...

• Microsoft is still hiding the migration tables
• Those tables simply are not provided in the specifications, despite the stated goal that Microsoft is doing so openly.

  • I wonder whether this remains true.
  Add Sticky Note
• OOXML is custom fit for Microsoft’s products only

1 more annotation...