XML and JSON Open Message Formats

Lecture 7 – Open Message Formats

• Covers XML and JSON for network object transfer via serialization.
• Focuses on XML and JSON since they are message formats widely used in RPC methods for data transfer.

What is XML?

• XML is the eXtensible Markup Language.
• It structures text by encasing it with textual tags, describing the meaning of text content.
• The markup language defines valid tags and structure without dictating their use.

Use of ASCII vs XML

• ASCII can store data for retrieval across platforms, however, the meaning is not clear. An example is "10, Nimal, 56"

• XML is self-describing and platform-neutral, like:

  10 Nimal 56

XML vs. HTML

• HTML defines valid tags and structure for web pages.
• HTML has a Fairly inconsistent standard; tags don't always need closing (e.g., <p>).
• XML can be used to mark up text
• It's a Meta Language: a language used to define other languages.
• XML itself has no tags: XML tags are not predefined, requiring you to 'invent' your own tags for a new language.
• HTML is about displaying information, while XML is about describing information.

Why Use XML?

• XML is a markup language for documents with structured data
• It describes data in human-readable, structured form.
• Structured information can contain both content, as well as information that defines the content.
• It's a cross-platform, software, and hardware-independent tool for transmitting information/data.

Where To Use XML

• XML finds its use in e-commerce (B2B), content management, web services, distributed computing, peer-to-peer (P2P), and the Semantic Web.
• However, it can need a lot of space
• 3-20 times larger compared to binary format.
• XML plays a role for the future of the Web, similar to HTML.
• It's a common tool for data manipulation and data transmission.

XML as a Universal Language

• XML is a "use everywhere" data specification.

XML Document Sample

• XML doc:

courselist
   course
      lecturer
         first name
         last name
      title
      code

Benefits of XML

• XML follows an Open W3C standard
• Data is represented across heterogeneous environments
• Supports cross-platform use
• High degree of interoperability
• XML dictates strict rules
  • Syntax
  • Structure
  • Case sensitivity

XML Syntax Composition

• An XML declaration (e.g., <?xml version = "1.0" encoding="ISO-8859-1"?>) is included though optional
• There should be identification of the XML version, namespace, and encoding schema to which the document conforms
• XML also contains Markup types
  • Elements (and attributes)
  • Entity references
  • Comments (<!-- what ever you want to say -->)
  • Processing instructions (<?instruction options?>)
• Content

Elements and Attributes

• Elements are tags
  • e.g. <course> </course>
• Attributes are
  • <course level = "undergraduate">
• Attributes provide information about an element
• An attribute value is required, optional, or fixed, with a default value possible.
• Attributes can often be replaced by nested elements.
• All values must be quoted, even for numbers, as in XHTML.

Element and Attribute Naming

• Elements and attributes can be named in nearly any manner.
• These are descriptive and not confusing with no length limit
• Are case sensitive, and can use the underscore (_).
• No names are reserved for XML; namespaces resolve naming conflicts.
• Names cannot start with a number, reserved punctuation, "XML" or "Xml".
• Contain not spaces, colons (:), greater-than (>), or less-than (<)
• Best practice to not use the hyphen (-) or period (.)

XML root element

• Each element (lecturer, title, and code) is rooted in “course”
• Each element (lecturer, title, and code) are children

XML Content

• The text lies within elements
• This is how documents are structured
• Content may consist of any data as long as it does not get confused with XML metadata
• Use Instructions use entity references for special characters!
• Each XML document must have one root element

Well-Formed XML

• XML must be formatted correctly
• All tags must be closed

XML Namespaces

• XML tags are user defined.
• Elements with the name can lead to naming confllicts
• XML applications use XML-based languages defined by various parties may have same element names.
• Online furniture is good example. Vendors might have similar tables
• To tackle table issues, use namespaces

XML namespace naming

<h:table>
<h:tr>
<h:td>Apples</h:td>
<h:td>Bananas</h:td>
</h:tr>
</h:table>

<f:table>
<f:name>African Coffee Table</f:name>
<f:width>80</f:width>
<f:length>120</f:length>
</f:table>

• There will be no conflict because elements have different names

XML Namespace Properties

• To use XML prefixes, a namespace must for the prefix itself must be defined
• This is defined by the xmlns attribute in the start tag
• This has the following syntax: xmlns:prefix="URI".
• Each namespace is identified by a URI.

Namespaces: Declaration examples

xmlns: bk = "http://www.example.com/bookinfo/"
xmlns: bk = "urn:mybookstuff.org:bookinfo"
xmlns: bk = "http://www.example.com/bookinfo/"

Multiple XML Namespace

<furniture xmlns = "http://www.itee.uq.edu.au/~zxf/furniture"
xmlns:ikea = "http://www.ikea.com/names"
xmlns:html="http://www.w3.org/TR_REC-html40">

• The code above shows the element can have 3 namespaces
  • One default, others for IKEA and HTML

Default and Scope

<BOOK xmlns="www.bookstuff.org/bookinfo">
<TITLE>All About XML</TITLE>
<AUTHOR>Joe Developer</AUTHOR>
<PUBLISHER xmlns="urn:publishers:publinfo">
<NAME>Microsoft Press</NAME>
</PUBLISHER>
</BOOK>

• An XML namespace without a prefix becomes the default for sub-elements
• Elements without a prefix are part of the default namespace
• Unqualified elements belong to the inner-most default namespace
  • In the example BOOK, TITLE, and AUTHOR are part of book’s namespace
  • However, PUBLISHER and NAME fall under the publisher namespace

Sample XML document with two namespaces

<courselist xmlns:c="http://www.university.com/courses"
xmlns:l="http://www.university.com/lecturers">
    <c:course id="001">
        <c:name>SPDII</c:name>
        <l:lecturers>
            <l:lecturer>
                <l:fname>Sheron</l:fname>
                <l.Iname>Dinushka</I.Iname>
            </l:lecturer>           
            <l:lecturer>
                <l:name>Nishani</l:name>
                <l:Iname>Ranpatabandi</l:Iname>
            </l:lecturer>
        </l:lecturers>
    </c:course>
</courselist>

XML Schema (XSD)

• XML Schema Definition language
• Description of the structure of an XML document
  • XSD is an XML file following a fixed standard
  • It Replaces the less-flexible DTD (Document Type Definition)
• Parsers use XD to check XML files; this is for validation as opposed to using well-formed structures:
  • With well-formed schemas: the general XML syntax rules are followed
    • e.g. Tags have end-tags, nesting is correct
  • Validity: The document follows XSD’s semantics
    • e.g. the tags/attributes used are defined in XSD, which means the structure is OK

What does XXSD Define?

• Contains what elements can appear in a document
• Dictates what attributes can appear
• Which elements are children
• Order of children
• How many children
• If the element can be empty, or include text
• What elements are to have default and fixed values

Note.xml Example

<?xml version="1.0"?>
<note
xmlns="http://www.w3schools.com/note"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.w3schools.com note.xsd">
    <to>Tove</to>
    <from>Jani</from>
    <heading>Reminder</heading>
    <body>Don't forget me this weekend!</body>
</note>

XSD for Note.xml Sample

<?xml version="1.0"?>
<xs:schema xmlns:xs= "http://www.w3.org/2001/XMLSchema""
targetNamespace=“http://www.w3schools.com/note”
xmlns="http://www.w3schools.com"
elementFormDefault= "qualified">
    <xs:element name="note">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="to" type="xs:string"/>
                <xs:element name="from" type="xs:string"/>
                <xs:element name="heading" type="xs:string"/>
                <xs:element name="body" type="xs:string"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>
</xs:schema>

Schema Element

• The element is the root for every XML Schema.
• The element can have attributes.

  xmlns:xs= http://www.w3.org/2001/XMLSchema
  

  -This states that the elements and data types are from “http://www.w3.org/2001/XMLSchema", which should be prefixed with xs

More on element

targetNamespace="http://www.w3schools.com/note"

• This states that the elements defined by the schema (note, to, from, heading, body.) are assigned to the target namespace.

xmlns=http://www.w3schools.com

• States the defualt namespace

elementFormDefault="qualified"

• Elements used by the XML instance document must be namespace qualified if declared in this schema

Referencing XSD

xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance
xsi:schemaLocation="http://www.w3schools.com note.xsd">

• This can be used to link the XML docs with the schemaLocation attribute
• Since the schemaLocation attribute is in the "http://www.w3.org/2001/XMLSchema-instance"" namespace, this MUST be declared and it MUST be mapped to xsi as a prefix

Complex Types in XSD

• A complex element is an XML element that contains other elements and/or attributes.
  • Example

  John
  Smith

• The "employee" element can be declared directly by naming the element, like this:

  
      
           
           ""/>
          

Simple elements in XSD

• Element has this syntax:
  • <xs:element name="xxx" type="yyy"/>
• xxx is for element name, yyy is for type
• XML has an array of built-in types, including:
  • string
  • decimal
  • integer
  • boolean
  • date
  • time

Defining data types example:

Smith
36
1970-03-27

• Corresponding simple element definitions:

  
        type="xs:string"/>

        type="xs:integer"/>

        type="xs:date"/>

Attributes in XSD

• Has this syntax:
  • <xs:attribute name="xxx" type="yyy"/>
• xxx to attribute name, and yyy to specify data type
• Simple elements DO NOT have attributes!

Attribute example

Smith

Structuring Data with XML for Stocks Example

<?xml version="1.0"?>
<?xml-stylesheet type="text/css" href=”StockPortfolio.css"?>
<StockPortfolio xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-Instance"
xsi:schemaLocation="http://www.cs.curtin.edu.au/spd361
StockPortfolio.xsd">
  <Stocks>
    <StockPurchase>
      <Ticker>GOOG</Ticker>
      <PurchasePrice>330.06</PurchasePrice>
      <NumPurchased>30</NumPurchased>
    </StockPurchase>
    <StockPurchase>
      <Ticker>MSFT</Ticker>
      <PurchasePrice>17.21</PurchasePrice>
      <NumPurchased>580</NumPurchased>
    </StockPurchase>
  </Stocks>
</StockPortfolio >

Stock with XML Schema

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="StockPortfolio">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="Stocks"/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  <xs:element name="Stocks">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="StockPurchase" minOccurs="0"
        maxOccurs="unbounded"/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  </xs:schema>

More on Stock with XML Schema

<xs:element name="StockPurchase">
   <xs:complexType>
   <xs:sequence>
     <xs:element name="Ticker"/>
     <xs:element name="PurchasePrice"/>
     <xs:element name="NumPurchased"/>
   </xs:sequence>
   </xs:complexType>
  </xs:element>
  <xs:element name="Ticker" type="xs:string"/>
  <xs:element name="NumPurchased" type="xs:int"/>
  <xs:element name="PurchasePrice" type="xs:double"/>
</xs:schema>

Exercise: Write the XSD for the following XML

<?xml version = "1.0" encoding="ISO-8859-1"?>
<courselist>
    <course>
        <lecturer>
            <firstname> H.T. </firstname>
            <lastname> Shen </lastname>
        </lecturer>
        <title>SOA</title>
        <code>INFS3204</code>
    </course>
</courselist>

Automating XSD Creation

• Although this example was created by hand
• Tools existing make is automatic now
  • .NET: xsd.exe
  • Java: JAXB (Java Architecture for XML Binding) xjc.exe

Displaying XML

• Displaying XML will require XSLT (extensible stylesheet language transformations)
• Which is used to format XML documents
• XML contains the content
• XSLT contains the formatting
• XML + XSLT = HTML

Text vs Binary Formats

• "binary” format means the data concepts get mapped to the native Entities for the computer system; that means bytes
• With the “text” format, the data concepts get mapped to characters which must be translated from or to computer entities
• Strings can be needed for arithmetic

XML vs Binary

XML Advantages Over Binary

• XML is self describing and also platform neutral

Binary Advantages Over XML

• Size: Smaller data size, minimal overhead size (ie: no tags)
• Speed: No need to convert between text and numeric data
• Simpler: Parsing XML is a complicated exercise
• And XML can be too flexible: multiple ways to do same thing

Why Text Over Binary

• Encoded well for the network. Protocols such as data transfer are traditionally binary
  • Java is an example
• XML formats data transmission entirely in text: XML tags
• Such as SOAP. Basis for web services

What is Serialization?

• Serialization is converting and encoding an object into a data stream
• This includes aggregated or contained objects
• Used for disk saving, or transfer over a network
• Deserialization in conversion back from disk
• Types Binary XML (single textual conversion)

Overview of JSON

• What is JSON?
• How does it compare with XML?
• Discuss syntax
• All data types
• Usage

JSON is...

• A lightweight data-interchange, text-based format
• Language independent
• Based on Javascript, therefore easy to understand

JSON is Not...

• Overly Complex
• a “document” format
• a markup language
• a programming language

Why Use JSON

• Has straght foward syntax
• IsEasy to create and manipulate
• Can be parsed by JS eval() function
• Supported by most JS frameworks

Does JSON compare with XML?

• JSON and XML have similaries

• Both are hierachical and good for data

• JSON is lighter and faster

• Unlike XML, JSON doesn't have

  • Namespaces
  • Inherit validation

JSON Object Syntax

• These are unordered
• Have name/value pairs
• Follow these syntax requirements
  • Begins with { (left brace)
  • Ends with } (right brace)
  • Each name is followed by : (colon)
  • Name/value pairs are separated by , (comma)

JSON Example:

var employeeData = {
 "employee_id": 1234567,
 "name": "Jeff Fox",
 "hire_date": "1/1/2013",
 "location": "Norwalk, CT",
 "consultant": false
};

Arrays in JSON

• Ordered collections of values.
• Begins with [ (left bracket)
• Ends with ] (right bracket).
• Name/value pairs are separated by , (comma)

Boaleans and NULL

• A boolean is either true or false
• NuLL is something that doesn't represent a value

Types of Data Objects and Arrays

• Objects: Unordered key/value pairs wrapped in { }
• Arrays: Ordered key/value pairs wrapped in [ ]
• You can nest objects (for supporting complex object)

Open Data Formats

• YAML is an example of open data, or markup languages
• Has these links to know more
  • https://blog.stackpath.com/yaml/
  • https://octopus.com/blog/state-of-config-file-formats
• Often used to define REST APIs. Defines component/service interfaces and key usage.

Data Formats: Summary

• Binary data formats are efficient to send and receive messages
• However due TEXT datatypes, they are more open
• XML therefore had richer features
• However:
  • JSON is more lightweight
  • "You can also use YAML