Creating XML file from the General JSON

I have written a Java class with following methods to create XML from the JSON generated by the UI. Here I have used the GSON library, added it as a dependency to my project .

• parse(s)
• artifactDetailstoXML(artifactDetails)
• UItoXML(ui)
• contentTtoXML(content)
• relationshiptoXML(relationship)
• writeToFile(name.getAsString(),XML)

parse(s) method

Is used to create JSON model from JSON.

JsonParser jsonParser = new JsonParser();

JsonElement rxtJsonElement = jsonParser.parse(jsonContentAsString);

JsonObject rxtJsonArtifactTypeObject = rxtJsonElement.getAsJsonObject().getAsJsonObject();

Then passed the JSON file as String content(jsonContentAsString) to parse method.

JsonObject is returned from parse method and that object is used to retrieve the hierarchical information that is needed to create the xml file.

artifactDetailstoXML(artifactDetails) method

Assumption was made that all mandatory fields are filled and only checked if user has input data to namespace, namespaceAttribute and lifecycle fields.

UitoXML(ui) method

Checked if the there are more that one column data exists and if it does JsonArray was created accordingly.

Separate JsonObjects are created for artifactDetails, ui, content and relationship and passed to different methods to create XML content accordingly. Advantage in creating separate XML sections is that it gives flexibility to make changes easily in separate methods.

Then the xml contents returned from each of the methods are added together and created the whole XML content and passed to writeToFile() method to be written to a .rxt file.

JsonObjects, JsonElements, JsonArrays were used to create XML.

JSON generation from UI is halfway done. But I am in difficulty in generating JSON for dynamic UI part and now working on it.

Update after mid review

Last week I met with WSO2 Governance registry team and I demonstrated the current progress of the Jaggery Application I have developed. We, my mentor and some experienced people in the team then discussed on the project scope and my approach so far and beyond.At the end of the session I was able to clarify my doubts on the problems I faced so far and on future tasks.

As I mentioned in my last update several issues were recorded in the existing RXT , XML model, so there were difficulties in rendering the UI (in ES) only with the information it is providing. The necessity of keeping other meta data along with the xml model was quiet apparent. To overcome this drawback , it was decided to generate a JSON along with sufficient meta data, such that UI rendering can be done without much effort. And then XML which suits the existing model is generated from that JSON to be sent to the stub.

Also I was able to clarify some doubts regarding the RXT definition whether certain data in rxt. sxd, which were apparent to be unused should be considered in the design or not. i. e date,calendar, href,menu,column. Discussions further went on regarding the capability of implementing nested rest support where user can associate rxt to each other on the fly. So that rxt s can be associated automatically.

Ability to add multiple tables and multiple lists in content, delete preferred rows in Relationship and UI sections are discussed in the meeting too. As enhancements it was discussed to consider the ability to populate a browse button to find relationship source /target.

These days I am working on creating JSON from the UI and retrieving the XML from that to be sent to the server side. I am writing a Java class to create XML from JSON, and JSON will be generated from UI in JavaScript.

Herewith I have drawn the updated overall architecture.

Project: Implement Registry Extension (RXT) 2.0 + Associated UI support - Updates and Notes

This is an update of  my project so far in GSOC 2014. There was a slight change in the project scope since the JSON support for RXT was temporarily removed and the scope was narrowed down to creating a Jaggery app which could support creating a new artifact type with an intuitive drag and drop UI.

I had a meeting with my mentor and few others in WSO2 last week and below are the facts which were discussed.

• Finalized the requirement of a Jaggery app to create a new artifact type which could be installed to either Greg or ES via Management Console
• RXT <content> should be generated using drag and drop components and user shall be able to change the fields easily by dragging components here and there.
• The RXT XML should be generated in the client side using JS as well as some metadata should be kept with UI fields in order for ES to generate its ‘add metadata’ model.
• This model should also facilitate creating a JSON out of the UI, at some point of time.
• Once the XML is generated, a backend call will be made to Registry to add the RXT via Jaggery.

Current Progress

In the Process of finding a suitable plugin for drag and drop functionality, I looked upon Gridster.js [1] was used in UES and is very flexible in generating UIs. But it seems to be bit difficult to use it in this purpose, since it needed a lot of fine tuning to be able to cater this requirement. Then I found Bootsnipp Form Builder [2] which seems to be bit similar to the one which is required under MIT license and built on bootstrap. So decided to go on with it.

I was able to create a simple UI, create a Jaggery app and host it in Greg/ES and test the functionality. Below is a sample UI which has only the basic functionality to create a RXT.

                                           

I was told several issues the existing RXT, XML model has and the difficulties in rendering the UI (in ES) only with the information it provides and necessity of keeping other metadata along with the xml model. So I’ working on a way to find out the best possible way to bridge the gap between the RXT xml and the UI it is generating in ES side.

Next Steps

• Write a JS script to generate the RXT xml from the form data/HTML
• Keep other metadata which are not going in to the xml in a JSON model (tentative)
• Call backend Registry client/stub and send the RXT to registry.
• Try to load RXT xml data to the UI back when editing a RXT artifact.

Github Repo : https://github.com/ireshapm0/ArtifactBuilder

[1] http://gridster.net/

[2] http://bootsnipp.com/forms

JSON Parsing and Validation

Step I - JSON Parsing

To parse JSON data we can use  GSON library.

http://google-gson.googlecode.com/svn/trunk/gson/docs/javadocs/com/google/gson/JsonParser.html

When given the json file as a String to parse method above, it will return the parsed JsonObject as tree.

public static JsonObject parse(String jsonString) {

JsonParser jsonParser = new JsonParser();

JsonElement rxtJsonElement = jsonParser.parse(jsonString);

JsonObject rxtJsonArtifactTypeObject = rxtJsonElement.getAsJsonObject().getAsJsonObject("artifactType");

return rxtJsonArtifactTypeObject;

}

Step II - JSON data format Validation

Develop Shema

JSON Schema is a tool for validating the structure of JSON data. When we consider data format we need to know what does data elements mean and what the valid inputs for these elements are. To ensure we have obtained the required data we write a schema.

I.Manually write Schema for JSON:

There are certain keywords that are used to develop a Schema for any sort of json data.

They can be found at http://tools.ietf.org/html/draft-zyp-json-schema-03 .

Most frequently used key words are:

Ø  type

Ø  properties

Ø  items

Ø  required

Ø  maxItems

JSON is built on two structures:

Ø  A collection of name/value pairs commonly realized as “objects”.

Ø  An ordered list of values commonly realized as “ arrays”.

This defines the type keyword in a Schema.

properties are defined to validate the attributes in an object.

Within an array the list of items are defined by items keyword.

Whether a particular object array or property and item is a must for it to validate is defined by required keyword. 

MaxItems defines the maximum number of elements in array type of object type.

Similarly you can find how any of this keyword defines the validity of the JSON data against the Schema we develop.

Example:

JSON data to be validated:

{

"address":{

"streetAddress": "21 2nd Street",

"city":"New York"

},

"phoneNumber":

[

{

"type":"home",

"number":"212 555-1234"

}

]

}

Schema developed:

{

"type":"object",

"required":false,

"properties":{

"address": {

"type":"object",

"required":false,

"properties":{

"city": {

"type":"string",

"required":false

},

"streetAddress": {

"type":"string",

"required":false

}

}

},

"phoneNumber": {

"type":"array",

"required":false,

"items":[

{

"type":"object",

"required":false,

"properties":{

"number": {

"type":"string",

"required":false

},

"type": {

"type":"string",

"required":false

}

}

}

] 

}

}

}

  

Reference: http://davidwalsh.name/json-validation

II. Directly generates Schema for specific parsed JSON data format:

http://www.jsonschema.net/#

Step III - Validate functionality.

We can directly validate Schema functionality online.

http://json-schema-validator.herokuapp.com/

http://jsonschemalint.com/

Note:

To validate JSON content - http://jsonschemalint.com/

Update on 11.05.2014 -18.05.2014 week

As far as I understand the project by now, I can divide it into following tasks:

Task 1:

Currently XML is read and UI is rendered using UIGenerator class. I have to do it using

the same flow(with minimal affect to existing code) and read a JSON to render a similar UI.

So I am looking at the governance. generic and governance. generic.ui classes. Now I am getting much familiar with code and I would like to know whether these are the relevent code segments I am referring at the moment.

First I hope to create and test new Java classes to replace Util ,GenericArtifactsUtil and Manage GenericArtifactsService to do novel changes without breaking the current model.

As a milestone I will try to plug JSON based rxt to existing model and validate and parse it within 2 weeks time.

For now it is the main thing I am working on.

Task 2:

Drag and Drop UI

I have run the G-Reg product and did some experiments on how to create new artifacts.

In the existing system user has to create the XML file according to his requirements and upload it in order to configure new artifact. User has to be familiar with coding in order to make use of this system.

After looking at artifact configuration XML files I got an idea on a general artifact structure.

If drag and drop UI is implemented what is going to be drag drop elements and what is going to be required information.

I designed simple UI for that drag drop UI interface and it's elements and still I am searching for some kind of drag and drop UI framework with Apache version 2.0 license to cater this, where user doesn't need to be much familiar with coding.

http://aribaweb.org/Home.htm seems somewhat relevant to it but I still go through its documentation and demos and not having an exact idea on its usability for this project.

I hope it would be better if I get to know any of such drag drop UI component available in WSO2 carbon and some people working in those areas to get some guidance. I will carry on searching about this in between task 1.

Task 3:

Create the JSON using a UI template which we construct using drag and drop sort of UI.

For this task I have to first look upon task 2. So I am not going to search more about it since it mostly depend on how I going to enhance user experience through task 2.

Also I understand how to build the individual components and finally how the whole product is built in order to create wso2greg-4.6.0.zip. That is what we run as the product in the browser. I was new to ubuntu and now I am gradually getting familiar with it. To understand the code and it's functionality I have to get familiar with remote debugging.

Remote debugging:

To start Carbon server with remote debugging enabled type:

./wso2server.sh -debug 5005

in the terminal.

before enter Create debug pointer in code and configure a new debug window. Then enter the above command and start debugging.

I hope the knowledge and the understanding I got within last weeks about WSO2 Carbon, XML to JSON conversion and JSON parsing and validation would help me to fulfill tasks, with not much difficulty.

Getting on....

As the initial phase of the project I had to get familiar with Carbon component architecture,WSO2 Governance Registry code base (specially governance component), UI generator models, Web services, JSON parsing, JSP, Test frameworks etc. during the interim period.

Building WSO2 Carbon from source:

I referred WSO2 documentation links on Carbon architecture & checked out Carbon and built them on Ubuntu 14.04.

Carbon comes as 3 directories:  Orbit, Kernel & Platforms.

Orbit bundles external third party dependencies which are not maintained by WSO2, but are needed for some products.

WSO2 products are built on top of WSO2 Carbon Kernel, which contains the Kernel libraries used by all products.

A WSO2 Platform release is a set of WSO2 products based on the same Carbon release.

In order to check out the code I had to install SVN.

Installing SVN:

Subversion is an open source version control system. To setup SVN in the Ubuntu environment:

$ apt-get install subversion

     $ apt-get install libapache2-svn

Referred http://lasindu.com/setup-svn-in-your-ubuntu-environment/

Installing Maven:

Maven basically is a build tool which can manage a project's build, reporting and documentation from a central piece of information. Below is an easy guide to install maven in your local setup.

1. Download apache-maven-3.0.4-bin.zip and unzip it.

 $ tar -xzvf apache-maven-3.0.4-bin.tar.gz

2. Create a new directory

$ sudo mkdir /usr/local/apache-maven

3. Copy extracted files into the new folder

$ sudo cp -R apache-maven-3.0.4 /usr/local/apache-maven/

4. Append these lines to .bashrc file in /home directory.

M2_HOME=/usr/local/apache-maven/apache-maven-3.0.4

M2=$M2_HOME/bin

MAVEN_OPTS="-Xms256m -Xmx512m"

PATH=$M2:$PATH

5. Run 'mvn –version' to verify that it is correctly installed.

When building from source, I first had to build orbit, then kernel and finally build the platform.

Referred http://lasindu.com/installing-maven3-in-ubuntu-12-04/

First I created separate folders to check out orbit, kernel and platforms.

Home > repos >wso2 > carbon > kernel > branches > 4.0.0

Home > repos >wso2 > carbon > orbit> branches > 4.0.0

Home > repos >wso2 > carbon > platforms > branches > 4.0.0

Then come out of directory 4.0.0 and executed the checkout command.

Check out command:

Orbit:

$ svn checkout https://svn.wso2.org/repos/wso2/carbon/orbit/branches/

Kernel:

$ svn checkout https://svn.wso2.org/repos/wso2/carbon/orbit/branches/

Platforms:

$ svn checkout https://svn.wso2.org/repos/wso2/carbon/platforms/branches/

Also I got a basic understanding on pom. xml which is essential for the product to be built.

JSON:

JSON (JavaScript Object Notation) is a lightweight data-interchange format. Derived from the JavaScript scripting language, JSON is a language for representing simple data structures and associative arrays, called objects. 

In my project XML based RXT format will be replaced by JSON. JSON would be more human friendly than XML and will be much efficient in processing. 

 XML to JSON:

One of the following two Methods can be used.

(I) Badgerfish implements the full XML infoset in JSON using various techniques.

(II) Mapped allows you to manually map XML namespaces to JSON element prefixes.

JSON Parsing:

The existing model used axiom which uses StAX based implementation to parse XMLs. To parse JSON in the proposed implementation II would use Jettinson.

Jettison is the JSON StAX parser and writer implementation used in implementing JSON support for Axis2. It is a collection of Java APIs (like STaX and DOM) which read and write JSON. It consists of two different parsers and writers for the “Badgerfish” and “Mapped” conventions. Using Jettison, both formats can be supported in Axis2, and the user can use the one which is more appropriate for the application.

JSON schema validators would become handy when validating the user created RXT files. Using the Java model created, org.wso2.carbon.governance.generic.ui component could be modified to automatically generate the necessary UIs. 

After referring to the code base I got some idea about governance component and its UI generator models.

Beginning of a New Era...

In my last post I described that awesome path I came towards GSOC 2014. Here, I would like to introduce my project for GSOC 2014.

Topic: Implement Registry Extension (RXT) 2.0 + Associated UI support

Organization: WSO2

Mentor: Shelan Perera (Senior Software Engineer – WSO2)

What is WSO2 governance Registry? 

           

“Governance encompasses more than just technology; governance also includes people and processes. The WSO2 Governance Registry provides the right level of structure straight out of the box to support SOA Governance, configuration governance, development process governance, design and run-time governance, lifecycle management, and team collaboration.”

That is how WSO2 describes Governance registry.

Introduction to project:

Configurable Governance Artifacts are one of many well-defined extension points supported by the WSO2 Governance Registry. It extends the functionality of WSO2 Governance Registry to support modeling any type of asset to suit the user requirements. The RXT model provides an extensible configuration language to create new types of artifacts. Governance API provides easy management and access of resources from outside.

Main objective of this project:

To provide enhanced usability in configuring new types of artifacts and introduce a new JSON based RXT format. After the successful completion of the project, the existing XML based RXT configuration will be tailored to use JSON and users with less programming background will be able to create new artifact types more easily. In addition, this project focuses on providing extended functionality to the RXT model and provides customizable UI to create/manage new artifact types.

Deliverables:

·        Replace the current XML based configuration with a JSON based configuration.

·        Refactor the existing UI generator model to use JSON based configuration instead of the XML based configuration and plug it into the governance API along with new JSON schema validators and parsers.

·        Implementing nested RXT support so that new artifact can be added inside another artifact.

·        Proved a UI based editing/creation of the configuration file along with the JSON edit.