Project: Implement Registry Extension (RXT) 2.0 + Associated UI support - Updates and Notes

This is an update of  my project so far in GSOC 2014. There was a slight change in the project scope since the JSON support for RXT was temporarily removed and the scope was narrowed down to creating a Jaggery app which could support creating a new artifact type with an intuitive drag and drop UI.

I had a meeting with my mentor and few others in WSO2 last week and below are the facts which were discussed.

• Finalized the requirement of a Jaggery app to create a new artifact type which could be installed to either Greg or ES via Management Console
• RXT <content> should be generated using drag and drop components and user shall be able to change the fields easily by dragging components here and there.
• The RXT XML should be generated in the client side using JS as well as some metadata should be kept with UI fields in order for ES to generate its ‘add metadata’ model.
• This model should also facilitate creating a JSON out of the UI, at some point of time.
• Once the XML is generated, a backend call will be made to Registry to add the RXT via Jaggery.

Current Progress

In the Process of finding a suitable plugin for drag and drop functionality, I looked upon Gridster.js [1] was used in UES and is very flexible in generating UIs. But it seems to be bit difficult to use it in this purpose, since it needed a lot of fine tuning to be able to cater this requirement. Then I found Bootsnipp Form Builder [2] which seems to be bit similar to the one which is required under MIT license and built on bootstrap. So decided to go on with it.

I was able to create a simple UI, create a Jaggery app and host it in Greg/ES and test the functionality. Below is a sample UI which has only the basic functionality to create a RXT.

                                           

I was told several issues the existing RXT, XML model has and the difficulties in rendering the UI (in ES) only with the information it provides and necessity of keeping other metadata along with the xml model. So I’ working on a way to find out the best possible way to bridge the gap between the RXT xml and the UI it is generating in ES side.

Next Steps

• Write a JS script to generate the RXT xml from the form data/HTML
• Keep other metadata which are not going in to the xml in a JSON model (tentative)
• Call backend Registry client/stub and send the RXT to registry.
• Try to load RXT xml data to the UI back when editing a RXT artifact.

Github Repo : https://github.com/ireshapm0/ArtifactBuilder

[1] http://gridster.net/

[2] http://bootsnipp.com/forms

JSON Parsing and Validation

Step I - JSON Parsing

To parse JSON data we can use  GSON library.

http://google-gson.googlecode.com/svn/trunk/gson/docs/javadocs/com/google/gson/JsonParser.html

When given the json file as a String to parse method above, it will return the parsed JsonObject as tree.

public static JsonObject parse(String jsonString) {

JsonParser jsonParser = new JsonParser();

JsonElement rxtJsonElement = jsonParser.parse(jsonString);

JsonObject rxtJsonArtifactTypeObject = rxtJsonElement.getAsJsonObject().getAsJsonObject("artifactType");

return rxtJsonArtifactTypeObject;

}

Step II - JSON data format Validation

Develop Shema

JSON Schema is a tool for validating the structure of JSON data. When we consider data format we need to know what does data elements mean and what the valid inputs for these elements are. To ensure we have obtained the required data we write a schema.

I.Manually write Schema for JSON:

There are certain keywords that are used to develop a Schema for any sort of json data.

They can be found at http://tools.ietf.org/html/draft-zyp-json-schema-03 .

Most frequently used key words are:

Ø  type

Ø  properties

Ø  items

Ø  required

Ø  maxItems

JSON is built on two structures:

Ø  A collection of name/value pairs commonly realized as “objects”.

Ø  An ordered list of values commonly realized as “ arrays”.

This defines the type keyword in a Schema.

properties are defined to validate the attributes in an object.

Within an array the list of items are defined by items keyword.

Whether a particular object array or property and item is a must for it to validate is defined by required keyword. 

MaxItems defines the maximum number of elements in array type of object type.

Similarly you can find how any of this keyword defines the validity of the JSON data against the Schema we develop.

Example:

JSON data to be validated:

{

"address":{

"streetAddress": "21 2nd Street",

"city":"New York"

},

"phoneNumber":

[

{

"type":"home",

"number":"212 555-1234"

}

]

}

Schema developed:

{

"type":"object",

"required":false,

"properties":{

"address": {

"type":"object",

"required":false,

"properties":{

"city": {

"type":"string",

"required":false

},

"streetAddress": {

"type":"string",

"required":false

}

}

},

"phoneNumber": {

"type":"array",

"required":false,

"items":[

{

"type":"object",

"required":false,

"properties":{

"number": {

"type":"string",

"required":false

},

"type": {

"type":"string",

"required":false

}

}

}

] 

}

}

}

  

Reference: http://davidwalsh.name/json-validation

II. Directly generates Schema for specific parsed JSON data format:

http://www.jsonschema.net/#

Step III - Validate functionality.

We can directly validate Schema functionality online.

http://json-schema-validator.herokuapp.com/

http://jsonschemalint.com/

Note:

To validate JSON content - http://jsonschemalint.com/

Update on 11.05.2014 -18.05.2014 week

As far as I understand the project by now, I can divide it into following tasks:

Task 1:

Currently XML is read and UI is rendered using UIGenerator class. I have to do it using

the same flow(with minimal affect to existing code) and read a JSON to render a similar UI.

So I am looking at the governance. generic and governance. generic.ui classes. Now I am getting much familiar with code and I would like to know whether these are the relevent code segments I am referring at the moment.

First I hope to create and test new Java classes to replace Util ,GenericArtifactsUtil and Manage GenericArtifactsService to do novel changes without breaking the current model.

As a milestone I will try to plug JSON based rxt to existing model and validate and parse it within 2 weeks time.

For now it is the main thing I am working on.

Task 2:

Drag and Drop UI

I have run the G-Reg product and did some experiments on how to create new artifacts.

In the existing system user has to create the XML file according to his requirements and upload it in order to configure new artifact. User has to be familiar with coding in order to make use of this system.

After looking at artifact configuration XML files I got an idea on a general artifact structure.

If drag and drop UI is implemented what is going to be drag drop elements and what is going to be required information.

I designed simple UI for that drag drop UI interface and it's elements and still I am searching for some kind of drag and drop UI framework with Apache version 2.0 license to cater this, where user doesn't need to be much familiar with coding.

http://aribaweb.org/Home.htm seems somewhat relevant to it but I still go through its documentation and demos and not having an exact idea on its usability for this project.

I hope it would be better if I get to know any of such drag drop UI component available in WSO2 carbon and some people working in those areas to get some guidance. I will carry on searching about this in between task 1.

Task 3:

Create the JSON using a UI template which we construct using drag and drop sort of UI.

For this task I have to first look upon task 2. So I am not going to search more about it since it mostly depend on how I going to enhance user experience through task 2.

Also I understand how to build the individual components and finally how the whole product is built in order to create wso2greg-4.6.0.zip. That is what we run as the product in the browser. I was new to ubuntu and now I am gradually getting familiar with it. To understand the code and it's functionality I have to get familiar with remote debugging.

Remote debugging:

To start Carbon server with remote debugging enabled type:

./wso2server.sh -debug 5005

in the terminal.

before enter Create debug pointer in code and configure a new debug window. Then enter the above command and start debugging.

I hope the knowledge and the understanding I got within last weeks about WSO2 Carbon, XML to JSON conversion and JSON parsing and validation would help me to fulfill tasks, with not much difficulty.

Getting on....

As the initial phase of the project I had to get familiar with Carbon component architecture,WSO2 Governance Registry code base (specially governance component), UI generator models, Web services, JSON parsing, JSP, Test frameworks etc. during the interim period.

Building WSO2 Carbon from source:

I referred WSO2 documentation links on Carbon architecture & checked out Carbon and built them on Ubuntu 14.04.

Carbon comes as 3 directories:  Orbit, Kernel & Platforms.

Orbit bundles external third party dependencies which are not maintained by WSO2, but are needed for some products.

WSO2 products are built on top of WSO2 Carbon Kernel, which contains the Kernel libraries used by all products.

A WSO2 Platform release is a set of WSO2 products based on the same Carbon release.

In order to check out the code I had to install SVN.

Installing SVN:

Subversion is an open source version control system. To setup SVN in the Ubuntu environment:

$ apt-get install subversion

     $ apt-get install libapache2-svn

Referred http://lasindu.com/setup-svn-in-your-ubuntu-environment/

Installing Maven:

Maven basically is a build tool which can manage a project's build, reporting and documentation from a central piece of information. Below is an easy guide to install maven in your local setup.

1. Download apache-maven-3.0.4-bin.zip and unzip it.

 $ tar -xzvf apache-maven-3.0.4-bin.tar.gz

2. Create a new directory

$ sudo mkdir /usr/local/apache-maven

3. Copy extracted files into the new folder

$ sudo cp -R apache-maven-3.0.4 /usr/local/apache-maven/

4. Append these lines to .bashrc file in /home directory.

M2_HOME=/usr/local/apache-maven/apache-maven-3.0.4

M2=$M2_HOME/bin

MAVEN_OPTS="-Xms256m -Xmx512m"

PATH=$M2:$PATH

5. Run 'mvn –version' to verify that it is correctly installed.

When building from source, I first had to build orbit, then kernel and finally build the platform.

Referred http://lasindu.com/installing-maven3-in-ubuntu-12-04/

First I created separate folders to check out orbit, kernel and platforms.

Home > repos >wso2 > carbon > kernel > branches > 4.0.0

Home > repos >wso2 > carbon > orbit> branches > 4.0.0

Home > repos >wso2 > carbon > platforms > branches > 4.0.0

Then come out of directory 4.0.0 and executed the checkout command.

Check out command:

Orbit:

$ svn checkout https://svn.wso2.org/repos/wso2/carbon/orbit/branches/

Kernel:

$ svn checkout https://svn.wso2.org/repos/wso2/carbon/orbit/branches/

Platforms:

$ svn checkout https://svn.wso2.org/repos/wso2/carbon/platforms/branches/

Also I got a basic understanding on pom. xml which is essential for the product to be built.

JSON:

JSON (JavaScript Object Notation) is a lightweight data-interchange format. Derived from the JavaScript scripting language, JSON is a language for representing simple data structures and associative arrays, called objects. 

In my project XML based RXT format will be replaced by JSON. JSON would be more human friendly than XML and will be much efficient in processing. 

 XML to JSON:

One of the following two Methods can be used.

(I) Badgerfish implements the full XML infoset in JSON using various techniques.

(II) Mapped allows you to manually map XML namespaces to JSON element prefixes.

JSON Parsing:

The existing model used axiom which uses StAX based implementation to parse XMLs. To parse JSON in the proposed implementation II would use Jettinson.

Jettison is the JSON StAX parser and writer implementation used in implementing JSON support for Axis2. It is a collection of Java APIs (like STaX and DOM) which read and write JSON. It consists of two different parsers and writers for the “Badgerfish” and “Mapped” conventions. Using Jettison, both formats can be supported in Axis2, and the user can use the one which is more appropriate for the application.

JSON schema validators would become handy when validating the user created RXT files. Using the Java model created, org.wso2.carbon.governance.generic.ui component could be modified to automatically generate the necessary UIs. 

After referring to the code base I got some idea about governance component and its UI generator models.

Beginning of a New Era...

In my last post I described that awesome path I came towards GSOC 2014. Here, I would like to introduce my project for GSOC 2014.

Topic: Implement Registry Extension (RXT) 2.0 + Associated UI support

Organization: WSO2

Mentor: Shelan Perera (Senior Software Engineer – WSO2)

What is WSO2 governance Registry? 

           

“Governance encompasses more than just technology; governance also includes people and processes. The WSO2 Governance Registry provides the right level of structure straight out of the box to support SOA Governance, configuration governance, development process governance, design and run-time governance, lifecycle management, and team collaboration.”

That is how WSO2 describes Governance registry.

Introduction to project:

Configurable Governance Artifacts are one of many well-defined extension points supported by the WSO2 Governance Registry. It extends the functionality of WSO2 Governance Registry to support modeling any type of asset to suit the user requirements. The RXT model provides an extensible configuration language to create new types of artifacts. Governance API provides easy management and access of resources from outside.

Main objective of this project:

To provide enhanced usability in configuring new types of artifacts and introduce a new JSON based RXT format. After the successful completion of the project, the existing XML based RXT configuration will be tailored to use JSON and users with less programming background will be able to create new artifact types more easily. In addition, this project focuses on providing extended functionality to the RXT model and provides customizable UI to create/manage new artifact types.

Deliverables:

·        Replace the current XML based configuration with a JSON based configuration.

·        Refactor the existing UI generator model to use JSON based configuration instead of the XML based configuration and plug it into the governance API along with new JSON schema validators and parsers.

·        Implementing nested RXT support so that new artifact can be added inside another artifact.

·        Proved a UI based editing/creation of the configuration file along with the JSON edit.

GSOC & ME…

Being an undergraduate of Department of Electronic & Telecommunication Engineering, University of Moratuwa, I wondered what Google Summer of Code is.

At first it was all about money in 2012! I was first year undergraduate then, my fellow undergraduates buy Apple stuff, DSLR cameras. So I felt it would be nice if I also could take part in GSOC. In 2013 it was a dream. Many of my own batch mates took part in it. 

Then it comes my industrial training semester and I selected to Virtusa, one of the biggest IT companies in Sri Lanka. I got exposure to coding and then what?

GSOC 2014 announced! Now that I understood this is not all about money, but will be a lifetime experience for me.

First I was looking on a  project from Joomla . But when the accepted organizations were announced, I saw WSO2 in the list. I got more information about the project “Implement Registry Extension (RXT) 2.0 + Associated UI support” from a senior who worked in WSO2 governance registry. I was interested on the topic & managed to understand the project to certain level even I was not much aware about all.

After some mail exchanges with some team members of Governance Registry, I cleared my vision on project and its deliverables, did some experiments on my own and with help from other resource people  I wrote the proposal for GSOC 2014.

It was exactly the beginning of an awesome journey.

On April 21 ^st  2014!

I was in a dream when got up and rushed to mélange site.

Yey…

Dream was out there…

How to play video in Android video view

Playing video in Android from a file or from a stream over the network is relatively simple as long as the video is in an acceptable format. The supported formats are

• MP4 (MPEG-4), H.263, H.264 AVC (for Android 3.0+) & VP8 (for Android 4.3+)

Method I – Video File is kept in res/raw folder

1.      Create folder called “raw” in res folder if it does not exist and put video file in raw folder.

Note: Try with the video in following link

http://www.ooklnet.com/files/368/368007/video.mp4 

(This video was played in emulator - Android 4.4, API 19)

2.      Add this to xml file

<VideoView

       android:id="@+id/video_view"

       android:layout_width="fill_parent"

       android:layout_height="fill_parent"/>

3.      Call “PlayVideo” method in “onCreate” method.

public void playVideo(){

mVideoView = (VideoView) findViewById(R.id.video_view);

mVideoView.setVideoURI(Uri.parse("android.resource://" +getPackageName()+"/"+R.raw.lenovo));

mVideoView.setMediaController(new MediaController(this));

mVideoView.requestFocus();

mVideoView.start();

  

}  

“lenovo” is the name of the video file without it’s extension.

I was able to play .avi .mp4 and .3gp video files but not .flv in the actual device. (I used SONY Xperia P - Android 4.1.2)

Method II – Video File is kept in SD card

Ø  If video files are kept in SD card

Then set path:

mVideoView.setVideoPath(sdcard/file_ name_ with_extension");

Update:

          Video in the SD card is store in
          Internal storage>Android><Package_name><files>

            So that the path would be set as

            File root = Environment.getExternalStorageDirectory();

String externalFilesDir = c.getExternalFilesDir(null).toString();

String videoResource = externalFilesDir +"/"+video_filename with extension;              //video_filename as String

mVideoView.setVideoPath(videoResource);

mVideoView.start();

If you want to put videos to emulator SD card go to

           

                 DDMS>File Explorer> Click pull files to SD card

Ø  Sometimes it will fail saying “Transfer Error Occurred, Read only file system”

Then try following:

·         Go to .android > avd >select emulator > go to properties > uncheck read only.

·         Go to AVD manger in eclipse > Edit emulator > Keep value to SD card memory 

      and try remounting SD card

Ø  Sometimes you won’t be able to uncheck read only. I don’t know

what to do then L.

You would better try with actual device. J

Connect your phone to PC .  

Go to Internal Storage > Android >Data > create folder by your package name > make a folder                   named “files” > put video file to files folder > copy paste . apk to phone and run.

Refer link : http://eagle.phys.utk.edu/guidry/android/playingVideo.html

Ø  If you see a message saying "Sorry, this video cannot be played"

    Try changing the name of your video file by excluding any white spaces. It looks like lower Android OS versions cannot play video files containing white spaces in their file name. For example, change "My Video.mp4" to "MyVideo.mp4".

Ø  If you still cannot play the videos check whether the video is in H.264 AVC Baseline profile.

P.S.

Instead using video View we can use media player in Android.

Refer link:

http://stackoverflow.com/questions/15839082/android-4-2-with-4-mediaplayers-cant-play-this-video