HTML5_Logo_2561

Developers should not believe the HTML5 hype

As an Architect and Lead developer, I have to sit down with the non-techie and convince them why I think a technology or framework is the right one for a project and then I build my team to deliver the solution. The past two years or so, the marketing engine of technology companies have been spinning at full throttle. HTML5 is a the future, everything will run in the browser and more blah blah blah.
Don’t get me wrong, HTML5 brings some good technologies to the web application developer. Nevertheless, I think there is a lot of hype as HTML5 doesn’t really bring anything new to the table.
First of all, there are multiple type of applications that we are all well-aware of;

  • Consumer
  • Business
  • Enterprise

Consumer Application

If you are building a consumer oriented application which doesn’t use any of the native functionality of OS, then I would recommend the application to be built in HTML5 and all the RIA fanfare that comes with it. You can best view this in the mobile space where developers build either native or web apps depending on the application requirement. Remember that web application (HTML5 ) do not have direct access to printers, USB port or any other hardware such as Bluetooth and network services. Simple word processing application that can be developed in HTML5 such as Google Doc (where this blog is typed from). I haven’t come across any serious application written in HTML5 or the likes yet (JavaFX, Flash and Silverlight are not HTML5).

Business Application

Critical business application such as POS which requires access to barcode scanner as an example can’t be written using HTML5. You can have a native application delivered through the browser such as Java Applets ( or JavaFX) using webstart. Java applets can access OS features and hardware and provides another layer of security. Business application are delivered in controlled environments, for example, the application can be deployed on Linux desktop in company “a” environment only. For as much “fanfare” one might create around HTML5, these type of application will not cease to exist.

Enterprise Application

Enterprise applications come in various forms, from desktop to servers. As this is a comparison to HTML5, I will only focus on desktop application. First, let take a financial company such as a stock brokerage firm. There is a reason why stock trading application run as closely to the OS as possible ( and also to the exchange), in one word, PERFORMANCE. Web browsers performance sucks regardless of which one you are using, JavaScript is just too slow to implement some of the logic. In the trading business, a millisecond is all that is required to lose millions of £. Can you for one second imagine building a Bloomberg trading platform using web technologies, that’s laughable. There is a reason why the finance industry are still using Java Swing as their desktop technology of choice.

Conclusion

HTML5 is a promising step in the right direction to building scalable robust web application but it will not obliterate desktop applications, not today, not tomorrow, not ever (really!?). We can already see it in the mobile space where developers rather write native application so that they can utilize OS features and hardware. Web applications cannot access you local files directory (I am not talking uploading a file to a site) to read or write.  A simple operation such as reading available space in a directory or writing a log to a local directory is not possible (again I am not talking about downloading or saving a page). Therefore, developers should not believe the hype. HTML5 is not the silver bullet and it is a shame when companies such as Adobe sends mix-messages by discontinuing their Flex offering. Anyway, Adobe has never been a big player in the enterprise desktop application market.

If you disagree with my points, feel free to share your thoughts.

vodafone-top-61

5 Things all Java developer should know when developing for the cloud

The last couple of years, “Cloud Computing” replaced Web 2.0 as the new buzzword. You can read, hear and see everywhere the cloud is coming. To most developer, this is still the same old sh*t. If you have experience in developing distributed system then you should be fine, you say. Well not entirely true, the IT department wants to deploy on cheap cloud and therefore some restrictions now applies. I will list 5 things that I think all developers should know when working with cloud Platform as a Service provider such as Amazon Beanstalk or Google App Engine. This list also applies to IaaS architecture. Some of the points might be obvious to the more experienced, nevertheless, they need to be mentioned.

  • Static objects

We all know the difference between instance variable (non-static) and class variable (static variable). We use static to tell the JVM that they should only be one instance of this variable (singleton). If the static variable is declared with the “final” keyword, this will not cause a problem in a distributed environment as the value will never change. The problem is when we expect the value of the variable to change. As in a cluster environment, GAE and Beanstalk run your application in multiple JVM. If a the value of your static variable has changed in JVM, it will not be propagated to the cluster therefore leading to inconsistencies. I recommend that you avoid static variable unless that set as “final” and their values are hard-coded so there is no way to change their values are runtime.

  • Caching Objects
This one is related to performance in order to avoid expensive operations such as running database queries and others. Sometimes we need to cache objects in memory and therefore we implement our own caching strategy through the use of simple HashMap or some other caching solutions available outthere. Caching has many benefits but implementing a caching strategy should be approached with care. This is because caching has the same problem as static objects. Your cache will be in the local JVM therefore not it will not be visible in the cluster. There are some solutions, for example, GAE uses Memcached and Beanstalk can make use of Amazon ElastiCache which is compliant with Memcached. When developing for a PaaS environment, make sure to not implement your own caching system but look for one that is supported by the vendor. I know this can lead to vendor lock-ins.
  • Server-side Session
Something we do take for granted in single environment is storing application session data on the server. Based on experiences, mainly using GAE, I encountered multiple issues with session management. Since then, Google has fixed alot of the issues with the way GAE handle sessions for Java application. To minimize writing session to a datastore, we store application state in memory. Most application are written without any vendor approach in mind; so we use JEE as-is. This approach would work in you deploy in any self hosted clustered environment but Google PaaS. Google implements their own session management which is off by default therefore you need to enable it in appengine-web.xml and make sure that all your objects implements the java.io.Serializable interface.

Note: Note, session data is always written synchronously to memcache. If a request tries to read the session data when memcache is not available (or the session data has been flushed), it will fail over to the datastore, which may not yet have the most recent session data. This means that asynchronous session persistence may cause your application to see stale session data. However, for most applications the latency benefit far outweighs the risk.

 

  • Event-driven Execution
This is more about running a process at a given time such as Scheduling task. Again, in a managed environment, it is straightforward to implement a timer or scheduler service. But this is a clustered environment which is not managed by yourself and their stack his different to yours. I personally use Quartz Scheduler when working in a single server environment. In a clustered environment such as Beanstalk or GAE, it is difficult to know which instance will be triggered and execute the task only once. The folks at Google have provided another solution with their own implementation of Cron for Java which can be used. At the time of writing, Amazon Beanstalk didn’t have a solution yet. Therefore, consider before-hand when designing your system, which approach to take in order to create scheduled tasks for your application.
  • JRE white list
I believe this related to GAE J only. Google App Engine for Java doesn’t allow the use for all available API in Java, especially if they do require access to the file system. The fact that there is a such a restriction impose by the Google has led us to look elsewhere for some of our projects. The cost of re-developing our application to please them is much higher than deploying them elsewhere. Also, another downside of GAE J is doesn’t fully support JEE servlet specification. You cannot implement custom security for your application through your web.xml therefore pushing you to use Google own security mechanism. I would recommedn using GAE J when developing a greenfield project which can be built with these restrictions here  and here in mind. If you want to be locked-in using GAE J for your application, then I recommend it as a cost efficient way to testing your application otherwise, look somewhere else.
I hope this was helpful and if there’s mistake, feel free to get back to me and I make any corrections. Also, I am sure that I am missing some other points, add them to the comments sections.P.S. here is a nice comparison from IBM

Cheers and Happy Coding.
bad_robot1

The traveling robot problem

I am currently between contracts and for first time in many years, I am requested to write some complex code from home so I took the liberties to share my result on my blog.

A startup based in London sent me this problem:

Problem:
A robot can be programmed to run 1, 2, 3, 5 and 10 kilometers and it takes 10, 5, 3, 2 and 1 minutes, respectively. Once it runs to programmed kilometers, it must be turned off for 2 minutes. After 2 minutes it can again be programmed to run for a further 1, 2, 3, 5 or 10 kilometers. How would you program this robot to go exactly 43 kilometers in minimum amount of time.

Please write a java program that solves the problem by finding the minimum time required to travel 43 kilometers. Your solution should print out the correct sequence of distances to be programmed into the robot: for example (10,5,10 and so on…)
When coding your solution please pay special attention to design, data structures, dynamic programming, faster execution time and creating generic solutions. (it should be fairly easy to change the parameters) Compute and provide Big O notation for main piece of your solution too.


END-OF-QUESTION

Before you look at my answers, it will be cool for you to try the exercise too so that we can share feedback in the comments section.

The question brought back memory of my Computer Sciences days, especially Data Structure and Algorithm lectures. At first glance, this looks like a simple problem but it goes deeper than that. Like artists, there are multiple ways to write code but once you look at someone else code, then it becomes difficult to be original ( IP patent law suit anyone?). Any experience developer can write code that would work perfectly for this problem, looking closely to what is being requested and you will notice that there is an extra requirement; dynamic
programming
. For those who needs more information about what dynamic programming is, I suggest the following wikipedia article. I had to spend sometimes reading the article to refresh my memories and off-I-went-hacking-away. Bear in mind that you could solve this problem by rewriting a recursive application but that is not a dynamic programming approach.

Anyway, here are my results ( the algorithm was inspired by a similar problem set).

I hope this helps someone in the future in any ways.

Happy Coding

You can download the source code this github https://github.com/armelnene/the-traveling-robot-problem

primefaces-mobile-weather-app1

Primefaces Mobile – Weather App Example

As a Java developer, I usually get requests of building mobile apps. I like building Java application; enterprise, web and mobile. The latest projects that I have been involved with make heavy use of JSF and Primefaces in particular.

I am quite confortable with JSF and therefore I decided to build a mobile application using JSF and Primefaces Mobile. Primefaces Mobile wraps JQuery as a JSF component so that you do not have to write any Javascript. This approach has huge benefits: JQuery is a well tested framework used by large companies such as Google and Microsoft.

I will make this post quite brief. I wanted to know how easy it would be to recreate the demo from the Primefaces Mobile labs page.

Here is a screenshot of the final application running in Firefox 11 (Windows 7 64 Bit).

This was a simple example taken directly from the website therefore I was expecting to a short exercise to make build and make it run. Well not so fast.

I created a Maven based Java EE 6 project using Netbeans IDE 7.1.1. After browsing Google for a bit, I spent a few minutes getting the right repository and dependency in place.

Once the dependency where in the place, I had to create the beans required for the actual JSF page to work. I found the Primefaces Mobile backing beans on Google Code. So now I had everything setup and running. At first glance, the application seemed to running fine and working. Then I tried the application on my iPad and the Android Emulator, and nothing was working. The user interface was displayed but the “get forecast” button was not making any Ajax calls.

So I started to debug the application everytime I had some spare time. I also noticed that, while running on a desktop browser, the application would be able to an Ajax called and updated the screen with the values (see screenshot above) but you wanted to make another to find out the temparature of let’s say London, the nothing would actually unless you refresh the page and try again. OK, so it’s not working as expected but the example on Primefaces Mobile labs worked fine on my iPad, emulator and desktop browser (IE 9 excluded).

So I ran the application using the NetBeans debugger and decided to look through Firebug. The first call goes through and stops at the breakpoint but subsquent calls do not even get to the managed bean. Firebug shows that the other code are being retrieve from the cache. I manually set all the HTTP headers so that it does not cache any content but this is still the same result NOT WORKING!

I have uploaded the code to GitHub click on the link to download it.

In conclusion, it’s not plug-and-play to make the examples on Primefaces.org labs work. The documentation for the examples are quite poor and I hope that the good folks from Primefaces can take look at my code and tell us what I am doing wrong.

A part from that, Primefaces is a good JSF framework that I use on a daily basis on multiple projects therefore I cannot really put them down but I wished the mobile examples work and tell us what is needed to make it work.

Please share your experience in the comments below or advice on how to fix it. Feel free to download the code from GitHub and have a look to.

Happy coding 🙂

670px-Cope-With-a-Double-Parachute-Failure-Step-31

Parachutist Dilemma – a parallel programming problem

Here is a little mental problem solving exercise for you to try to do in 30 seconds; that was the time I was allowed. So I had a telephone interview from a company today and this guy asked me this question utilizing a parrallel programming algorithm. Here are the details:

Two parachutist (A and B: A is in front of B) land on single line; one behind the other. Their parachutes also lands behind them respectively. 
We have a set of instructions which are as follow (all instructions are executed, meaning A and B will executed instructions at the same time).

  1. move forward 1 step
  2. move back 1 step
  3. if standing on a parachute go to (1 | 2)

Question:

We need our parachutist to somehow meet each other, what are the sets of instructions that can make that happen?


Please try to answer before reading my answer below

My answer:

  1. execute 2
  2. execute 3

Explanation:

  • When both parachutist move back 1 step, they will both be on their parachutes
    • when parachutist A stands on his parachute, he will move back 1 step therefore go to 1
    • when parachutist B lands on his parachute, he will move forward 1 step 

Therefore our two parachutist will meet in B original position.

NB I had to change the original questions as the interviewer explanations led to the parachutist to never meet.

20090420-oracle-buys-sun1

Will Oracle really make NetBeans the BEST IDE for Java

Not yet!!! But according to their webcast they want to make NetBeans the best Java IDE. This is a statement not to be taken lightly. Oracle are now investing in three IDEs which are all established in different ways. JDeveloper is Oracle’s IDE of choice, I am currently using it in my current project as I am developing for Oracle WebCenter. Oracle is also a strategic developer and board member of the Eclipse foundation. I do not understand what exactly is a strategic developer but I know that JDeveloper is their strategic IDE. Now, introducing NetBeans, Oracle has inherited NetBeans as part of Sun acquisitions (and a ton of products and services). Until now, Oracle has never made any plug-ins for the NetBeans platform but now they promise to make it the best IDE for Java, how will they make that happen?

I am a NetBeans evangelist and as the rest of the community, it was good news to actually know where it stands in the “Oracle” vision. So Oracle will never drop JDeveloper therefore the real fight is between Eclipse and NetBeans (as it always has been). In the first step of making NetBeans the best IDE, Oracle shold immediately start porting all their Eclipse plug-ins to NetBeans. Eclipse has strength in its alliance which also includes IBM (fierce rival to Oracle). Something has to give and Eclipse will not never close shop because Oracle has left them which is not as worse as lack of investment in the NetBeans community. It would also be a foolish decision to give NetBeans to the Apache Foundation (no disrespect here but it seems that public funds do not grow on trees).

There is something funny about Oracle “choice” statement. Did anybody tried to develop for WebCenter using NetBeans or Eclipse? That’s a nightmare!!! There is no choice given, they literally forced us to use JDeveloper (Do you require any Oracle support?) in our company which is the largest in its field in Europe (I am avoiding to mention the company name but you should get it from my twitter stream).

Eclipse and NetBeans are direct competitors with Eclipse being the de facto IDE for Java, dropping Eclipse support would close doors to millions of developers and dropping NetBeans will disaster for many paying customers (Sun’s partners).  I am not sure what is going to happen in the NetBeans/ Eclipse and JDeveloper saga but something has to give.

Let me know what you think.

openid-1

Dummies Guide to GWT and OpenID – with example code

Once (?) in awhile, we developers browse the web for some quick and dirty solutions to some of our coding problems (sometimes not even problems, right?). This is another of those days, everything was going fine on this project I am currently working on. This is not a social networking sites but a site for sharing and comunicating (how I missed the days of the chatrooms) similar in many aspect to Digg and DZone but which also allows users to communicate in realtime and leave comments. So the back-end was sort of completed but so now time to focus on the user registration.

The sites needed to be open and avoid repeatitive tasks such registering users. The way forward was to implement some authentication to allow users to register with some sort of universally accessible ID (or sort of). Facebook Connect, OpenSocial and Google Accounts have their advantages but to me personally; the disadvantages outweight the pros. These are some of the disadvantages of those platform:

  1. Facebook is a popular site hence is its platform, Facebook Connect, in mostly Europe and America but not everybody in Europe and America have a Facebook account.
  2. OpenSocial, when it comes to single logon, has more advantages than Facebook. I supposed that if we do take into consideration all the OpenSocial sites, we might have apossible larger user based than supporting Facebook alone. Even that was too limited.
  3. Google Accounts, one thing Google does not advertise its user base. I could be wrong but do anybody actually knows how many people uses the various Google services (excluding search). Google has the same disadvantages as Facebook.

Enter OpenID. OpenID has been around longer than most Internet-based decentralised authentication platform. The beauty of this platform is that it also supported by most (if not all) large site on the Internet. From AOL, BBC, CNN to YAHOO and ZIMBRA, as I said most sites (based on the alphabet) support OpenID, check the OpenID Directory. It was recently announced that OpenID has reached 1Billion enabled accounts (read here). Based on those figures, I decided that OpenID for now was the authentication choice for this application. I will not be discussing any security issues in this entry, there are plenty of resources available on the Internet for that.

OK, so I am a Java developer and I am developing the front-end using GWT and Java on the server-side. I searched on the Internet for solutions on how to implement the authentication as GWT RPC are different to normal servlet call. I spent more time reading about the OpenID specification and implementations examples on the Internet. I have to admit that some of tutorials that I found on the Internet were somehow confusing and not helpful at all. Therefore I decided to write my own Dummies Guide to GWT and OpenID.

Dummies Guide to GWT and OpenID

First of all, it is important to know how OpenID works (please check the OpenID site for more info).

In a nutshell, OpenID allows you to authenticate with website (supporting the standard) with just an URL and voila. Your URL has to be hosted by an OpenID provider in order for it to work. For example, my blog is hosted by Google on blogspot.com. Google supports OpenID authentication therefore if a reader wanted to leave a comment on my blog, he does not have to have a Google Account as he can log-in with his Yahoo or AOL or Facebook or WordPress or any other OpenID provider sites, and then leave a comment, that simple.

GWT to OpenID and back

There are two ways to authenticate a user with an OpenID provider and GWT supports both. When authenticating a user, the relaying site (the site the user is trying to access) redirects to user to the OpenID provider (i.e. Google) login page.

The problem will be with the GWT RPC mechanism. GWT RPC calls are asynchronous and you cannot make any redirections. Therefore we need a way to execute the redirection from the client side, here is the code (I use OpenID for Java to handle the openID discovery from the RPC servlet), I then used USER object (just a simple POJO which only stores the redirection URL and the parameters) to be sent back and forth between the front-end and back-end.

#######################################################
public User authenticateOpenId(String provider_url) {
    try {
        ConsumerManager manager = new ConsumerManager();
        // This is the URL where the OpenID provider will redirect the user
        // once logged in.
        String returnToUrl = “http://localhost:8084/GwtOpenId“;
       
        // the provider URL will be used to identify the user OpenID provider
        List discoveries = manager.discover(provider_url);
        DiscoveryInformation discovered = manager.associate(discoveries);
        // OpenID 4 Java needs to  have an HttpServletRequest object, GWT sevlet 
have
        // convenient methods to retrieve the HttpServletRequest object 
and manipulate its
        // parameters
        this.getThreadLocalRequest().setAttribute(“openid-disc”, discovered);
        this.getThreadLocalRequest().setAttribute(“openid.mode”, “popup”);
       
        AuthRequest authReq = manager.authenticate(discovered, returnToUrl);
        FetchRequest fetch = FetchRequest.createFetchRequest();
        // I want to exchange the following attributes from the OpenID provider
        // remember that teh attribute will be returned only if it exits
        fetch.addAttribute(“email”,”http://schema.openid.net/contact/email“,true);
        authReq.addExtension(fetch);
        // Simple POJO to persist the data
        User user = new User();
        // In a normal servlet development, the following statement would make
        // a redirect call but this would not work when using GWT RPC
        if(!discovered.isVersion2()){
            user.setRedirect(authReq.getDestinationUrl(true));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
            } else{
            user.setParams(authReq.getParameterMap());
            user.setRedirect(authReq.getDestinationUrl(false));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
        }
        } catch (MessageException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (DiscoveryException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (ConsumerException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
    }
}
#############################################################

The above codes will format the request in order for the front-end to execute a redirect and allow the user to authenticate with his OpenID provider. Now here is the front-end code which executes the authentication and reads the data back.

#################################################################
// This is an extract from the openidEntryPoint.java

// This is where the magic happens – This code is only usefull when the OpenID 
provider
// redirects the user back to your site – please visit openid.net for valid 
parameters.
// The “if” statement checks to make sure that it is a valid response from 
the OpenID
// provider – You can do anything you want with the results here such 
as verifying the
// response with the server-side code
if(Window.Location.getParameter(“openid.rpnonce”) != null){
    String s = Window.Location.getParameter(“openid.mode”);
    // executes this if the user cancels the authentication process and the OpenID
    // providers returns to the your site
    if(s.equals(“cancel”)){
        sign.setText(“Server returned openid.mode=cancel”);
        openIdUrl.setText(“You need to Accept not Cancel”);
    }
    // Here, I am assuming that everything is fine and that the user has 
been sucessfully logged in
    else{
        sign.setText(“You have successfully signed in”);
        vp.setVisible(false);
    }
   
}
RootPanel.get().add(contentPanel);

}

private class UserAsyncCallback implements AsyncCallback<User> {
   
    public void onFailure(Throwable caught) {
        Window.alert(caught.getLocalizedMessage());
    }
   
    public void onSuccess(User result) {
        if (result != null) {
            //                    Window.open(result.getRedirect(), “_blank”,
 “height=200,width=400,left=100,” +
            //                            “top=100,resizable=no,scrollbars=no,
toolbar=no,status=yes”);
            // this the most important line in order to make the authentication.
 Here, I am redirecting the user
            // from the client side to the OpenID provider URL with the discovery
 data generated from the
            // RPC call to the servlet.
            Window.Location.assign(result.getRedirect());
            } else {
            Window.alert(“Ooops!!! Couldn’t find your provider”);
        }
    }
}
#######################################################################################################

I have attached the full NetBeans project with dependencies. The code is provided as-is and use at your own risk ;). Here is a screenshot of the working application:

Step1: Authenticate on the site (enter the URL)
moz-screenshot-6

Step2: Redirect to OpenID provider (Google is my provider 🙂 ), authenticate with your provider
moz-screenshot-7

Step3: Allow the application to access your OpenID details and redirect back to the original site
moz-screenshot-8

Step4: final step, check the parameters from the provider and proceeds accordingly
moz-screenshot-9

Take a look at the URL in each of the above step to see the OpenID data. OK, so my code actually works (yuppie), now you know that it is possible form GWT to OpenID and it’s not as complicated as many other sites are trying to show. The code is just for authentication but once authenticated, you can retrieve any parameters that you need. In this example, the query is sent through the browser URL (GET) but you can change it to be encoded in a form submit action. Iwrote some of the code to allow the user to authenticate via a popup window, the code is not complete and maybe someone else might want to have a go. My only problem is getting the redirect back to the original window but apart from that it works.

Download the source code

I hope you guys enjoyed it and Happy Coding.

4124677046_69aa7a07b3

GWT Visualization Example – Annotated Time Line Chart Tutorial

If you are a server-side Java developer such as myself, I am sure you want an easy way to create AJAX based application but pure JavaScript (ECMAScript) is not your strong skills. The beauty of JavaScript is that it runs on any web server, no need for a servlet container or anything of that sort.

So, I have been writing AJAX based UI using the GWT framework for the past year and half. I am still not a web developer but I understand enough CSS to spice up my site. Anyway, I am writing this tutorial because I think that the GWT Visualization team, even tough they did a good job, over complicates the creation of charts in their tutorial. I was trying to create an Annotated Time Line chart so I looked up my previous code I wrote for the GWT Portlet JSR-168, see here. I also decided to run a search (don’t be evil, Google is your friend) to try to find different examples and hopefully some nice custom charts. First of all, based on the search result; I thought that some of examples, including Google’s own, were confusing and over complex. Most examples uses AJAXLoader to load the Visualization API, but you shouldn’t have to use this. This is the simplest way to create a GWT Annotated Time Line Chart.

gwt annotated time line chart

1. Using your favorite development tool with support for GWT framework, create a new Java web application, you can export the compiled JavaScript later to a non-based web application, if you use an IDEW you should code-completion support.

2. Make you sure you have included the GWT Visualization API module in your classpath.

3. In your project source code root directory, you should <AppNamexxxx>.gwt.xml file. make sure to add teh following line to it in order to make the  module available to your application.

<inherits name=”com.google.gwt.visualization.Visualization”/>

4. In order to keep this tutorial short and straight to point, I have included the chart creation code in my <AppNamexxxx>EntryPoint.java class

public class MainEntryPoint implements EntryPoint
{
    
    
    /**
    * Creates a new instance of MainEntryPoint
    */
    public MainEntryPoint()
    {
        
        
    }
    
    
    
    /**
    * The entry point method, called automatically by loading a module
    * that declares an implementing class as an entry-point
    */
    public void onModuleLoad()
    {
        
        Runnable onLoadCallback = new Runnable()
        {
            
            
            public void run()
            {
                
                DataTable data = createHistoryTable(result);
                AnnotatedTimeLine.Options options = AnnotatedTimeLine.Options.create();
                options.setDisplayAnnotations(true);
                options.setDisplayZoomButtons(true);
                options.setScaleType(AnnotatedTimeLine.ScaleType.ALLFIXED);
                options.setLegendPosition(AnnotatedTimeLine.AnnotatedLegendPosition.SAME_ROW);
                AnnotatedTimeLine atl = new AnnotatedTimeLine(data, options, “600px”, “200px”);
                RootPanel.get().add(atl);
                
            }
            
            
            
        }      
        ;
        VisualizationUtils.loadVisualizationApi(onLoadCallback, AnnotatedTimeLine.PACKAGE);
        
    }
    
    
    
    // This method will create the Data used by the chart
    private DataTable createHistoryTable()
    {
        
        DataTable data = DataTable.create();
        data.addColumn(AbstractDataTable.ColumnType.DATE, “Date”);
        data.addColumn(AbstractDataTable.ColumnType.NUMBER, “Price”);
        data.addColumn(AbstractDataTable.ColumnType.NUMBER, “Low”);
        data.addColumn(AbstractDataTable.ColumnType.NUMBER, “High”);
        data.addRows(5);
                  
            DateTimeFormat dtf = DateTimeFormat.getFormat(“yyyy-MM-dd”);
            data.setValue(0, 0, dtf.parse(“2009-11-21”));
            data.setValue(0, 1, 100);
            data.setValue(0, 2, 120);
            data.setValue(0, 3, 90);

            data.setValue(1, 0, dtf.parse(“2009-11-22”));
            data.setValue(1, 1, 90);
            data.setValue(1, 2, 110);
            data.setValue(1, 3, 100);

            data.setValue(2, 0, dtf.parse(“2009-11-23”));
            data.setValue(2, 1, 100);
            data.setValue(2, 2, 180);
            data.setValue(2, 3, 80);

            data.setValue(3, 0, dtf.parse(“2009-11-20”));
            data.setValue(3, 1, 130);
            data.setValue(3, 2, 100);
            data.setValue(3, 3, 130);
          

            data.setValue(4, 0, dtf.parse(“2009-11-19”));
            data.setValue(4, 1, 130);
            data.setValue(4, 2, 170);
            data.setValue(4, 3, 150);
            
        }
        return data;  
    }  
}

Don’t forget to import all the necessary classes and voila!

I hope this will save you some time in creating your charts.

 

4099041220_b0ccec7f10

Develop High Transaction Web Applications with Java MySQL & NetBeans

This entry is a brief tutorial on how to develop high transactional web application. I will be looking at how to develop a high transactional application while maintaining data integrityFor the purpose of keeping this entry simple, I will be using some RAD tools, the NetBeans IDE, to generate most of the code.

In order to follow the tutorial, you will need the following:

  1. NetBeans IDE 6.7+
  2. Java JDK 1.6+ (my version is 1.6.0_17)
  3. MySQL 5.1+
  4. MySQL Sakila database
  5. Apache Tomcat 6.x
  6. An understanding of JPA transactions and ReSTful web services

This is my definition of High Transaction Applications:

A high transaction application is one that can serve multiple simultaneous requests from clients and keep them secure from each other. The application has only two purposes: read or write data to/from a repository either JMS or DB. The transactions have to meet the ACID criteria in order to be deployed in the real world.

Ok, the above is my definition and you are free to redefine it. I am going to build a web application that will be an n-tier application:

  • Database back-end (MySQL)
  • Entity classes
  • ReSTful Web Services to allow other developers to integrate the application with theirs
  • A web based front-end

For the simplicity of the article, I will not implement any security such as user or application level security (authentication and database table privileges). This tutorial is mostly geared toward the newbies but I am sure that more advanced developers would benefit too.

Let’s get coding.

  1. Make sure that you have loaded the MySQL Sakila database into your MySQL database. You can download the Sakila database from the NetBeans plugin centre (see here).
  2. Create a new web application and name whatever you like. I have named mine “WebApplication”. I will now refer the application as WebApplication. Make sure to choose Tomcat as your deployment server
  3. Add the MySQL driver “mysql-connector-java-5.0.7-bin” to the WebApplication libraries. NetBeans will work and connect to the DB even without the driver but once you deploy your application to a server, the application will not be able to connect to the DB and throw a ClassNotFoundException com.mysql.jdbc.driver.
  4. We are going to develop the back-end first. NetBeans makes it very easy for us to create Entity classes from database. I would recommend newbies to learn how to manually create Entity classes and configure the persistence.xml file. This tutorial makes use of JPA but one should ignore the drawbacks of JPA/ ORM frameworks.
  5. Right click on the project name and choose Entity Classes from Database… On the next screen choose the “filmActor” table and click on the Add button. Make sure that “Include Related Tables” box is checked underneath the Selected Tables panel. The screenshot does not show the “filmActor” table as I have previously generated the Entity class but I am sure you get the idea.

  6. If you are required to create a Persistence Unit, click on the persistence button -> you can accept the default name -> choose your Persistence Library -> choose your database connection -> choose “None” for Table Generation Strategy -> click Finish
  7. After choosing the table to generate the Entity from click next -> fill Package name -> tick Generate Name Query… -> click Finish
  8.  

    NetBeans generates all the Entity classes based on the database table that you have chosen. The next thing that we want to do is generate a set of ReSTful web services from the generated Entity classes. Again, NetBeans facilitates the work for us (it is important that you also know how to create the classes manually or you will not know how to debug them if there is any problem in the future).

  9. Right click on the project name “WebApplication” -> RESTful Web Services from Entity Classes… -> Choose the Entity Classes that you would like to generate the WS for and click add or add all -> click next -> on the following screen, accept the default values and click Finish

  10. You can go ahead and test your ReSTFul Web Services by right clicking on the name of the application and click on Test RESTful Web Services

  11. The previous step will launch your web browser within which you can test your web services (click on the node on the left and see the queries on the right)

  12. Back in the NetBeans IDE; right click on the project name “WebApplication” -> click on JSF Pages from Entity Classes… (The JSF pages will not use the web services as there are packaged together with the Entity Classes. This will improve performances and still allow external applications to integrate). Choose the Entity Classes that you would like to generate the pages for -> click next. On the final screen, fill in the package name for the JPA Controller and JSF Classes -> click finish

    NetBeans will generate the necessary files to create a CRUD application with a user interface. I suggest that you familiarize yourself with the generated code.

  13. Expand the Configuration folder under your project name “WebApplication” and the web.xml file. At around line 38, change the content of the welcome-file-list to look as follow
    <welcome-file-list>
            <welcome-file>faces/welcomeJSF.jsp</welcome-file>
    </welcome-file-list>

    This will make the generate JSF page to be the landing page for the application when requested. Make sure that you do have a “welcomeJSF.jsp” file before making the change.

  14. Right click on the application name -> Run. The application should load in your web browser. Now go on, play around with the application. And why not create a client to send request to the web services? (not today)

You can load test your application by using Apache JMeter. It is easy to run and configure. If you want to take a look at how JPA implements the ACID features, browse to your controller classes. Here is a short introduction to JPA transaction.

Hope you enjoyed and if you need any clarification, just leave me a comment and try to get back to you ASAP (if time permits).

Develop Your Own Google with Apache Lucene (Java Nutch Solr)

Apache Lucene is Open Source API that allows a Java developer (.Net libraries available) to write indexing and full-text search capable applications. I have been writing applications based on Lucene for the last 3 years and some of the applications have been deployed at large corporations. I know there are other libraries available to developers who wish to write indexing engine but this blog will solely focus on Apache Lucene. I will not compare it to other API.

Lucene is a very mature API and can be found in NetBeans IDE, Liferay, JackRabbit among others. IBM has written a very good document about the Lucene architecture, therefore I will not delve into it here.

Lucene, alone, is pretty much useless as any other API. Let’s now introduce Nutch. Nutch is a web crawler built-on top of Lucene to provide file crawling capabilities. Nutch was designed to handle large amount of data from the internet (http). Due to its plugin architecture, it was later extended to provide local network crawling such as FTP, databases and Microsoft Windows Shares (I am the author of the protocol-smb plugin and co-author of the index-extra plugin found on the Nutch site). We had extended Nutch and turned it into an Enterprise Search application but most of the source codes were locked behind closed doors (company politics).Anyway, Nutch has evolved to become but still very complex in its inner working. The initial Nutch was developed to process data in a batch but there are ways to turn it real-time but that’s for another day. Ok, so Nutch is good for crawling and indexing of data but it does not handle search directly. There is a web application available with Nutch but it is quite poor so let’s now introduce the Solr.

Solr is a powerful web-based search server built-on top of Lucene. The application was developed by CNET Networks and donated to the Apache Foundation. I believe, not too sure so I might need some references here, Solr was powering the search feature on their site but it is definitely used internally by the company. Late 2009, Lucid Imaginations receives $7.5 million in funding to provide commercial services built around Solr (and Lucene possibly?). Here is a very good presentation about Solr. Solr is a very good indexing engine. The keyword here is “indexing engine”. It does not have any support for crawling data therefore requiring the developer to create applications that will feed it the data to index. I do believe that it is a good feature of the application as it gives the ability to integrate with various systems as long as they can post data over HTTP.

Nutch is a good crawler but it does not provide an enterprise-grade search interface to its data. Solr, in the other hand, is powerful indexer and has an enterprise grade search interface but it does not know how to gather data in its own. I am sure by now it has become obvious how we can integrate them both together.

We want Nutch to gather the data, by-pass its indexing cycle and feed the data directly to Solr. Lucid Imagination has a good tutorial about it here.

After reading the tutorial from Lucid Imagination, you will notice that Nutch is run by executing some bash files. This is something I strongly disagree with. If Nutch is based on Java (an OS independent language), why do we need to execute some UNIX/LINUX shell script. Also, the fact we need to install CygWin on MS Windows platform to be able to run is a big negative for me. I wrote a simple Java application that will launch Nutch and send the indexing to Solr but as you can see in the source code, you still need a UNIX like environment to run successfully. You can write a platform independent version by looking up Nutch API and calling the methods directly.

Well, I hope that this entry help you understand how to use Nutch and Solr built-on top of Apache Lucene. If you need any clarification, leave comments and I will try to gave ASAP if time permits.

package com.etapix.nutchsolr;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileFilter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Armel Nene
 */
public class Indexer {

    public static void main(String args[]) {
        if (args.length < 3) {
            System.out.println("Usage:" +
                    "ncrawlName        -   This will be used to store crawler files in CrawlName directory" +
                    "nurlFolder        -   The path to the folder containing the URL to crawl" +
                    "nsolrUrl          -   The URL to the Solr server");
            return;
        }
        String crawlerName = args[0];
        String urlFolder = args[1];
        String solrUrl = args[2];
        String inject = "bash bin/nutch2.sh inject " + crawlerName + "/crawldb " + urlFolder;
        String generate = "bash bin/nutch2.sh generate " + crawlerName + "/crawldb " + crawlerName + "/segments -topN 10 -numFetchers 5";
        String export = crawlerName + "/segments/";

        String invertLinks = "bash bin/nutch2.sh invertlinks " + crawlerName + "/linkdb -dir " + crawlerName + "/segments";
        String indexSolr = "bash bin/nutch2.sh solrindex " + solrUrl + " " + crawlerName + "/crawldb " + crawlerName + "/linkdb " + crawlerName + "/segments/*";
        try {
            System.out.println("Injecting URLs in crawldb");
//            int state = 0;
            InputStream in = Runtime.getRuntime().exec(inject).getInputStream();
            System.out.println(convertStreamToString(in));


//            state = Runtime.getRuntime().exec(inject).waitFor();
//            System.out.println("process completed: " + state);

            for (int i = 0; i < 3; i++) {
                System.out.println("Generating segments");
                in = Runtime.getRuntime().exec(generate).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Setting environment variable $SEGMENT");
//            String segs = convertStreamToString(Runtime.getRuntime().exec("ls -tr " + crawlerName + "/segments|tail -1").getInputStream());

                String segments = export + lastFileModified(export).getName();
                System.out.println("$segments: " + segments);
//            in = Runtime.getRuntime().exec(export + segs).getInputStream();
            System.out.println(convertStreamToString(in));

                String fetch = "bash bin/nutch2.sh fetch " + segments + " -noParsing";
                String parse = "bash bin/nutch2.sh parse " + segments;
                String update = "bash bin/nutch2.sh updatedb " + crawlerName + "/crawldb " + segments + " -filter -normalize";

                System.out.println("fetch segments");
                in = Runtime.getRuntime().exec(fetch).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Parse segments");
                in = Runtime.getRuntime().exec(parse).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Update crawldb");
                in = Runtime.getRuntime().exec(update).getInputStream();
                System.out.println(convertStreamToString(in));
            }
            System.out.println("Inverting links");
            in = Runtime.getRuntime().exec(invertLinks).getInputStream();
            System.out.println(convertStreamToString(in));

            System.out.println("Indexing contents to Solr " + solrUrl);
            in = Runtime.getRuntime().exec(indexSolr).getInputStream();
            System.out.println(convertStreamToString(in));

        } catch (Exception ex) {
            Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    public static String convertStreamToString(InputStream is) {
        /*
         * To convert the InputStream to String we use the BufferedReader.readLine()
         * method. We iterate until the BufferedReader return null which means
         * there's no more data to read. Each line will appended to a StringBuilder
         * and returned as String.
         */
        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        StringBuilder sb = new StringBuilder();

        String line = null;
        System.out.println("Now converting inputstream to text");
        try {
            while ((line = reader.readLine()) != null) {
                sb.append(line + "n");
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                is.close();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        System.out.println("Finish converting to text");
        return sb.toString();
    }

    public static File lastFileModified(String dir) {
        File fl = new File(dir);
        File[] files = fl.listFiles(new FileFilter() {

            public boolean accept(File file) {
                return file.isDirectory();
            }
        });
        long lastMod = Long.MIN_VALUE;
        File choice = null;
        for (File file : files) {
            if (file.lastModified() > lastMod) {
                choice = file;
                lastMod = file.lastModified();
            }
        }
        return choice;
    }
}