primefaces-mobile-weather-app1

Primefaces Mobile – Weather App Example

As a Java developer, I usually get requests of building mobile apps. I like building Java application; enterprise, web and mobile. The latest projects that I have been involved with make heavy use of JSF and Primefaces in particular.

I am quite confortable with JSF and therefore I decided to build a mobile application using JSF and Primefaces Mobile. Primefaces Mobile wraps JQuery as a JSF component so that you do not have to write any Javascript. This approach has huge benefits: JQuery is a well tested framework used by large companies such as Google and Microsoft.

I will make this post quite brief. I wanted to know how easy it would be to recreate the demo from the Primefaces Mobile labs page.

Here is a screenshot of the final application running in Firefox 11 (Windows 7 64 Bit).

This was a simple example taken directly from the website therefore I was expecting to a short exercise to make build and make it run. Well not so fast.

I created a Maven based Java EE 6 project using Netbeans IDE 7.1.1. After browsing Google for a bit, I spent a few minutes getting the right repository and dependency in place.

Once the dependency where in the place, I had to create the beans required for the actual JSF page to work. I found the Primefaces Mobile backing beans on Google Code. So now I had everything setup and running. At first glance, the application seemed to running fine and working. Then I tried the application on my iPad and the Android Emulator, and nothing was working. The user interface was displayed but the “get forecast” button was not making any Ajax calls.

So I started to debug the application everytime I had some spare time. I also noticed that, while running on a desktop browser, the application would be able to an Ajax called and updated the screen with the values (see screenshot above) but you wanted to make another to find out the temparature of let’s say London, the nothing would actually unless you refresh the page and try again. OK, so it’s not working as expected but the example on Primefaces Mobile labs worked fine on my iPad, emulator and desktop browser (IE 9 excluded).

So I ran the application using the NetBeans debugger and decided to look through Firebug. The first call goes through and stops at the breakpoint but subsquent calls do not even get to the managed bean. Firebug shows that the other code are being retrieve from the cache. I manually set all the HTTP headers so that it does not cache any content but this is still the same result NOT WORKING!

I have uploaded the code to GitHub click on the link to download it.

In conclusion, it’s not plug-and-play to make the examples on Primefaces.org labs work. The documentation for the examples are quite poor and I hope that the good folks from Primefaces can take look at my code and tell us what I am doing wrong.

A part from that, Primefaces is a good JSF framework that I use on a daily basis on multiple projects therefore I cannot really put them down but I wished the mobile examples work and tell us what is needed to make it work.

Please share your experience in the comments below or advice on how to fix it. Feel free to download the code from GitHub and have a look to.

Happy coding ūüôā

s-FACEBOOK-POKE-JAIL-large1

Facebook Patents RSS-like Technology – Is That a Joke?

First thing first, Facebook did not patent RSS but their news feed on their site!!! I have to admit, this is a quite neat feature and what would be the point of  a social (networking) site if you did not know what your entourage were upto. So Facebook included a friends activity log (news stream) to make it easy for people to find out. Thats a good feature but it’s by no means an innovative technology. DZone, a developer social “networking” site provides a log about news that users have posted, commented on and rated. This is to show that there’s nothing innovative about that feature but a convenience.

I have been very busy lately and did not have time to check my Google Reader but I did today. And voila! I see the news flash from Mashable – Facebook Secures Patent for News Feed. My first reaction, based on the title before I even read the article, was WOW!!! Facebook managed to patent RSS or ATOM or anything else on that scale. Then came the joke when I started to read the article specially the abstract of the patent:

‚ÄúA method for displaying a news feed in a social network environment is described. The method includes generating news items regarding activities associated with a user of a social network environment and attaching an informational link associated with at least one of the activities, to at least one of the news items, as well as limiting access to the news items to a predetermined set of viewers and assigning an order to the news items. The method further may further include displaying the news items in the assigned order to at least one viewing user of the predetermined set of viewers and dynamically limiting the number of news items displayed.‚ÄĚ

I don’t know about you genius guys but I could not make much sense of the abstract but I did pickup a few keywords such as:

  • News Feed
  • Social Networking

That’s it! Everything else is just pure jibberish. I can summarized their abstract as follow:

We know about RSS/ ATOM which are real news feed. RSS provides real-time (depending on how often you poll) updates; can that be classified as a site master activity update. It also includes links and media about the site or users generating the feeds. That’s a cool feature and we copied the idea and implemented in our site, now we are the biggest community site in the western hemisphere so we are telling not to use any RSS or derivated technology when building a community site (social networking).

Let’s face it, I feel sorry for the smaller community sites but I do not think that patent will affect Twitter or Google but watch out MySpace. I did not even know you could patent someone else already patented technology. This is not a new way of doing something but more like patenting the environment where the technology is applied. To be very geeky about this, Facebook News Feed are just a bunch of SQL queries displaying latest records on a User Interface. I did not know you could even dare trying to patent that. To be frank, I am not sure that they could have patented that in Europe. I am looking forward to seeing how to will now exercise their new POWER. What would happen to Ning, BuddyPress and Elg, those are software and platform used to create social site but what about Youtube and UStream. Anyway site operating within the US and implementing the patent features of Facebook will have to pay them. But outside the US territoty, those company would be free to operate. How to you control access to the Internet without censoring like China and Iran?

Anyway, this is just my opinion but you are welcomed to share yours with the rest of us. For now, why not read about RSS here and see exatly what I am talking about.

happy20101

Happy New Year Techies!!!

I just wanted to say happy new year to the community, without you guys, we would still be living in the dark ages. Here are a few that I am looking forward to in the new year:

  1. Oracle Sun merger:- The untold future of NetBeans, MySQL, Swing and every open source that Sun has been working on in the past. What would happen to Sun Open Source (SOS!!!) Movement.
  2. Looking forward to JavaFX1.3 release, Authoring tool and improved Composer plugin.
  3. Android uprise against iPhone (not because I can easily write Android based application) and hopefully with Visual XML builder to build Android UI. JavaFX running on Android, anybody????
  4.  Looking forward to seeing real-world JavaFX application and possibly a showcase linked to the JavaFX.com site.
  5. Java EE 6 support in Google App Engine
  6. Google Wave open to the public and how well it does against Twitter and Facebook
  7. Java Store open to Europe (well this is where I am based and want to sell applications not just provide free).
  8. My new video blog ( well that’s me trying to be more Armel 2.0 – the sequel) to better engage with the community.
  9. looking forward to networking with fellow developers and techies.
  10. looking forward to the new buzzwords (old technology, new name)

Well, 2009 is was good year for Java and tech scene. I hope that you all enjoyed the year as much as I did. Why not share what wish list with me (comment box). Subscribe to follow my blog as I will try to bring more interesting articles in the new year.

Happy 2010 New Year, wish you all; success, properity, fame (yup) and fortune. Don’t forget, if you need a server-side, UI, Android or JavaFx developer, just mail me.

openid-1

Dummies Guide to GWT and OpenID – with example code

Once (?) in awhile, we developers browse the web for some quick and dirty solutions to some of our coding problems (sometimes not even problems, right?). This is another of those days, everything was going fine on this project I am currently working on. This is not a social networking sites but a site for sharing and comunicating (how I missed the days of the chatrooms) similar in many aspect to Digg and DZone but which also allows users to communicate in realtime and leave comments. So the back-end was sort of completed but so now time to focus on the user registration.

The sites needed to be open and avoid repeatitive tasks such registering users. The way forward was to implement some authentication to allow users to register with some sort of universally accessible ID (or sort of). Facebook Connect, OpenSocial and Google Accounts have their advantages but to me personally; the disadvantages outweight the pros. These are some of the disadvantages of those platform:

  1. Facebook is a popular site hence is its platform, Facebook Connect, in mostly Europe and America but not everybody in Europe and America have a Facebook account.
  2. OpenSocial, when it comes to single logon, has more advantages than Facebook. I supposed that if we do take into consideration all the OpenSocial sites, we might have apossible larger user based than supporting Facebook alone. Even that was too limited.
  3. Google Accounts, one thing Google does not advertise its user base. I could be wrong but do anybody actually knows how many people uses the various Google services (excluding search). Google has the same disadvantages as Facebook.

Enter OpenID. OpenID has been around longer than most Internet-based decentralised authentication platform. The beauty of this platform is that it also supported by most (if not all) large site on the Internet. From AOL, BBC, CNN to YAHOO and ZIMBRA, as I said most sites (based on the alphabet) support OpenID, check the OpenID Directory. It was recently announced that OpenID has reached 1Billion enabled accounts (read here). Based on those figures, I decided that OpenID for now was the authentication choice for this application. I will not be discussing any security issues in this entry, there are plenty of resources available on the Internet for that.

OK, so I am a Java developer and I am developing the front-end using GWT and Java on the server-side. I searched on the Internet for solutions on how to implement the authentication as GWT RPC are different to normal servlet call. I spent more time reading about the OpenID specification and implementations examples on the Internet. I have to admit that some of tutorials that I found on the Internet were somehow confusing and not helpful at all. Therefore I decided to write my own Dummies Guide to GWT and OpenID.

Dummies Guide to GWT and OpenID

First of all, it is important to know how OpenID works (please check the OpenID site for more info).

In a nutshell, OpenID allows you to authenticate with website (supporting the standard) with just an URL and voila. Your URL has to be hosted by an OpenID provider in order for it to work. For example, my blog is hosted by Google on blogspot.com. Google supports OpenID authentication therefore if a reader wanted to leave a comment on my blog, he does not have to have a Google Account as he can log-in with his Yahoo or AOL or Facebook or WordPress or any other OpenID provider sites, and then leave a comment, that simple.

GWT to OpenID and back

There are two ways to authenticate a user with an OpenID provider and GWT supports both. When authenticating a user, the relaying site (the site the user is trying to access) redirects to user to the OpenID provider (i.e. Google) login page.

The problem will be with the GWT RPC mechanism. GWT RPC calls are asynchronous and you cannot make any redirections. Therefore we need a way to execute the redirection from the client side, here is the code (I use OpenID for Java to handle the openID discovery from the RPC servlet), I then used USER object (just a simple POJO which only stores the redirection URL and the parameters) to be sent back and forth between the front-end and back-end.

#######################################################
public User authenticateOpenId(String provider_url) {
    try {
        ConsumerManager manager = new ConsumerManager();
        // This is the URL where the OpenID provider will redirect the user
        // once logged in.
        String returnToUrl = “http://localhost:8084/GwtOpenId“;
       
        // the provider URL will be used to identify the user OpenID provider
        List discoveries = manager.discover(provider_url);
        DiscoveryInformation discovered = manager.associate(discoveries);
        // OpenID 4 Java needs to  have an HttpServletRequest object, GWT sevlet 
have
        // convenient methods to retrieve the HttpServletRequest object 
and manipulate its
        // parameters
        this.getThreadLocalRequest().setAttribute(“openid-disc”, discovered);
        this.getThreadLocalRequest().setAttribute(“openid.mode”, “popup”);
       
        AuthRequest authReq = manager.authenticate(discovered, returnToUrl);
        FetchRequest fetch = FetchRequest.createFetchRequest();
        // I want to exchange the following attributes from the OpenID provider
        // remember that teh attribute will be returned only if it exits
        fetch.addAttribute(“email”,”http://schema.openid.net/contact/email“,true);
        authReq.addExtension(fetch);
        // Simple POJO to persist the data
        User user = new User();
        // In a normal servlet development, the following statement would make
        // a redirect call but this would not work when using GWT RPC
        if(!discovered.isVersion2()){
            user.setRedirect(authReq.getDestinationUrl(true));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
            } else{
            user.setParams(authReq.getParameterMap());
            user.setRedirect(authReq.getDestinationUrl(false));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
        }
        } catch (MessageException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (DiscoveryException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (ConsumerException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
    }
}
#############################################################

The above codes will format the request in order for the front-end to execute a redirect and allow the user to authenticate with his OpenID provider. Now here is the front-end code which executes the authentication and reads the data back.

#################################################################
// This is an extract from the openidEntryPoint.java

// This is where the magic happens – This code is only usefull when the OpenID 
provider
// redirects the user back to your site – please visit openid.net for valid 
parameters.
// The “if” statement checks to make sure that it is a valid response from 
the OpenID
// provider – You can do anything you want with the results here such 
as verifying the
// response with the server-side code
if(Window.Location.getParameter(“openid.rpnonce”) != null){
    String s = Window.Location.getParameter(“openid.mode”);
    // executes this if the user cancels the authentication process and the OpenID
    // providers returns to the your site
    if(s.equals(“cancel”)){
        sign.setText(“Server returned openid.mode=cancel”);
        openIdUrl.setText(“You need to Accept not Cancel”);
    }
    // Here, I am assuming that everything is fine and that the user has 
been sucessfully logged in
    else{
        sign.setText(“You have successfully signed in”);
        vp.setVisible(false);
    }
   
}
RootPanel.get().add(contentPanel);

}

private class UserAsyncCallback implements AsyncCallback<User> {
   
    public void onFailure(Throwable caught) {
        Window.alert(caught.getLocalizedMessage());
    }
   
    public void onSuccess(User result) {
        if (result != null) {
            //                    Window.open(result.getRedirect(), “_blank”,
 “height=200,width=400,left=100,” +
            //                            “top=100,resizable=no,scrollbars=no,
toolbar=no,status=yes”);
            // this the most important line in order to make the authentication.
 Here, I am redirecting the user
            // from the client side to the OpenID provider URL with the discovery
 data generated from the
            // RPC call to the servlet.
            Window.Location.assign(result.getRedirect());
            } else {
            Window.alert(“Ooops!!! Couldn’t find your provider”);
        }
    }
}
#######################################################################################################

I have attached the full NetBeans project with dependencies. The code is provided as-is and use at your own risk ;). Here is a screenshot of the working application:

Step1: Authenticate on the site (enter the URL)
moz-screenshot-6

Step2: Redirect to OpenID provider (Google is my provider ūüôā ), authenticate with your provider
moz-screenshot-7

Step3: Allow the application to access your OpenID details and redirect back to the original site
moz-screenshot-8

Step4: final step, check the parameters from the provider and proceeds accordingly
moz-screenshot-9

Take a look at the URL in each of the above step to see the OpenID data. OK, so my code actually works (yuppie), now you know that it is possible form GWT to OpenID and it’s not as complicated as many other sites are trying to show. The code is just for authentication but once authenticated, you can retrieve any parameters that you need. In this example, the query is sent through the browser URL (GET) but you can change it to be encoded in a form submit action. Iwrote some of the code to allow the user to authenticate via a popup window, the code is not complete and maybe someone else might want to have a go. My only problem is getting the redirect back to the original window but apart from that it works.

Download the source code

I hope you guys enjoyed it and Happy Coding.

4193869276_1c43c12a9a

Firefox IE8 Chrome Opera Safari – best browser for Java?

Initially, I was not going to write up about browsers until I had to download all the known web browsers to test a project front-end. I already had Firefox, IE8 and Chrome installed therefore all I had to do was to download and install Opera and Safari. There might be other browsers outthere but this what I am working with for the moment. So what’s the problem? You might ask. Well, I decided to see how simple it would be for a normal user without much computer experience to view a Java-enabled application embedded in a web app or simple HTML page.

Before I go on about what I think, first let me give you an idea of the PC I am currently using:

  • Dell PC with a dual 22″ screen (customised version of Inspiron 530)
  • 4Gb of RAM
  • Intel Pentium Dual 1.80GHz (looking to upgrade this after xmas)
  • Windows Vista Ultimate (how rubbish is that?)
  • Java RE version 1.6.0_17
  • and a few other hardware and software.

So as you can see, my personal PC is quite simple in settings. Ok, for some reasons, I decided to launch all the browsers at once and load the JavaFx homepage (http://www.javafx.com) and all I wanted to do was to compare which one loaded the fastest. Here are my findings, in brief:

Firefox
This is probably the best browser outthere but lately it has been crashing on me. And btw, it does not cope well with Google Wave. Anyway, here is what Firefox gave (see screenshot):

As you can see on the above screenshot, the JavaFx homepage loads in its entirety. To be quite honest, the page was loaded without any fuss but bear in mind, I already had Java installed. The “Demos & Samples” box in blue requires Java to be installed as it a JavaFx applet. As I already have the JRE installed, the browser did not ask me for any plug-ins ( I could be wrong feel free to share your experience).

Internet Explorer 8

I am not a great fan of IE in general, but for this simple test; IE8 loaded the page faster than Firefox. Also, it did not ask me to install a specific plugin in order to load the JavaFx portion of the page. IE handles Google Wave much better than Firefox, so this is the browser I use to interact on the Wave.

Google Chrome

Google Chrome, like Firefox and IE, did not cause any problem and it was the fastest at loading the JavaFx page including the Java applet. So far, it is the best performing browser but it is not my personal choice so I rarely use it. Also, I like the fact that Chrome scales the page to fit nicely in its window.

Opera

As you can see in the above screenshot, it seems as if the applet was not fully loaded. Indeed, Opera has loaded some portions of the JavaFx applet but I do not know what is happening here now. This page has been the same since I loaded the browser until now. Again, Opera did not ask me for any plugins to be installed nor did it ask for anything else. I suppose this is not an issue from Sun ūüėČ but something that the Opera team should look at. I know Opera can find the JRE on my system (see screenshot below) but it seems not to be a big fan of Java. In the screenshot below, you can see the JavaFx splash screen but this is all you will see and nothing else. Off-subject, I really like the user interface and I might start using it for a bit longer just to see if it can convert me.

Safari

Ha! what the hell happened here. I have my JRE installed, the previous four browsers did not ask me to install Java (again!!!). The demo applet did not even load, this is Safari running on Windows machine not Mac. So I did what it has asked me for the thousandth time. I clicked on the provided link, takes me to the Java RE download page and I did all the required and look!!!!

Java!!!!! This is the confirmation that I already have the software installed so what’s the big idea? The Apple team are not doing a good job to support Java but I can browse to Youtube and watch video without installing Flash player (it recognises that it’s already installed). Well I do not think I will be using this browser again anytime soon.

Final words
When it comes to browser to support, you have to test your application on multiple browser ( and OS platform too) to make sure it will not affect the expected user experience that we are so accustomed to.

I do not know why Opera and Safari have issues with the JavaFx site, it could be anything from the site designers to the respective browsers’ team. I believe that Java is probably the best Cloud computing platform currently outthere (when it comes to high performance applications) and that browser providers should make sure that at least they do support the platform.

I have a question:

Who should be to blame if Java or any other plugins such Flash is not supported in the browser? The plug-ins developer or the browser developer?

I believe the plug-ins developer should be to blame as they can be a million plug-ins on the net and you cannot possibly cater for all of them. But on the other hand, as in Java ‘s case, the plug-in is supported without any major issues by the top 3 which accounts for 93.9% percent of the market (based on November 2009), should you as the plugin developer care? I say yes at least for the Mac users’ sake. We can overlook opera for now but I am sure that all the Apple fans still use Safari as their prefered browser. Do not try to leave them out, they brought us the iPhone.

Anyway, based on my simple test, I think the best browser for Java applet is (in order of best performance):

  1. Google Chrome
  2. Internet Explorer (make sure you do support this browser as a priority)
  3. Mozilla Firefox (this should also be supported right after IE)

The other two browsers are not included as they did not even successfully launch the applet. I am not saying that you guys are going to have the exact same issue as me and therefore looking forward to your comments.

4190189872_6bb23523e5

JavaFX Composer RAD Tool – First Review

Just over a week ago, Sun announced a RAD tool for JavaFx built on the Matisse framework, I believe. I was very critical of JavaFx for its lack of tool for building UI and I think this is a step in the right direction. The tool was made available through the NetBeans Update Center on the 14/12/2009. OK, so I have installed the plugin and here are my views; not just on the tools but also on other stuff I think would benefit the JavaFx community:

When comparing something, it only makes sense when we use a benchmark; here my benchmark will be Adobe Flash Builder (formerly known as Flex Builder). Over the years, Adobe had made it easy for the designer to build impressive user interface with minimal coding. Sun, in the other hand, made it easier for developers to build application, yes I am aware of some nice UI in Java but they still do not compare to the eye candy of Flash/ Flex.

I am going to look at the tools; Flash Builder and JavaFx Composer plugin, from a developer perspective.

Components:
A key feature of RAD tools is the amount of components they make available to developers without having to write too many codes. I understand the plugin is at a “preview” stage, whatever that mean (alpha?), but there are alot of missing components; as an example, this release version was meant to be a “preview” of what to look forward to but I cannot drag a “combox” from the components palette into my form, no data grid, no chart, no menu bar, no date components and can’t even draw a rectangle which is possible but only through coding. I hope the JavaFx team add all the components available in JavaFx plugin to the Composer.

moz-screenshot-2NetBeans JavaFx UI Composer

moz-screenshot-4Adobe Flex/ Flash Builder

Round-trip code update
One thing I dislike with Java IDE’s (or most of them) is the inability to change the generated code without requiring you to write more code. The Adobe team actually made a good job in giving more freedom to the developer. In Flash Builder, you can design your UI through drag-n-drop but also customise it directly through the XML file (MXML). This feature was not available in NetBeans Matisse, I could be wrong, but and again Matisse was not really used in large project (no comments, thanks), at least not were I worked. Why all the fuss, you might ask? Try to create a simple interface and add a “rectangle” object to it which you will use as a toolbar and tell me how simple that was.

The coding style in JavaFx is very similar to Flex/ Flash ActionScript (so why the “V”oid instead of void, but that’s another issue) and very easy to learn. So far, I found it easier to actually code the canvas then using the Composer plugin. Another thing, when inspecting object properties; not all properties are available, for example: the gradient properties are not available, which will require you to write more lines of code.

UI Preview Panel
Again, a feature which Adobe Flex/ Flash builder excels at (I feel like I am starting to sound like an Adobe salesman) is the synchronize preview of codes. This is not a due to JavaFx Composer plugin but this seems as a bug as sometimes, you might have to restart the IDE in order for the UI Preview panel to start working again. Hopefully the introduction of the RAD (or not so much RAD) tool, will fix this issue.

Conclusion
You might feel that I was on a JavaFx bashing quest but this is not the case. JavaFx might not have a large components set (well what about all the SWING components available which you can use? You might not be able to “skin” them to your application look and feel but they still availabe to you) but I still think it has a good future. If you take a look at the screenshot below which was built with Adobe Flex, it took me less than five (5) minutes to build. Now time to synchronize your watches and tell me how fast it will take you to build the same interface using JavaFx Composer plugin. It will probably take me less than five (5) minutes if I was designing it with Matisse. My point is; a RAD tool is supposed to promote productivity and YES!!! I have realized this is a “PREVIEW” release but can you actually use it? I know I will still be coding JavaFx for the foreseable future and I would love it to succeed. If you are going to call a tool “a RAD tool for building Form-based JavaFx UI” then I suggest that you provide most of the form components.

Should JavaFx UI RAD tools be based on XML like Android and Adobe Flash Builder?

moz-screenshot-5

5 components I like to have on JavaFx

  1. Menu and tool bar (come on guys, this was there in previous release)
  2. Grids (even just a simple table will do. For now I use JTable)
  3. Date picker, Rich Text and Navigator components
  4. HTML panel (something that display HTML and can also be used as iFrames)
  5. Panels similar to JInternalFrame (this will be useful in portlet-like applications)

I know it’s not really 5 things but they will make a difference. Thanks for reading and tell me what you think about JavaFx and its UI Composer.

pixy

JavaFX vs GWT – A developer dilemma in building a RIA application

For the past few days, I have been working on a simple proof of concept; a web based stock streaming application with a community theme to it. I set myself to building the backend first. As you can guess, I am developing in Java and here are some of the requirements that I have to meet, at least:

  1. The application shall allow the user to query a stock (quotes) either using the stock ticker or the company name and display the following: stream stock price with basic info, stock historical price; tabular view and financial chart analysis.
  2. The ability to rate, comments and share stock information.
  3. provide a real-time chat features for users of the application to interact together.
  4. single sign-on integration with OpenSocial, OpenID and Facebook connect.

The above are some basic features in order for people to actually use the site. As you can see, they are mostly front-end requirements. Here is another important requirement:

  • The application will be hosted on a Cloud-based service, the application shall keep to a minimum the load on the server as CPU and memory use will be charged per request.

I believe that last requirement was the key decision factor. I needed something that will make use of the user resources instead of stressing the server. Something that would execute on the browser, client-side. Being a Java developer, it is only normal for me to look for something that I can grasp easily and freely available on the market. I looked at a few RIA frameworks and tools but I needed something that can integrate within my development environment without the need switch between IDEs.

I had a look at JavaFX and GWT. Having worked on GWT in the past, I found that I could implement most of the requirements. GWT has a rich api and components sets (widgets). I could use the Google search, map, visualization… to build the front-end and implement the client side code.

It took me about two days to implement the basic front-end and back-end features; mostly concentrating requirements 1 and the last. But then I hit a brick wall, something that all Javascript developers already knew but I just came across:

The application is a mashup of different services from around the net. Some of those services were available for free and others were not available at all but I will still be connecting and parsing the relevant data to displayed on the screen. As requirement “1” was working fine, there was a disaster about to happen. Here is the basic architecture:

  • when a user queries a stock, the data is sent to the back-end services.
  • the back-end services queries a third-party web service which I do not have control over. ZERO.
  • when the third-party web service returns, the data is then parsed into a format understood by the front-end and sent to the front-end services to display.¬†

The services was working fine until I had about 12 stocks displayed on my screen for testing purpose. The data was being streamed with a 1 minute interval. After about 30minutes later (and 360 requests = 12 *1 pm * 36m), I have noticed that the stock data was not being updated anymore and some screens were blank. No errors were logged and some other queries were still working such as the historical data. As I could not see the errrors from my debugging session, I therefore decided to query the third-party service manually through Firefox and IE and VOILA! The service provider had put a temporary ban on my IP address for making too many requests, it says. This was a major blocker as it seems that I could not access my primary data anymore. This left me questioning the feasibility of the project.

If a single user gets banned for tracking 12 stocks, what would happen when 100 or more users start watching their stocks?

OK, so I did not give up, at least not without a fight!

I looked at the problem from another perspective, the third-party view. How do they track my request as I do not have to register or sign-in? The obvious answer is my IP. So I thought, instead of making the request to their services from the back-end, I should send the request directly from the front-end. If each users make the same request, the third-party services would register different IP addresses for each one of them. Correct indeed but one big problem:

The front-end code is written in GWT which is then compiled to Javascript. Being a Java developer, I was not aware of the limitation of using AJAX in a mashup such as mine. After making the necessary changes, I then run the app just to get hit with another blocker – SOP (Javascript Same Origin Policy). What’s in a monkey name was this all about? So after researching it – Google and Wikipedia, I knew what I was dealing with and I am sure all Javascript developers already knew that but not me :(. I never give up, it is not part of my nature, therefore I decided to look for ways around it and came across a tutorial on GWT site that shows you how to implement a workaround. Did the tutorial, implemented some basic methods and that worked fine but implementing the third-party service, did not work :(. The problem was simple, the workaround needs to be in JSON format which I coded but the third-party service needs to be able to send the request back to a “callback” method. This is it!!! Remember that I mentioned that I had ZERO control over the thrid-party service and therefore I could not implement this feature.

Remember, I never give up; when there’s a will there’s a way. So I hit a major blocker due to the technology, in this case it was Javascript. I know that if I was working on a pure java front-end GUI, this would have never occurred and Flex allows you to query other third-party services but I have no knowledge of ActionScript. I decided to go for JavaFX just because it is Java and allows me to connect and get my data (actually it’s not mine). My options were Webstart, Applet and JavaFX. First of all, it has to be part of the site so my real options were Applet and JavaFX. Applet do not have a rich set of tools such as Map, Chart and everything else that JavaFX provides and I do not want to waste time coding them.

So same adventure but different episode; I am currently looking to implement the JavaFX UI so stay tune for the sequel and screenshot. I could have never anticipated those problems with the third-party and then Javascript but believe me it was a good experience and my Javascript just went +1.

4124677046_69aa7a07b3

GWT Visualization Example – Annotated Time Line Chart Tutorial

If you are a server-side Java developer such as myself, I am sure you want an easy way to create AJAX based application but pure JavaScript (ECMAScript) is not your strong skills. The beauty of JavaScript is that it runs on any web server, no need for a servlet container or anything of that sort.

So, I have been writing AJAX based UI using the GWT framework for the past year and half. I am still not a web developer but I understand enough CSS to spice up my site. Anyway, I am writing this tutorial because I think that the GWT Visualization team, even tough they did a good job, over complicates the creation of charts in their tutorial. I was trying to create an Annotated Time Line chart so I looked up my previous code I wrote for the GWT Portlet JSR-168, see here. I also decided to run a search (don’t be evil, Google is your friend) to try to find different examples and hopefully some nice custom charts. First of all, based on the search result; I thought that some of examples, including Google’s own, were confusing and over complex. Most examples uses AJAXLoader to load the Visualization API, but you shouldn’t have to use this. This is the simplest way to create a GWT Annotated Time Line Chart.

gwt annotated time line chart

1. Using your favorite development tool with support for GWT framework, create a new Java web application, you can export the compiled JavaScript later to a non-based web application, if you use an IDEW you should code-completion support.

2. Make you sure you have included the GWT Visualization API module in your classpath.

3. In your project source code root directory, you should <AppNamexxxx>.gwt.xml file. make sure to add teh following line to it in order to make the  module available to your application.

<inherits name=”com.google.gwt.visualization.Visualization”/>

4. In order to keep this tutorial short and straight to point, I have included the chart creation code in my <AppNamexxxx>EntryPoint.java class

public class MainEntryPoint implements EntryPoint
{
    
    
    /**
    * Creates a new instance of MainEntryPoint
    */
    public MainEntryPoint()
    {
        
        
    }
    
    
    
    /**
    * The entry point method, called automatically by loading a module
    * that declares an implementing class as an entry-point
    */
    public void onModuleLoad()
    {
        
        Runnable onLoadCallback = new Runnable()
        {
            
            
            public void run()
            {
                
                DataTable data = createHistoryTable(result);
                AnnotatedTimeLine.Options options = AnnotatedTimeLine.Options.create();
                options.setDisplayAnnotations(true);
                options.setDisplayZoomButtons(true);
                options.setScaleType(AnnotatedTimeLine.ScaleType.ALLFIXED);
                options.setLegendPosition(AnnotatedTimeLine.AnnotatedLegendPosition.SAME_ROW);
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†AnnotatedTimeLine¬†atl¬†=¬†new¬†AnnotatedTimeLine(data,¬†options,¬†“600px”,¬†“200px”);
                RootPanel.get().add(atl);
                
            }
            
            
            
        }      
        ;
        VisualizationUtils.loadVisualizationApi(onLoadCallback, AnnotatedTimeLine.PACKAGE);
        
    }
    
    
    
    // This method will create the Data used by the chart
    private DataTable createHistoryTable()
    {
        
        DataTable data = DataTable.create();
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.DATE,¬†“Date”);
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.NUMBER,¬†“Price”);
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.NUMBER,¬†“Low”);
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.NUMBER,¬†“High”);
        data.addRows(5);
                  
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†DateTimeFormat¬†dtf¬†=¬†DateTimeFormat.getFormat(“yyyy-MM-dd”);
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(0,¬†0,¬†dtf.parse(“2009-11-21”));
            data.setValue(0, 1, 100);
            data.setValue(0, 2, 120);
            data.setValue(0, 3, 90);

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(1,¬†0,¬†dtf.parse(“2009-11-22”));
            data.setValue(1, 1, 90);
            data.setValue(1, 2, 110);
            data.setValue(1, 3, 100);

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(2,¬†0,¬†dtf.parse(“2009-11-23”));
            data.setValue(2, 1, 100);
            data.setValue(2, 2, 180);
            data.setValue(2, 3, 80);

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(3,¬†0,¬†dtf.parse(“2009-11-20”));
            data.setValue(3, 1, 130);
            data.setValue(3, 2, 100);
            data.setValue(3, 3, 130);
          

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(4,¬†0,¬†dtf.parse(“2009-11-19”));
            data.setValue(4, 1, 130);
            data.setValue(4, 2, 170);
            data.setValue(4, 3, 150);
            
        }
        return data;  
    }  
}

Don’t forget to import all the necessary classes and voila!

I hope this will save you some time in creating your charts.

 

Develop Your Own Google with Apache Lucene (Java Nutch Solr)

Apache Lucene is Open Source API that allows a Java developer (.Net libraries available) to write indexing and full-text search capable applications. I have been writing applications based on Lucene for the last 3 years and some of the applications have been deployed at large corporations. I know there are other libraries available to developers who wish to write indexing engine but this blog will solely focus on Apache Lucene. I will not compare it to other API.

Lucene is a very mature API and can be found in NetBeans IDE, Liferay, JackRabbit among others. IBM has written a very good document about the Lucene architecture, therefore I will not delve into it here.

Lucene, alone, is pretty much useless as any other API. Let’s now introduce Nutch. Nutch is a web crawler built-on top of Lucene to provide file crawling capabilities. Nutch was designed to handle large amount of data from the internet (http). Due to its plugin architecture, it was later extended to provide local network crawling such as FTP, databases and Microsoft Windows Shares (I am the author of the protocol-smb plugin and co-author of the index-extra plugin found on the Nutch site). We had extended Nutch and turned it into an Enterprise Search application but most of the source codes were locked behind closed doors (company politics).Anyway, Nutch has evolved to become but still very complex in its inner working. The initial Nutch was developed to process data in a batch but there are ways to turn it real-time but that’s for another day. Ok, so Nutch is good for crawling and indexing of data but it does not handle search directly. There is a web application available with Nutch but it is quite poor so let’s now introduce the Solr.

Solr is a powerful web-based search server built-on top of Lucene. The application was developed by CNET Networks and donated to the Apache Foundation. I believe, not too sure so I might need some references here, Solr was powering the search feature on their site but it is definitely used internally by the company. Late 2009, Lucid Imaginations receives $7.5 million in funding to provide commercial services built around Solr (and Lucene possibly?). Here is a very good presentation about Solr. Solr is a very good indexing engine. The keyword here is “indexing engine”. It does not have any support for crawling data therefore requiring the developer to create applications that will feed it the data to index. I do believe that it is a good feature of the application as it gives the ability to integrate with various systems as long as they can post data over HTTP.

Nutch is a good crawler but it does not provide an enterprise-grade search interface to its data. Solr, in the other hand, is powerful indexer and has an enterprise grade search interface but it does not know how to gather data in its own. I am sure by now it has become obvious how we can integrate them both together.

We want Nutch to gather the data, by-pass its indexing cycle and feed the data directly to Solr. Lucid Imagination has a good tutorial about it here.

After reading the tutorial from Lucid Imagination, you will notice that Nutch is run by executing some bash files. This is something I strongly disagree with. If Nutch is based on Java (an OS independent language), why do we need to execute some UNIX/LINUX shell script. Also, the fact we need to install CygWin on MS Windows platform to be able to run is a big negative for me. I wrote a simple Java application that will launch Nutch and send the indexing to Solr but as you can see in the source code, you still need a UNIX like environment to run successfully. You can write a platform independent version by looking up Nutch API and calling the methods directly.

Well, I hope that this entry help you understand how to use Nutch and Solr built-on top of Apache Lucene. If you need any clarification, leave comments and I will try to gave ASAP if time permits.

package com.etapix.nutchsolr;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileFilter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Armel Nene
 */
public class Indexer {

    public static void main(String args[]) {
        if (args.length < 3) {
            System.out.println("Usage:" +
                    "ncrawlName        -   This will be used to store crawler files in CrawlName directory" +
                    "nurlFolder        -   The path to the folder containing the URL to crawl" +
                    "nsolrUrl          -   The URL to the Solr server");
            return;
        }
        String crawlerName = args[0];
        String urlFolder = args[1];
        String solrUrl = args[2];
        String inject = "bash bin/nutch2.sh inject " + crawlerName + "/crawldb " + urlFolder;
        String generate = "bash bin/nutch2.sh generate " + crawlerName + "/crawldb " + crawlerName + "/segments -topN 10 -numFetchers 5";
        String export = crawlerName + "/segments/";

        String invertLinks = "bash bin/nutch2.sh invertlinks " + crawlerName + "/linkdb -dir " + crawlerName + "/segments";
        String indexSolr = "bash bin/nutch2.sh solrindex " + solrUrl + " " + crawlerName + "/crawldb " + crawlerName + "/linkdb " + crawlerName + "/segments/*";
        try {
            System.out.println("Injecting URLs in crawldb");
//            int state = 0;
            InputStream in = Runtime.getRuntime().exec(inject).getInputStream();
            System.out.println(convertStreamToString(in));


//            state = Runtime.getRuntime().exec(inject).waitFor();
//            System.out.println("process completed: " + state);

            for (int i = 0; i < 3; i++) {
                System.out.println("Generating segments");
                in = Runtime.getRuntime().exec(generate).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Setting environment variable $SEGMENT");
//            String segs = convertStreamToString(Runtime.getRuntime().exec("ls -tr " + crawlerName + "/segments|tail -1").getInputStream());

                String segments = export + lastFileModified(export).getName();
                System.out.println("$segments: " + segments);
//            in = Runtime.getRuntime().exec(export + segs).getInputStream();
            System.out.println(convertStreamToString(in));

                String fetch = "bash bin/nutch2.sh fetch " + segments + " -noParsing";
                String parse = "bash bin/nutch2.sh parse " + segments;
                String update = "bash bin/nutch2.sh updatedb " + crawlerName + "/crawldb " + segments + " -filter -normalize";

                System.out.println("fetch segments");
                in = Runtime.getRuntime().exec(fetch).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Parse segments");
                in = Runtime.getRuntime().exec(parse).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Update crawldb");
                in = Runtime.getRuntime().exec(update).getInputStream();
                System.out.println(convertStreamToString(in));
            }
            System.out.println("Inverting links");
            in = Runtime.getRuntime().exec(invertLinks).getInputStream();
            System.out.println(convertStreamToString(in));

            System.out.println("Indexing contents to Solr " + solrUrl);
            in = Runtime.getRuntime().exec(indexSolr).getInputStream();
            System.out.println(convertStreamToString(in));

        } catch (Exception ex) {
            Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    public static String convertStreamToString(InputStream is) {
        /*
         * To convert the InputStream to String we use the BufferedReader.readLine()
         * method. We iterate until the BufferedReader return null which means
         * there's no more data to read. Each line will appended to a StringBuilder
         * and returned as String.
         */
        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        StringBuilder sb = new StringBuilder();

        String line = null;
        System.out.println("Now converting inputstream to text");
        try {
            while ((line = reader.readLine()) != null) {
                sb.append(line + "n");
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                is.close();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        System.out.println("Finish converting to text");
        return sb.toString();
    }

    public static File lastFileModified(String dir) {
        File fl = new File(dir);
        File[] files = fl.listFiles(new FileFilter() {

            public boolean accept(File file) {
                return file.isDirectory();
            }
        });
        long lastMod = Long.MIN_VALUE;
        File choice = null;
        for (File file : files) {
            if (file.lastModified() > lastMod) {
                choice = file;
                lastMod = file.lastModified();
            }
        }
        return choice;
    }
}

Top-10

10 Things all Java developers should know

Since Java (I know it’s not an acronym, but it stands out like that) was officially introduced in 1995, it has changed the way most of us look at the Operating System. Bill Gate (how ironic) once said that it was not about the hardware but the software which will be the future. A decade or more later, the fifth employee of SUN, John Gage said “The Network is the Computer”. Fast-forwarding to the 21st century and John seemed to be right. Anyway, Java was built not to depend on an Operating System and deployed through the network. Java through its applet technology gave birth to Rich Network Application aka Rich Internet Application (RIA). Java is not perfect; or we would not have various releases and more on the way, but Java has given birth to a wide range of programming language (just Google it to find out more).

Without further ado, I am going to get back to subject. This is a brief article on what I believe that every Java developers should know regardless of their experience. I do not personally believe that someone with 5 years experience is not as good as someone with 10 years experience. We all develop our own methods of working but as a developer you need to stay abreast of your technology. So, here are my top 10 not in order of importance (or?):

  1. Remember the basic of Java language and OOP paradigm.
    Most experience developers seem to forget the theory behind the language. I am not saying that they are not good at their job but can they explain to junior developers why they have used interfaces instead of abstract classes or why implement a pattern over another one? As a programmer, you become very arrogant as you believe that you write the best code but in the real world, people work in teams with different skill set and experiences. It is important that you can backup your actions/ codes. A very simple question such as; when should I use a String object instead of a StringBuilder/ StringBuffer? You might take this question lightly but can you actually tell someone else the difference?
  2. Know your technology stack
    All developers have to know their technology stack. What does it mean? Java is not like other languages; Java has subsets such J2ME and superset such as Java EE. We have our own area of expertise but it is important to know the differences between the various sets of Java. Some basic questions such as the differences between SWING, Applet, Servlets, EJBs and JavaFX will boost your confidence. Most developers do not know how to tweak the JVM and the differences between the JRE and the SDK environment. Do you know why you need the SDK to be installed to run Tomcat but you only need the JRE to run an application?
  3. Experiment with various Java EE framework
    I am not asking you to be an expert in every single Java EE framework but it will make the difference if you are familiar with Spring and EJB. That should actually be the de facto framework that should be on every developers CV. Developers should know the difference between Java EE 5 (soon 6) and Spring. Hibernate is also brilliant and it’s used for data access but all developers should have moved to JPA by now. Hibernate also comply with JPA therefore there is no more excuses.
  4. Know a scripting language
    Java can be heavyweight for some simple tasks which can be implemented using a simple dynamic language such as Python, Perl(?) and others. I would also recommend to developers to learn shell scripting on their target OS.
  5. Know how to develop web services
    The network is the computer, therefore it is important to know the different web services framework available. Data are integrated through web services and opening your services to the “cloud”. SWING developers will probably not develop web services but I am sure that they will be connecting to data through web services clients. Understanding the difference between the standardised SOAP and non-standardised ReST will help you choose which is best to implement your services.
  6. Know how and when to multithread your application
    I have to put that in there. Developers should know when and why to multithread an application, thread inter-communication and monitoring. All developers, junior or not, should know how to write a multi-threaded application.
  7. Database development using JDBC and JPA
    This should be a development law. All developers should know how to write SQL queries and how to create databases. All enterprise applications store data in some sort of relational database systems and it is therefore imperative that this knowledge should be of second nature. Java EE 5 introduced JPA (JDO was there before) but it is not applicable to all situation. Hence, knowing the difference and when to implement one instead of the other is important.
  8. Know a client side scripting language and what is AJAX
    The network is the computer and Internet is the deployment platform. Java EE and its various framework are server side executiong which can put extra “load” on the server. If you are looking to move a cloud based system where the providers charges you per resources used, it might be wise to move some of the execution to the client side. AJAX has been buzzing the scene for the last 3 years and more. This is not a technology but a new way of doing something that already existed. There are numerous Java AJAX framework such as GWT and DWR which makes it easy to develop AJAX based application which are compiled to JavaScript. Developers should also know what is the AJAX theories.
  9. Know your competitors and do not take part on “what is the best IDE” discussion
    Java is not the only language that can do what it does. I think that Java is more mature and complete as opposed to other languages. Knowing the difference between Java and .NET or Java and Ruby is a good asset to have. You also need to know when and why to use one instead of the other. Please please please, do not get into “my IDE is better than x because…” discussion as it is good for the Java community to have multiple IDEs and framework available to use. Every tools have their place as for example JDeveloper is better than x if you are going to solely develop on an Oracle stack and etc…
  10. Know ANT (MAVEN?), TOMCAT and any other mainstrean application server
    ANT is the de facto build script for Java and its IDE-based development. Maven is becoming popular and soon can be as popular as ANT (not sure of its popularity in the financial sector). TOMCAT  should be immortalised as the based servlet container that all developers should be familiar with.

There are alot more to add but this is just some of the basics that I think all developers should have in their repertoire. Feel free to add to this list in the comments box. If I could had another one to this, would be; all developers not just Java, should know how to search the web and Google is your best friends (now support my advertisers by clicking on the links on the right ūüėČ ). I hope you enjoyed the entry and feel free to comments good or bad!!! are welcome.