the-journey-to-the-paas_2294x12031

PaaS Cloud – The Rise of The Developer Entrepreneur

In bad times such as recession, there is the inevitable fact that some people will be laid off by their employer. In these circumstances, when software developer are made redundant they usually set-up shop. The majority goes into providing software development services for small company or friends. They set-up websites to promote their services and few focuses on building their own product.

In London, services based company took a massive hit during the downturn and it is still felt now; well the UK and now in a double-dip recession. So what is my point here? It is much harder to sell services and even harder to export services than products. When providing services, your market has geographical constraint; just think what would take for your to sell your services to a potential client 100 miles from your local?
It used to be hard to build products in the past due to high cost of hardware; to build a java-based product you had to find a suitable host and then pay for a dedicated server or VPS. As you are already tightening your belt, it is likely that you do not want to spend £60 + VAT a month on services to host your product and all the headache of troubleshooting it in case of problem.
Enter the PaaS, Platform as a Service. For this posting I am going to look at PaaS from a Java perspective but it should be applicable to any other programming language.
The competition in the PaaS market is fierce there is no single winner yet. Amazon is the leader for IaaS but not much to be deployed on Beanstalk as yet. If you look around the market, you can see that major technology vendor are jumping on the bandwagon:
Some of those PaaS services are free such as OpenShift and CloudBees. OpenShift runs on JBossAS7 and if you are using Eclipse for it becomes a breeze to develop and deploy, but the only downside is that it’s still in beta stage.
Utilising the current ecosystem of PaaS, you can go from:
inception (idea) -> development -> product
for almost nothing. The prototyping is now free for example you can use Heroku to build Facebook apps  making all the Facebook API directly available to your app and hosted for free (until you out grow your free account of course). I will suggest that developers familiarises themselves with Lean Startup and Lean Software Development.
If you are a developer and out-of work, you have no excuse of no building your cloud apps. Even if you do not want to start a business and you can do it as a learning practice and use that to your advantage next time you are interviewing for a new role.
With PaaS Cloud, every developer is a potential entrepreneur and the barriers to entry are almost non-existent. If you are looking for a job, it is no longer acceptable just to have some sample code on GitHub but you should also have a sample project hosted on a free cloud provider to showcase.
  This is blog was written from Cloudstock 2012

vodafone-top-61

5 Things all Java developer should know when developing for the cloud

The last couple of years, “Cloud Computing” replaced Web 2.0 as the new buzzword. You can read, hear and see everywhere the cloud is coming. To most developer, this is still the same old sh*t. If you have experience in developing distributed system then you should be fine, you say. Well not entirely true, the IT department wants to deploy on cheap cloud and therefore some restrictions now applies. I will list 5 things that I think all developers should know when working with cloud Platform as a Service provider such as Amazon Beanstalk or Google App Engine. This list also applies to IaaS architecture. Some of the points might be obvious to the more experienced, nevertheless, they need to be mentioned.

  • Static objects

We all know the difference between instance variable (non-static) and class variable (static variable). We use static to tell the JVM that they should only be one instance of this variable (singleton). If the static variable is declared with the “final” keyword, this will not cause a problem in a distributed environment as the value will never change. The problem is when we expect the value of the variable to change. As in a cluster environment, GAE and Beanstalk run your application in multiple JVM. If a the value of your static variable has changed in JVM, it will not be propagated to the cluster therefore leading to inconsistencies. I recommend that you avoid static variable unless that set as “final” and their values are hard-coded so there is no way to change their values are runtime.

  • Caching Objects
This one is related to performance in order to avoid expensive operations such as running database queries and others. Sometimes we need to cache objects in memory and therefore we implement our own caching strategy through the use of simple HashMap or some other caching solutions available outthere. Caching has many benefits but implementing a caching strategy should be approached with care. This is because caching has the same problem as static objects. Your cache will be in the local JVM therefore not it will not be visible in the cluster. There are some solutions, for example, GAE uses Memcached and Beanstalk can make use of Amazon ElastiCache which is compliant with Memcached. When developing for a PaaS environment, make sure to not implement your own caching system but look for one that is supported by the vendor. I know this can lead to vendor lock-ins.
  • Server-side Session
Something we do take for granted in single environment is storing application session data on the server. Based on experiences, mainly using GAE, I encountered multiple issues with session management. Since then, Google has fixed alot of the issues with the way GAE handle sessions for Java application. To minimize writing session to a datastore, we store application state in memory. Most application are written without any vendor approach in mind; so we use JEE as-is. This approach would work in you deploy in any self hosted clustered environment but Google PaaS. Google implements their own session management which is off by default therefore you need to enable it in appengine-web.xml and make sure that all your objects implements the java.io.Serializable interface.

Note: Note, session data is always written synchronously to memcache. If a request tries to read the session data when memcache is not available (or the session data has been flushed), it will fail over to the datastore, which may not yet have the most recent session data. This means that asynchronous session persistence may cause your application to see stale session data. However, for most applications the latency benefit far outweighs the risk.

 

  • Event-driven Execution
This is more about running a process at a given time such as Scheduling task. Again, in a managed environment, it is straightforward to implement a timer or scheduler service. But this is a clustered environment which is not managed by yourself and their stack his different to yours. I personally use Quartz Scheduler when working in a single server environment. In a clustered environment such as Beanstalk or GAE, it is difficult to know which instance will be triggered and execute the task only once. The folks at Google have provided another solution with their own implementation of Cron for Java which can be used. At the time of writing, Amazon Beanstalk didn’t have a solution yet. Therefore, consider before-hand when designing your system, which approach to take in order to create scheduled tasks for your application.
  • JRE white list
I believe this related to GAE J only. Google App Engine for Java doesn’t allow the use for all available API in Java, especially if they do require access to the file system. The fact that there is a such a restriction impose by the Google has led us to look elsewhere for some of our projects. The cost of re-developing our application to please them is much higher than deploying them elsewhere. Also, another downside of GAE J is doesn’t fully support JEE servlet specification. You cannot implement custom security for your application through your web.xml therefore pushing you to use Google own security mechanism. I would recommedn using GAE J when developing a greenfield project which can be built with these restrictions here  and here in mind. If you want to be locked-in using GAE J for your application, then I recommend it as a cost efficient way to testing your application otherwise, look somewhere else.
I hope this was helpful and if there’s mistake, feel free to get back to me and I make any corrections. Also, I am sure that I am missing some other points, add them to the comments sections.P.S. here is a nice comparison from IBM

Cheers and Happy Coding.
happy20101

Happy New Year Techies!!!

I just wanted to say happy new year to the community, without you guys, we would still be living in the dark ages. Here are a few that I am looking forward to in the new year:

  1. Oracle Sun merger:- The untold future of NetBeans, MySQL, Swing and every open source that Sun has been working on in the past. What would happen to Sun Open Source (SOS!!!) Movement.
  2. Looking forward to JavaFX1.3 release, Authoring tool and improved Composer plugin.
  3. Android uprise against iPhone (not because I can easily write Android based application) and hopefully with Visual XML builder to build Android UI. JavaFX running on Android, anybody????
  4.  Looking forward to seeing real-world JavaFX application and possibly a showcase linked to the JavaFX.com site.
  5. Java EE 6 support in Google App Engine
  6. Google Wave open to the public and how well it does against Twitter and Facebook
  7. Java Store open to Europe (well this is where I am based and want to sell applications not just provide free).
  8. My new video blog ( well that’s me trying to be more Armel 2.0 – the sequel) to better engage with the community.
  9. looking forward to networking with fellow developers and techies.
  10. looking forward to the new buzzwords (old technology, new name)

Well, 2009 is was good year for Java and tech scene. I hope that you all enjoyed the year as much as I did. Why not share what wish list with me (comment box). Subscribe to follow my blog as I will try to bring more interesting articles in the new year.

Happy 2010 New Year, wish you all; success, properity, fame (yup) and fortune. Don’t forget, if you need a server-side, UI, Android or JavaFx developer, just mail me.

openid-1

Dummies Guide to GWT and OpenID – with example code

Once (?) in awhile, we developers browse the web for some quick and dirty solutions to some of our coding problems (sometimes not even problems, right?). This is another of those days, everything was going fine on this project I am currently working on. This is not a social networking sites but a site for sharing and comunicating (how I missed the days of the chatrooms) similar in many aspect to Digg and DZone but which also allows users to communicate in realtime and leave comments. So the back-end was sort of completed but so now time to focus on the user registration.

The sites needed to be open and avoid repeatitive tasks such registering users. The way forward was to implement some authentication to allow users to register with some sort of universally accessible ID (or sort of). Facebook Connect, OpenSocial and Google Accounts have their advantages but to me personally; the disadvantages outweight the pros. These are some of the disadvantages of those platform:

  1. Facebook is a popular site hence is its platform, Facebook Connect, in mostly Europe and America but not everybody in Europe and America have a Facebook account.
  2. OpenSocial, when it comes to single logon, has more advantages than Facebook. I supposed that if we do take into consideration all the OpenSocial sites, we might have apossible larger user based than supporting Facebook alone. Even that was too limited.
  3. Google Accounts, one thing Google does not advertise its user base. I could be wrong but do anybody actually knows how many people uses the various Google services (excluding search). Google has the same disadvantages as Facebook.

Enter OpenID. OpenID has been around longer than most Internet-based decentralised authentication platform. The beauty of this platform is that it also supported by most (if not all) large site on the Internet. From AOL, BBC, CNN to YAHOO and ZIMBRA, as I said most sites (based on the alphabet) support OpenID, check the OpenID Directory. It was recently announced that OpenID has reached 1Billion enabled accounts (read here). Based on those figures, I decided that OpenID for now was the authentication choice for this application. I will not be discussing any security issues in this entry, there are plenty of resources available on the Internet for that.

OK, so I am a Java developer and I am developing the front-end using GWT and Java on the server-side. I searched on the Internet for solutions on how to implement the authentication as GWT RPC are different to normal servlet call. I spent more time reading about the OpenID specification and implementations examples on the Internet. I have to admit that some of tutorials that I found on the Internet were somehow confusing and not helpful at all. Therefore I decided to write my own Dummies Guide to GWT and OpenID.

Dummies Guide to GWT and OpenID

First of all, it is important to know how OpenID works (please check the OpenID site for more info).

In a nutshell, OpenID allows you to authenticate with website (supporting the standard) with just an URL and voila. Your URL has to be hosted by an OpenID provider in order for it to work. For example, my blog is hosted by Google on blogspot.com. Google supports OpenID authentication therefore if a reader wanted to leave a comment on my blog, he does not have to have a Google Account as he can log-in with his Yahoo or AOL or Facebook or WordPress or any other OpenID provider sites, and then leave a comment, that simple.

GWT to OpenID and back

There are two ways to authenticate a user with an OpenID provider and GWT supports both. When authenticating a user, the relaying site (the site the user is trying to access) redirects to user to the OpenID provider (i.e. Google) login page.

The problem will be with the GWT RPC mechanism. GWT RPC calls are asynchronous and you cannot make any redirections. Therefore we need a way to execute the redirection from the client side, here is the code (I use OpenID for Java to handle the openID discovery from the RPC servlet), I then used USER object (just a simple POJO which only stores the redirection URL and the parameters) to be sent back and forth between the front-end and back-end.

#######################################################
public User authenticateOpenId(String provider_url) {
    try {
        ConsumerManager manager = new ConsumerManager();
        // This is the URL where the OpenID provider will redirect the user
        // once logged in.
        String returnToUrl = “http://localhost:8084/GwtOpenId“;
       
        // the provider URL will be used to identify the user OpenID provider
        List discoveries = manager.discover(provider_url);
        DiscoveryInformation discovered = manager.associate(discoveries);
        // OpenID 4 Java needs to  have an HttpServletRequest object, GWT sevlet 
have
        // convenient methods to retrieve the HttpServletRequest object 
and manipulate its
        // parameters
        this.getThreadLocalRequest().setAttribute(“openid-disc”, discovered);
        this.getThreadLocalRequest().setAttribute(“openid.mode”, “popup”);
       
        AuthRequest authReq = manager.authenticate(discovered, returnToUrl);
        FetchRequest fetch = FetchRequest.createFetchRequest();
        // I want to exchange the following attributes from the OpenID provider
        // remember that teh attribute will be returned only if it exits
        fetch.addAttribute(“email”,”http://schema.openid.net/contact/email“,true);
        authReq.addExtension(fetch);
        // Simple POJO to persist the data
        User user = new User();
        // In a normal servlet development, the following statement would make
        // a redirect call but this would not work when using GWT RPC
        if(!discovered.isVersion2()){
            user.setRedirect(authReq.getDestinationUrl(true));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
            } else{
            user.setParams(authReq.getParameterMap());
            user.setRedirect(authReq.getDestinationUrl(false));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
        }
        } catch (MessageException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (DiscoveryException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (ConsumerException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
    }
}
#############################################################

The above codes will format the request in order for the front-end to execute a redirect and allow the user to authenticate with his OpenID provider. Now here is the front-end code which executes the authentication and reads the data back.

#################################################################
// This is an extract from the openidEntryPoint.java

// This is where the magic happens – This code is only usefull when the OpenID 
provider
// redirects the user back to your site – please visit openid.net for valid 
parameters.
// The “if” statement checks to make sure that it is a valid response from 
the OpenID
// provider – You can do anything you want with the results here such 
as verifying the
// response with the server-side code
if(Window.Location.getParameter(“openid.rpnonce”) != null){
    String s = Window.Location.getParameter(“openid.mode”);
    // executes this if the user cancels the authentication process and the OpenID
    // providers returns to the your site
    if(s.equals(“cancel”)){
        sign.setText(“Server returned openid.mode=cancel”);
        openIdUrl.setText(“You need to Accept not Cancel”);
    }
    // Here, I am assuming that everything is fine and that the user has 
been sucessfully logged in
    else{
        sign.setText(“You have successfully signed in”);
        vp.setVisible(false);
    }
   
}
RootPanel.get().add(contentPanel);

}

private class UserAsyncCallback implements AsyncCallback<User> {
   
    public void onFailure(Throwable caught) {
        Window.alert(caught.getLocalizedMessage());
    }
   
    public void onSuccess(User result) {
        if (result != null) {
            //                    Window.open(result.getRedirect(), “_blank”,
 “height=200,width=400,left=100,” +
            //                            “top=100,resizable=no,scrollbars=no,
toolbar=no,status=yes”);
            // this the most important line in order to make the authentication.
 Here, I am redirecting the user
            // from the client side to the OpenID provider URL with the discovery
 data generated from the
            // RPC call to the servlet.
            Window.Location.assign(result.getRedirect());
            } else {
            Window.alert(“Ooops!!! Couldn’t find your provider”);
        }
    }
}
#######################################################################################################

I have attached the full NetBeans project with dependencies. The code is provided as-is and use at your own risk ;). Here is a screenshot of the working application:

Step1: Authenticate on the site (enter the URL)
moz-screenshot-6

Step2: Redirect to OpenID provider (Google is my provider 🙂 ), authenticate with your provider
moz-screenshot-7

Step3: Allow the application to access your OpenID details and redirect back to the original site
moz-screenshot-8

Step4: final step, check the parameters from the provider and proceeds accordingly
moz-screenshot-9

Take a look at the URL in each of the above step to see the OpenID data. OK, so my code actually works (yuppie), now you know that it is possible form GWT to OpenID and it’s not as complicated as many other sites are trying to show. The code is just for authentication but once authenticated, you can retrieve any parameters that you need. In this example, the query is sent through the browser URL (GET) but you can change it to be encoded in a form submit action. Iwrote some of the code to allow the user to authenticate via a popup window, the code is not complete and maybe someone else might want to have a go. My only problem is getting the redirect back to the original window but apart from that it works.

Download the source code

I hope you guys enjoyed it and Happy Coding.

4064537510_e727865913

ReST Web Services on Google App Engine using NetBeans 6.7

When Google announced early this year the availability of the Java language on their cloud system “App Engine”, this was turning point for Java on the web. The reason I am saying that is due to a simple questions; how many hosting providers support Java? And providing free hosting for that purpose… This was a good move in the direction of the JAVA community and the vast list of all the languages that run on the JVM.  But App Engine does not support the full range of the J2EE API. If you are looking to build any EJB, SOAP services or anything that access the file system, then the App Engine is not for you as it does not support it. To make matter worst, some URL connections are not supported by the Google cloud services. I am looking forward to the day when JBoss Seam will be supported entirely by their services without having to hack the code. Alright, it is not that bad; Google App engine supports a number of Java EE frameworks such as Spring.

Alright people, this was a long introduction but I believe it is worth it. You cannot create web services such as SOAP in the Google App Engine therefore, in this short article, I will show you how to develop a ReST based web services that works with the App Engine using NetBeans and Jersey API. Actually, NetBeans comes with Jersey support out the box. Now, let’s get started.
In order to follow the instructions, you will need the following:

NetBeans 6.7+

Google App Engine SDK

Google App Engine Plugin for NetBeans

I would expect you to know to be familiar with JAVA EE development and Google App Engine development. Once you have all the software and components installed, now we can start.

1.    Create a database structure to store your data. I used MySQL Workbench to design my DB structure (Google App Engine does not used MySQL and does not have plan to support it in the future). This schema is to help you understand the relationship between your entities. You can use any UML tools to design your objects’ relationship. There is a reason I chose to design DB in MySQL, the application generates SQL script which I will upload to MySQL DB. I will show you in the next step the main benefits.
 

2.    Based on your DB schema, create a database. You can use any database you want. I used MySQL to initially store my entities.

3.    Launch your NetBeans application and create new Web Application.
 

3.1    Choose Google App Engine as deployment server, click here to see how to register the App Engine in NetBeans.

4.    Now click on: File -> New File. In the popup window, choose Web Services in the Categories panel and RESTFul Web Services from Database and click next.
 

5.    Choose your the database that you want to generate your Entity classes from and click next. ( in the screenshot I am using the sample DB which came with the NetBeans and JAVA DB server. This is just for illustration purposes only as I have previously generated my Entity Classes from the DB schema I generated in step 1).

6.    Check your Class Names and how they relate to the database. Make any changes that you require in this screen, and then click next.
 

7.    In the next screen, just accept the default values and click finish.
 

Well we have done the hard parts. There is another step that missed out due to the fact that I already written the application. When generating Entities from Database, if you do not have a persistence unit available, it will ask to generate one. Here is more information on how to create a persistence unit with NetBeans. Make sure to make sure to choose “Create” as the table generation strategy.
 

By Now, you should be aware that we have create a back-end application store which we can call using normal http post, get, delete and create. NetBeans RestFull methods allows to use XML or JSON to send data to the services. The responses MIME can be anything you like as long as the application server supports it.
WARNING: JERSEY XML processing is not supported by Google App Engine as it uses JAXB. JAXB accesses API which is forbidden by the App Engine stack. If testing the application on the App engine, use the JSON MIME for your data.
Ok so we have generate the classes and methods require to expose our back-end to other application. As it is, this will not work on Google App Engine so we need to make a final change, this time we need to change the persistence.xml file manually. In the project window under the name of your project, click on Configuration files -> persistence.xml and open the file in the editor. Once the file is open in the editor, click on the XML tab and make the necessary changes to make it look like the screenshot.
 

Here is the content of persistence.xml; modify to reflect your entities and make sure the name of the provider is the same as mine and the properties are exactly the same.

    <?xml version=”1.0″ encoding=”UTF-8″?>
    <persistence version=”1.0″ xmlns=”http://java.sun.com/xml/ns/persistence” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd”>
        <persistence-unit name=”ReSTWebApplicationPU” transaction-type=”RESOURCE_LOCAL”>
            <provider>org.datanucleus.store.appengine.jpa.DatastorePersistenceProvider</provider>
            <class>com.test.api.Userlogs</class>
            <class>com.test.api.Customer</class>
            <class>com.test.api.User</class>
            <class>com.test.api.Business</class>
            <class>com.test.api.Reviews</class>
            <exclude-unlisted-classes>false</exclude-unlisted-classes>
            <properties>
                <property name=”datanucleus.NontransactionalRead” value=”true”/>
                <property name=”datanucleus.NontransactionalWrite” value=”true”/>
                <property name=”datanucleus.ConnectionURL” value=”appengine”/>
            </properties>
        </persistence-unit>
    </persistence>

Now we are ready to run the application and test it in our local environment. Right click your project name in the projects window and click Test RestFul web services, your favourite web browser with page that shows your resources. You can test your services by clicking on the nodes and or drilling through the nodes.
 

You can also use the Firefox RESTClient Plugin to test your app. Also, Google App Engine does not fully support JPA and might throw some exception about not supporting “integer type object for primary key”. If you do experience this issue, change the type of your key.
This is it for now, looking forward to your feedback, good or bad, they are all welcome. Also, if you need JAVA EE help, contact me directly. Also join me on facebook. Hope the guide or tutorial or whatever you call it, was good enough to help you with something.

Please support my blog and its advertisers by clicking on the interesting products/ services on the right (Google ads).  Cheers.