openid-1

Dummies Guide to GWT and OpenID – with example code

Once (?) in awhile, we developers browse the web for some quick and dirty solutions to some of our coding problems (sometimes not even problems, right?). This is another of those days, everything was going fine on this project I am currently working on. This is not a social networking sites but a site for sharing and comunicating (how I missed the days of the chatrooms) similar in many aspect to Digg and DZone but which also allows users to communicate in realtime and leave comments. So the back-end was sort of completed but so now time to focus on the user registration.

The sites needed to be open and avoid repeatitive tasks such registering users. The way forward was to implement some authentication to allow users to register with some sort of universally accessible ID (or sort of). Facebook Connect, OpenSocial and Google Accounts have their advantages but to me personally; the disadvantages outweight the pros. These are some of the disadvantages of those platform:

  1. Facebook is a popular site hence is its platform, Facebook Connect, in mostly Europe and America but not everybody in Europe and America have a Facebook account.
  2. OpenSocial, when it comes to single logon, has more advantages than Facebook. I supposed that if we do take into consideration all the OpenSocial sites, we might have apossible larger user based than supporting Facebook alone. Even that was too limited.
  3. Google Accounts, one thing Google does not advertise its user base. I could be wrong but do anybody actually knows how many people uses the various Google services (excluding search). Google has the same disadvantages as Facebook.

Enter OpenID. OpenID has been around longer than most Internet-based decentralised authentication platform. The beauty of this platform is that it also supported by most (if not all) large site on the Internet. From AOL, BBC, CNN to YAHOO and ZIMBRA, as I said most sites (based on the alphabet) support OpenID, check the OpenID Directory. It was recently announced that OpenID has reached 1Billion enabled accounts (read here). Based on those figures, I decided that OpenID for now was the authentication choice for this application. I will not be discussing any security issues in this entry, there are plenty of resources available on the Internet for that.

OK, so I am a Java developer and I am developing the front-end using GWT and Java on the server-side. I searched on the Internet for solutions on how to implement the authentication as GWT RPC are different to normal servlet call. I spent more time reading about the OpenID specification and implementations examples on the Internet. I have to admit that some of tutorials that I found on the Internet were somehow confusing and not helpful at all. Therefore I decided to write my own Dummies Guide to GWT and OpenID.

Dummies Guide to GWT and OpenID

First of all, it is important to know how OpenID works (please check the OpenID site for more info).

In a nutshell, OpenID allows you to authenticate with website (supporting the standard) with just an URL and voila. Your URL has to be hosted by an OpenID provider in order for it to work. For example, my blog is hosted by Google on blogspot.com. Google supports OpenID authentication therefore if a reader wanted to leave a comment on my blog, he does not have to have a Google Account as he can log-in with his Yahoo or AOL or Facebook or WordPress or any other OpenID provider sites, and then leave a comment, that simple.

GWT to OpenID and back

There are two ways to authenticate a user with an OpenID provider and GWT supports both. When authenticating a user, the relaying site (the site the user is trying to access) redirects to user to the OpenID provider (i.e. Google) login page.

The problem will be with the GWT RPC mechanism. GWT RPC calls are asynchronous and you cannot make any redirections. Therefore we need a way to execute the redirection from the client side, here is the code (I use OpenID for Java to handle the openID discovery from the RPC servlet), I then used USER object (just a simple POJO which only stores the redirection URL and the parameters) to be sent back and forth between the front-end and back-end.

#######################################################
public User authenticateOpenId(String provider_url) {
    try {
        ConsumerManager manager = new ConsumerManager();
        // This is the URL where the OpenID provider will redirect the user
        // once logged in.
        String returnToUrl = “http://localhost:8084/GwtOpenId“;
       
        // the provider URL will be used to identify the user OpenID provider
        List discoveries = manager.discover(provider_url);
        DiscoveryInformation discovered = manager.associate(discoveries);
        // OpenID 4 Java needs to  have an HttpServletRequest object, GWT sevlet 
have
        // convenient methods to retrieve the HttpServletRequest object 
and manipulate its
        // parameters
        this.getThreadLocalRequest().setAttribute(“openid-disc”, discovered);
        this.getThreadLocalRequest().setAttribute(“openid.mode”, “popup”);
       
        AuthRequest authReq = manager.authenticate(discovered, returnToUrl);
        FetchRequest fetch = FetchRequest.createFetchRequest();
        // I want to exchange the following attributes from the OpenID provider
        // remember that teh attribute will be returned only if it exits
        fetch.addAttribute(“email”,”http://schema.openid.net/contact/email“,true);
        authReq.addExtension(fetch);
        // Simple POJO to persist the data
        User user = new User();
        // In a normal servlet development, the following statement would make
        // a redirect call but this would not work when using GWT RPC
        if(!discovered.isVersion2()){
            user.setRedirect(authReq.getDestinationUrl(true));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
            } else{
            user.setParams(authReq.getParameterMap());
            user.setRedirect(authReq.getDestinationUrl(false));
            // fakes the redirect by sending the POJO with the required parameters
            // to make a client-side redirect
            return user;
        }
        } catch (MessageException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (DiscoveryException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
        } catch (ConsumerException ex) {
        Logger.getLogger(GWTServiceImpl.class.getName()).log(Level.SEVERE, null, ex);
        return null;
    }
}
#############################################################

The above codes will format the request in order for the front-end to execute a redirect and allow the user to authenticate with his OpenID provider. Now here is the front-end code which executes the authentication and reads the data back.

#################################################################
// This is an extract from the openidEntryPoint.java

// This is where the magic happens – This code is only usefull when the OpenID 
provider
// redirects the user back to your site – please visit openid.net for valid 
parameters.
// The “if” statement checks to make sure that it is a valid response from 
the OpenID
// provider – You can do anything you want with the results here such 
as verifying the
// response with the server-side code
if(Window.Location.getParameter(“openid.rpnonce”) != null){
    String s = Window.Location.getParameter(“openid.mode”);
    // executes this if the user cancels the authentication process and the OpenID
    // providers returns to the your site
    if(s.equals(“cancel”)){
        sign.setText(“Server returned openid.mode=cancel”);
        openIdUrl.setText(“You need to Accept not Cancel”);
    }
    // Here, I am assuming that everything is fine and that the user has 
been sucessfully logged in
    else{
        sign.setText(“You have successfully signed in”);
        vp.setVisible(false);
    }
   
}
RootPanel.get().add(contentPanel);

}

private class UserAsyncCallback implements AsyncCallback<User> {
   
    public void onFailure(Throwable caught) {
        Window.alert(caught.getLocalizedMessage());
    }
   
    public void onSuccess(User result) {
        if (result != null) {
            //                    Window.open(result.getRedirect(), “_blank”,
 “height=200,width=400,left=100,” +
            //                            “top=100,resizable=no,scrollbars=no,
toolbar=no,status=yes”);
            // this the most important line in order to make the authentication.
 Here, I am redirecting the user
            // from the client side to the OpenID provider URL with the discovery
 data generated from the
            // RPC call to the servlet.
            Window.Location.assign(result.getRedirect());
            } else {
            Window.alert(“Ooops!!! Couldn’t find your provider”);
        }
    }
}
#######################################################################################################

I have attached the full NetBeans project with dependencies. The code is provided as-is and use at your own risk ;). Here is a screenshot of the working application:

Step1: Authenticate on the site (enter the URL)
moz-screenshot-6

Step2: Redirect to OpenID provider (Google is my provider ūüôā ), authenticate with your provider
moz-screenshot-7

Step3: Allow the application to access your OpenID details and redirect back to the original site
moz-screenshot-8

Step4: final step, check the parameters from the provider and proceeds accordingly
moz-screenshot-9

Take a look at the URL in each of the above step to see the OpenID data. OK, so my code actually works (yuppie), now you know that it is possible form GWT to OpenID and it’s not as complicated as many other sites are trying to show. The code is just for authentication but once authenticated, you can retrieve any parameters that you need. In this example, the query is sent through the browser URL (GET) but you can change it to be encoded in a form submit action. Iwrote some of the code to allow the user to authenticate via a popup window, the code is not complete and maybe someone else might want to have a go. My only problem is getting the redirect back to the original window but apart from that it works.

Download the source code

I hope you guys enjoyed it and Happy Coding.

4124677046_69aa7a07b3

GWT Visualization Example – Annotated Time Line Chart Tutorial

If you are a server-side Java developer such as myself, I am sure you want an easy way to create AJAX based application but pure JavaScript (ECMAScript) is not your strong skills. The beauty of JavaScript is that it runs on any web server, no need for a servlet container or anything of that sort.

So, I have been writing AJAX based UI using the GWT framework for the past year and half. I am still not a web developer but I understand enough CSS to spice up my site. Anyway, I am writing this tutorial because I think that the GWT Visualization team, even tough they did a good job, over complicates the creation of charts in their tutorial. I was trying to create an Annotated Time Line chart so I looked up my previous code I wrote for the GWT Portlet JSR-168, see here. I also decided to run a search (don’t be evil, Google is your friend) to try to find different examples and hopefully some nice custom charts. First of all, based on the search result; I thought that some of examples, including Google’s own, were confusing and over complex. Most examples uses AJAXLoader to load the Visualization API, but you shouldn’t have to use this. This is the simplest way to create a GWT Annotated Time Line Chart.

gwt annotated time line chart

1. Using your favorite development tool with support for GWT framework, create a new Java web application, you can export the compiled JavaScript later to a non-based web application, if you use an IDEW you should code-completion support.

2. Make you sure you have included the GWT Visualization API module in your classpath.

3. In your project source code root directory, you should <AppNamexxxx>.gwt.xml file. make sure to add teh following line to it in order to make the  module available to your application.

<inherits name=”com.google.gwt.visualization.Visualization”/>

4. In order to keep this tutorial short and straight to point, I have included the chart creation code in my <AppNamexxxx>EntryPoint.java class

public class MainEntryPoint implements EntryPoint
{
    
    
    /**
    * Creates a new instance of MainEntryPoint
    */
    public MainEntryPoint()
    {
        
        
    }
    
    
    
    /**
    * The entry point method, called automatically by loading a module
    * that declares an implementing class as an entry-point
    */
    public void onModuleLoad()
    {
        
        Runnable onLoadCallback = new Runnable()
        {
            
            
            public void run()
            {
                
                DataTable data = createHistoryTable(result);
                AnnotatedTimeLine.Options options = AnnotatedTimeLine.Options.create();
                options.setDisplayAnnotations(true);
                options.setDisplayZoomButtons(true);
                options.setScaleType(AnnotatedTimeLine.ScaleType.ALLFIXED);
                options.setLegendPosition(AnnotatedTimeLine.AnnotatedLegendPosition.SAME_ROW);
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†AnnotatedTimeLine¬†atl¬†=¬†new¬†AnnotatedTimeLine(data,¬†options,¬†“600px”,¬†“200px”);
                RootPanel.get().add(atl);
                
            }
            
            
            
        }      
        ;
        VisualizationUtils.loadVisualizationApi(onLoadCallback, AnnotatedTimeLine.PACKAGE);
        
    }
    
    
    
    // This method will create the Data used by the chart
    private DataTable createHistoryTable()
    {
        
        DataTable data = DataTable.create();
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.DATE,¬†“Date”);
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.NUMBER,¬†“Price”);
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.NUMBER,¬†“Low”);
¬†¬†¬†¬†¬†¬†¬†¬†data.addColumn(AbstractDataTable.ColumnType.NUMBER,¬†“High”);
        data.addRows(5);
                  
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†DateTimeFormat¬†dtf¬†=¬†DateTimeFormat.getFormat(“yyyy-MM-dd”);
¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(0,¬†0,¬†dtf.parse(“2009-11-21”));
            data.setValue(0, 1, 100);
            data.setValue(0, 2, 120);
            data.setValue(0, 3, 90);

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(1,¬†0,¬†dtf.parse(“2009-11-22”));
            data.setValue(1, 1, 90);
            data.setValue(1, 2, 110);
            data.setValue(1, 3, 100);

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(2,¬†0,¬†dtf.parse(“2009-11-23”));
            data.setValue(2, 1, 100);
            data.setValue(2, 2, 180);
            data.setValue(2, 3, 80);

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(3,¬†0,¬†dtf.parse(“2009-11-20”));
            data.setValue(3, 1, 130);
            data.setValue(3, 2, 100);
            data.setValue(3, 3, 130);
          

¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†¬†data.setValue(4,¬†0,¬†dtf.parse(“2009-11-19”));
            data.setValue(4, 1, 130);
            data.setValue(4, 2, 170);
            data.setValue(4, 3, 150);
            
        }
        return data;  
    }  
}

Don’t forget to import all the necessary classes and voila!

I hope this will save you some time in creating your charts.

 

Develop Your Own Google with Apache Lucene (Java Nutch Solr)

Apache Lucene is Open Source API that allows a Java developer (.Net libraries available) to write indexing and full-text search capable applications. I have been writing applications based on Lucene for the last 3 years and some of the applications have been deployed at large corporations. I know there are other libraries available to developers who wish to write indexing engine but this blog will solely focus on Apache Lucene. I will not compare it to other API.

Lucene is a very mature API and can be found in NetBeans IDE, Liferay, JackRabbit among others. IBM has written a very good document about the Lucene architecture, therefore I will not delve into it here.

Lucene, alone, is pretty much useless as any other API. Let’s now introduce Nutch. Nutch is a web crawler built-on top of Lucene to provide file crawling capabilities. Nutch was designed to handle large amount of data from the internet (http). Due to its plugin architecture, it was later extended to provide local network crawling such as FTP, databases and Microsoft Windows Shares (I am the author of the protocol-smb plugin and co-author of the index-extra plugin found on the Nutch site). We had extended Nutch and turned it into an Enterprise Search application but most of the source codes were locked behind closed doors (company politics).Anyway, Nutch has evolved to become but still very complex in its inner working. The initial Nutch was developed to process data in a batch but there are ways to turn it real-time but that’s for another day. Ok, so Nutch is good for crawling and indexing of data but it does not handle search directly. There is a web application available with Nutch but it is quite poor so let’s now introduce the Solr.

Solr is a powerful web-based search server built-on top of Lucene. The application was developed by CNET Networks and donated to the Apache Foundation. I believe, not too sure so I might need some references here, Solr was powering the search feature on their site but it is definitely used internally by the company. Late 2009, Lucid Imaginations receives $7.5 million in funding to provide commercial services built around Solr (and Lucene possibly?). Here is a very good presentation about Solr. Solr is a very good indexing engine. The keyword here is “indexing engine”. It does not have any support for crawling data therefore requiring the developer to create applications that will feed it the data to index. I do believe that it is a good feature of the application as it gives the ability to integrate with various systems as long as they can post data over HTTP.

Nutch is a good crawler but it does not provide an enterprise-grade search interface to its data. Solr, in the other hand, is powerful indexer and has an enterprise grade search interface but it does not know how to gather data in its own. I am sure by now it has become obvious how we can integrate them both together.

We want Nutch to gather the data, by-pass its indexing cycle and feed the data directly to Solr. Lucid Imagination has a good tutorial about it here.

After reading the tutorial from Lucid Imagination, you will notice that Nutch is run by executing some bash files. This is something I strongly disagree with. If Nutch is based on Java (an OS independent language), why do we need to execute some UNIX/LINUX shell script. Also, the fact we need to install CygWin on MS Windows platform to be able to run is a big negative for me. I wrote a simple Java application that will launch Nutch and send the indexing to Solr but as you can see in the source code, you still need a UNIX like environment to run successfully. You can write a platform independent version by looking up Nutch API and calling the methods directly.

Well, I hope that this entry help you understand how to use Nutch and Solr built-on top of Apache Lucene. If you need any clarification, leave comments and I will try to gave ASAP if time permits.

package com.etapix.nutchsolr;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileFilter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Armel Nene
 */
public class Indexer {

    public static void main(String args[]) {
        if (args.length < 3) {
            System.out.println("Usage:" +
                    "ncrawlName        -   This will be used to store crawler files in CrawlName directory" +
                    "nurlFolder        -   The path to the folder containing the URL to crawl" +
                    "nsolrUrl          -   The URL to the Solr server");
            return;
        }
        String crawlerName = args[0];
        String urlFolder = args[1];
        String solrUrl = args[2];
        String inject = "bash bin/nutch2.sh inject " + crawlerName + "/crawldb " + urlFolder;
        String generate = "bash bin/nutch2.sh generate " + crawlerName + "/crawldb " + crawlerName + "/segments -topN 10 -numFetchers 5";
        String export = crawlerName + "/segments/";

        String invertLinks = "bash bin/nutch2.sh invertlinks " + crawlerName + "/linkdb -dir " + crawlerName + "/segments";
        String indexSolr = "bash bin/nutch2.sh solrindex " + solrUrl + " " + crawlerName + "/crawldb " + crawlerName + "/linkdb " + crawlerName + "/segments/*";
        try {
            System.out.println("Injecting URLs in crawldb");
//            int state = 0;
            InputStream in = Runtime.getRuntime().exec(inject).getInputStream();
            System.out.println(convertStreamToString(in));


//            state = Runtime.getRuntime().exec(inject).waitFor();
//            System.out.println("process completed: " + state);

            for (int i = 0; i < 3; i++) {
                System.out.println("Generating segments");
                in = Runtime.getRuntime().exec(generate).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Setting environment variable $SEGMENT");
//            String segs = convertStreamToString(Runtime.getRuntime().exec("ls -tr " + crawlerName + "/segments|tail -1").getInputStream());

                String segments = export + lastFileModified(export).getName();
                System.out.println("$segments: " + segments);
//            in = Runtime.getRuntime().exec(export + segs).getInputStream();
            System.out.println(convertStreamToString(in));

                String fetch = "bash bin/nutch2.sh fetch " + segments + " -noParsing";
                String parse = "bash bin/nutch2.sh parse " + segments;
                String update = "bash bin/nutch2.sh updatedb " + crawlerName + "/crawldb " + segments + " -filter -normalize";

                System.out.println("fetch segments");
                in = Runtime.getRuntime().exec(fetch).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Parse segments");
                in = Runtime.getRuntime().exec(parse).getInputStream();
                System.out.println(convertStreamToString(in));

                System.out.println("Update crawldb");
                in = Runtime.getRuntime().exec(update).getInputStream();
                System.out.println(convertStreamToString(in));
            }
            System.out.println("Inverting links");
            in = Runtime.getRuntime().exec(invertLinks).getInputStream();
            System.out.println(convertStreamToString(in));

            System.out.println("Indexing contents to Solr " + solrUrl);
            in = Runtime.getRuntime().exec(indexSolr).getInputStream();
            System.out.println(convertStreamToString(in));

        } catch (Exception ex) {
            Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

    public static String convertStreamToString(InputStream is) {
        /*
         * To convert the InputStream to String we use the BufferedReader.readLine()
         * method. We iterate until the BufferedReader return null which means
         * there's no more data to read. Each line will appended to a StringBuilder
         * and returned as String.
         */
        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        StringBuilder sb = new StringBuilder();

        String line = null;
        System.out.println("Now converting inputstream to text");
        try {
            while ((line = reader.readLine()) != null) {
                sb.append(line + "n");
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                is.close();
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        System.out.println("Finish converting to text");
        return sb.toString();
    }

    public static File lastFileModified(String dir) {
        File fl = new File(dir);
        File[] files = fl.listFiles(new FileFilter() {

            public boolean accept(File file) {
                return file.isDirectory();
            }
        });
        long lastMod = Long.MIN_VALUE;
        File choice = null;
        for (File file : files) {
            if (file.lastModified() > lastMod) {
                choice = file;
                lastMod = file.lastModified();
            }
        }
        return choice;
    }
}

Dream_Job_Icon

Java Interview: tips for the interviewer

This is a brief entry about how an interviewer should conduct a Java based interview. I have been on both side of the table and this is only based on my own observations.  I think most interviews that I attended in the past, and possibly some I have conducted myself, focus on the basic concept. I accept that developers should be aware of the language principles, but how can an interviewer test more advanced principles?

The interview time have to be split into: 

  1. Theoretical conceptsThis 1st phase is about asking the common questions about OOP concepts, differences between interfaces and abstract classes, differences between List and Set, multithreading and so on. If it is required to store answers, do provide a written paper and avoid multiple choice questions. In my experience, open-ended questions allow the developer to express his knowledge better. Multiple choice questions provide unnecessary hints to question answers. Even if I did forget a terminology or a definition by looking at the various answers provided, I can pick the right answer 9/10 times. Theories are central to practices but make sure can be related to the role you are recruiting for.An important point when interviewing a developer, if you have a developer present during the interview, avoid asking domain specific question unless it is something common. I had some interview with senior and lead developers asking questions about a programming problem they have recently encountered and were able to fix after spending months working on it, how am I supposed to know about that unless I have encountered the same issues in the past.
  2. Practical test
    In order to test a developer capability for the role, it is a must that he/she undertakes a practical test. You do not need to give the developer code with bugs for him/ her to fix, I believe this approach is not very useful. I would suggest that all interviewers prepare the developer to take a practical test using their favourite tools (provide two of the most common open source IDE) and provide them with a simple problem domain. In my experience, I had to write some simple factorial algorithm (a single recursive method) to a domain centric web service application (no DB, store data in memory instead) and to more advance concepts. If application multithreading is part of the main day to day job, then ask the developer to write a simple application that shows that.
    This practice exposes several features of the developer; from its reasoning by writing simple algorithm to coding practice (commenting and Java best practices) and problem solving.

After all, I believe that programming is more about logical reasoning and coding. The more senior we get in our profession, the harder it is to answer simple questions with straightforward answers; we get carried away and provide a complex answer to something so simple. We cannot talk baby talk anymore; our tongue is full of jargon. All developers should follow the KISS principle when answering theoretical questions but ultimately they should excel in the practical test. If they are good enough, they should use a text editor to write their codes and compile it through the shell (JVM command line).
Another thing that all interviewers should know; developers have very good memories and therefore can memorise more than 115 interview questions; I would recommend you to focus more on a practical test.

This was a brief entry for interviewers. The web is full of tips and questions for the interviewees and I do not see any point in duplicating them here.  As usual, this is based on my experience and belief, you are always welcome to comment. Also, support my blog by visiting my advertisers (by clicking on the link on the right) they might have something that you might need.

4064537510_e727865913

ReST Web Services on Google App Engine using NetBeans 6.7

When Google announced early this year the availability of the Java language on their cloud system ‚ÄúApp Engine‚ÄĚ, this was turning point for Java on the web. The reason I am saying that is due to a simple questions; how many hosting providers support Java? And providing free hosting for that purpose… This was a good move in the direction of the JAVA community and the vast list of all the languages that run on the JVM.  But App Engine does not support the full range of the J2EE API. If you are looking to build any EJB, SOAP services or anything that access the file system, then the App Engine is not for you as it does not support it. To make matter worst, some URL connections are not supported by the Google cloud services. I am looking forward to the day when JBoss Seam will be supported entirely by their services without having to hack the code. Alright, it is not that bad; Google App engine supports a number of Java EE frameworks such as Spring.

Alright people, this was a long introduction but I believe it is worth it. You cannot create web services such as SOAP in the Google App Engine therefore, in this short article, I will show you how to develop a ReST based web services that works with the App Engine using NetBeans and Jersey API. Actually, NetBeans comes with Jersey support out the box. Now, let’s get started.
In order to follow the instructions, you will need the following:

NetBeans 6.7+

Google App Engine SDK

Google App Engine Plugin for NetBeans

I would expect you to know to be familiar with JAVA EE development and Google App Engine development. Once you have all the software and components installed, now we can start.

1.    Create a database structure to store your data. I used MySQL Workbench to design my DB structure (Google App Engine does not used MySQL and does not have plan to support it in the future). This schema is to help you understand the relationship between your entities. You can use any UML tools to design your objects‚Äô relationship. There is a reason I chose to design DB in MySQL, the application generates SQL script which I will upload to MySQL DB. I will show you in the next step the main benefits.
 

2.    Based on your DB schema, create a database. You can use any database you want. I used MySQL to initially store my entities.

3.    Launch your NetBeans application and create new Web Application.
 

3.1    Choose Google App Engine as deployment server, click here to see how to register the App Engine in NetBeans.

4.    Now click on: File -> New File. In the popup window, choose Web Services in the Categories panel and RESTFul Web Services from Database and click next.
 

5.    Choose your the database that you want to generate your Entity classes from and click next. ( in the screenshot I am using the sample DB which came with the NetBeans and JAVA DB server. This is just for illustration purposes only as I have previously generated my Entity Classes from the DB schema I generated in step 1).

6.    Check your Class Names and how they relate to the database. Make any changes that you require in this screen, and then click next.
 

7.    In the next screen, just accept the default values and click finish.
 

Well we have done the hard parts. There is another step that missed out due to the fact that I already written the application. When generating Entities from Database, if you do not have a persistence unit available, it will ask to generate one. Here is more information on how to create a persistence unit with NetBeans. Make sure to make sure to choose ‚ÄúCreate‚ÄĚ as the table generation strategy.
 

By Now, you should be aware that we have create a back-end application store which we can call using normal http post, get, delete and create. NetBeans RestFull methods allows to use XML or JSON to send data to the services. The responses MIME can be anything you like as long as the application server supports it.
WARNING: JERSEY XML processing is not supported by Google App Engine as it uses JAXB. JAXB accesses API which is forbidden by the App Engine stack. If testing the application on the App engine, use the JSON MIME for your data.
Ok so we have generate the classes and methods require to expose our back-end to other application. As it is, this will not work on Google App Engine so we need to make a final change, this time we need to change the persistence.xml file manually. In the project window under the name of your project, click on Configuration files -> persistence.xml and open the file in the editor. Once the file is open in the editor, click on the XML tab and make the necessary changes to make it look like the screenshot.
 

Here is the content of persistence.xml; modify to reflect your entities and make sure the name of the provider is the same as mine and the properties are exactly the same.

    <?xml version=”1.0″ encoding=”UTF-8″?>
    <persistence version=”1.0″ xmlns=”http://java.sun.com/xml/ns/persistence” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd”>
        <persistence-unit name=”ReSTWebApplicationPU” transaction-type=”RESOURCE_LOCAL”>
            <provider>org.datanucleus.store.appengine.jpa.DatastorePersistenceProvider</provider>
            <class>com.test.api.Userlogs</class>
            <class>com.test.api.Customer</class>
            <class>com.test.api.User</class>
            <class>com.test.api.Business</class>
            <class>com.test.api.Reviews</class>
            <exclude-unlisted-classes>false</exclude-unlisted-classes>
            <properties>
                <property name=”datanucleus.NontransactionalRead” value=”true”/>
                <property name=”datanucleus.NontransactionalWrite” value=”true”/>
                <property name=”datanucleus.ConnectionURL” value=”appengine”/>
            </properties>
        </persistence-unit>
    </persistence>

Now we are ready to run the application and test it in our local environment. Right click your project name in the projects window and click Test RestFul web services, your favourite web browser with page that shows your resources. You can test your services by clicking on the nodes and or drilling through the nodes.
 

You can also use the Firefox RESTClient Plugin to test your app. Also, Google App Engine does not fully support JPA and might throw some exception about not supporting ‚Äúinteger type object for primary key‚ÄĚ. If you do experience this issue, change the type of your key.
This is it for now, looking forward to your feedback, good or bad, they are all welcome. Also, if you need JAVA EE help, contact me directly. Also join me on facebook. Hope the guide or tutorial or whatever you call it, was good enough to help you with something.

Please support my blog and its advertisers by clicking on the interesting products/ services on the right (Google ads).  Cheers.

3979528205_20b651d7e2

TwittDM – Automated Twitter Mass Direct Messenger (in JAVA)

Awhile ago, I wrote about the about an automated  mass direct messenger for Twitter to allow users to contact all their followers at once, and making sure that their tweets will not be lost in the wild. Well, the wait is now over. I wrote a proof concept that I will enhance with more features as users make request.

Here is a simple list of current features:

  • Direct Message – Users can write a single message and have sent directly to all their followers.
  • Mentions – Users can write a single message and have all their followers mentioned so that followers will read.

Future feature

  • Select message recipients – Users will be able to select which users will receive the direct messages or which users will be mentioned in their updates.
  • any feature requested by users.

The application was developed using JAVA, ReST API and Twitter4J API. This is a secure interface and I am not storing users’ passwords and username and authentication are handled by Twitter web services.

You can find the application @ http://twittdm.appspot.com/. Yup, the application is deployed on Google App engine.

Feel free to let me know what you think of the app, and please do not mentioned by graphic skills because I am not a designer.

All comments are welcome, and you request features by sending me a tweet.
          

Web-WWW-©-Scanrail-Fotolia.com_

URL Shortner Service – how to monetize?

In the world of microblogging where users are trying say more with less, it is important that the resource url do not take more than the actual message. Enter the “URL Shortner Services” such as Bit.ly, is.gd, TinyURL among others, to the rescue to shortened those long URLs. From a user point of view, this is not much life to shortener services once the url has been turned to a small set of characters. I personally do not believe that anybody uses url shorteners outside the microblogging realms. We all know that there is no such thing as a free service, especially if you are looking for the service to stick around for a while. Therefore one question comes to mind, how can we make monetize the service.

I was at bar in Canary wharf with one of my investment friend when we were discussing several monetization ideas in regard to social networking based businesses. Obviously, Twitter was discussed but then the conversation shifted toward the third party apps built for Twitter and microblogging sites.

I am sure that anybody using Twitter has come across those funny urls in twits. When you clicked a link, there is a redirection from the url hosting server to the original source. So how can we monetize this service? That same question again. Here are some ideas and you are welcome to agree or disagree.

  1. charge microblogging site and third party developer to use the service and implement a free quota. I do not personally see how this is going to work as the microblogging site are currently in the driving seats. This option might not be viable for the url shortener service providers.
  2. when a shortened url has been clicked, the hosting server should redirect them to their site and present the user with an advertisement which will be displayed for a few second or until the user clicks it off. The ads have to be targeted the user this might involve parsing the title of the original source for keywords. This solution might be a good approach but how will user react to this. It can put them off therefore the services might have to offer some incentives to the url referrer.
  3. This probably the simplest way and less intrusive from a user’s perspective. When a user clicks on a link, the original site should be opened into a frame which is integrated into url shortener service provider’s site. The top frame of the window could display some ads.

The above are just some suggestions and they might not even be feasible or might be receive with hostility from the community but it is worth trying. Here is my final option:

  • Twitter should implement is own url shortener service where the destination page will be displayed within a frame simialr to option 3 above. Or twitter should open all links within a frame and advertise similar to option 3 but this time without cutting out the the third party url shortener service providers.

That’s all folks!

3826183272_bf7dc0cd93

Next stop… Google Wave… I have received my invite!!!

I am sure by now, must geeks around the world have heard about Google Wave and its endless capabilities. I am not sure of the last one. Anyway, I have received my invitation from Google about 2 weeks, yippie, but I did not logged in until today mainly because I was too busy on other projects of mine.

When first logged, I have to admit that I was not very much impressed. I thought looked like a better version of Twitter based on the user interface. Nevertheless, I am looking to build a demo Wave application that can be embed into the Google Wave. I want to build something simple but I am not getting any inspiration. By the way, initially this blog entry was going to be a tutorial about how to build a wave app.  This is what I decided to do. I have created a wave or should i say a wavelet, which will be used to brainstorm ideas about what to build.

Ooh, I have an idea. I think that I should make a tutorial which will show to build a Google Wave application that extract phone numbers from “blips” and make it possible to dial them straight¬† using the “callto” protocol. This will allow a participant to dial a number from the wave app to let’s say their Skype phone. This should be simple enough. Now let’s find the time to do it. Hopefully my next entry will be it.

Until then, keep the channel open.

My Google Wave Screenshot

The Anatomy of Twitter Automated Mass Direct Message

So Twitter is the new web phenomenon and it is still shy of its third birthday. Since it has come to the scene, we had heard much about it in the media; about its popularity and security breach. So what is twitter? This is how I define it:

  1. It is a social (networking) site J where users can send short messages to each other in the same manner as SMS (short message service) and mainly targeted to the mobile market; but a lot of people still use the web to twit messages including myself.
  2. It is a “PUSH” application where when a user publishes a message all the registered interested parties will automatically receive a copy’ this is almost similar to RSS feeds. When using the Application Programming Interface to develop third-party client, it then becomes a “PULL” based broadcasting system.
  3. It is also a marketer best friend ( but I do not see how Twitter as a company can monetize on it), people on Twitter are more likely to click on a URL link, due to the fact that only 160 characters are allowed, either by curiosity or in hope to find out more about the subject of the twit.

The popularity of Twitter is a funny one. When the web first surfaced, technologists were discussing its lack of multimedia and its static type, then came Youtube and Facebook among others. Twitter on the other hand is based on text and that’s all! Takes much of its features from SMS but it does not even allow for smillies L
J but I am sure that they are probably working on that and hopefully it will not cause any problems to Twitter current internal commands.
Anyway getting back to the subject at hand (see blog title), let me explain how to implement automated direct messages on the Twitter platform.
Twitter has some cool functionalities but it has a lot of restrictions. First, there are two types of messages available on the platform:

  1. Public Messages ‚Äď this type of messages can be viewed by anyone with an account on twitter.
  2. Direct Messages ‚Äď these messages are private messages that you send to people and hidden from the public stream. They are a good of making sure that the recipients notices (not sure they will read it but that’s a start).

The platform allows you to monitor (tracking) words in the public stream which is very useful if you are running an automated Mass Direct Message campaign. So how does the mass message works?

  1. You can only send DM to people who are friends define by Twitter as:
    1. Friends are those who follow you and reciprocal meaning you need to be following them as well.
  2. Mentions and Replies messages are very useful for automated messages and will be of greater value if you do not have many friends.

To have an automated mass Direct Message, you need to do the following:

  1. Setup a (ro)bot which tracks certain words on twitter.
  2. Every time a track word is flagged, the bot should send a public reply to the originator.
    1. A public reply will make your message available in the public stream.
    2. A mention of the message originator will bring your message to his attention thus a very good marketing tool.
  3. Use a bot to send a DM messages to all your followers to make sure that they have a look at what you have to say.

Nothing is guarantee but when I run a test for a friend online shop, he was getting at least 100 visitors a day when he usually only gets about that a month or so. They are tools that can do the job but there is a chance of you getting banned by Twitter.
I have written a working prototype in JAVA, so it should be able to run on any platform, get in touch with me for a copy. It is free but I do not want to post it online in case it against the Twitter terms and conditions. You can also DM me on @armelnene.