- The application shall allow the user to query a stock (quotes) either using the stock ticker or the company name and display the following: stream stock price with basic info, stock historical price; tabular view and financial chart analysis.
- The ability to rate, comments and share stock information.
- provide a real-time chat features for users of the application to interact together.
- single sign-on integration with OpenSocial, OpenID and Facebook connect.
The above are some basic features in order for people to actually use the site. As you can see, they are mostly front-end requirements. Here is another important requirement:
- The application will be hosted on a Cloud-based service, the application shall keep to a minimum the load on the server as CPU and memory use will be charged per request.
I believe that last requirement was the key decision factor. I needed something that will make use of the user resources instead of stressing the server. Something that would execute on the browser, client-side. Being a Java developer, it is only normal for me to look for something that I can grasp easily and freely available on the market. I looked at a few RIA frameworks and tools but I needed something that can integrate within my development environment without the need switch between IDEs.
I had a look at JavaFX and GWT. Having worked on GWT in the past, I found that I could implement most of the requirements. GWT has a rich api and components sets (widgets). I could use the Google search, map, visualization… to build the front-end and implement the client side code.
The application is a mashup of different services from around the net. Some of those services were available for free and others were not available at all but I will still be connecting and parsing the relevant data to displayed on the screen. As requirement “1” was working fine, there was a disaster about to happen. Here is the basic architecture:
- when a user queries a stock, the data is sent to the back-end services.
- the back-end services queries a third-party web service which I do not have control over. ZERO.
- when the third-party web service returns, the data is then parsed into a format understood by the front-end and sent to the front-end services to display.
The services was working fine until I had about 12 stocks displayed on my screen for testing purpose. The data was being streamed with a 1 minute interval. After about 30minutes later (and 360 requests = 12 *1 pm * 36m), I have noticed that the stock data was not being updated anymore and some screens were blank. No errors were logged and some other queries were still working such as the historical data. As I could not see the errrors from my debugging session, I therefore decided to query the third-party service manually through Firefox and IE and VOILA! The service provider had put a temporary ban on my IP address for making too many requests, it says. This was a major blocker as it seems that I could not access my primary data anymore. This left me questioning the feasibility of the project.
If a single user gets banned for tracking 12 stocks, what would happen when 100 or more users start watching their stocks?
OK, so I did not give up, at least not without a fight!
I looked at the problem from another perspective, the third-party view. How do they track my request as I do not have to register or sign-in? The obvious answer is my IP. So I thought, instead of making the request to their services from the back-end, I should send the request directly from the front-end. If each users make the same request, the third-party services would register different IP addresses for each one of them. Correct indeed but one big problem: