Coding Sandbox - Tutorials and How Tos

Category:
  • Coding
  • No categories

How to Take Screenshot of Websites Using Java and Firefox

5th November 2018 in Coding

Let’s set the scene why you would require to take a screenshot of website. A couple years ago, I was working for an intellectual property law firm who’s clients included some of the world largest software houses. I was working as a developer and my requirements was to scrap known website where the clients software were more likely to be distributed illegally, ie. torrents site.

This was back in ’09 and the initial PoC save the pages HTML which required to opened in a frame in order to be viewed. The application did not have a UI as the results was sent to Salesforce case management system. I recently came across a project trying to do the same thing but with illegal use of photos.

I decided to take a look at how this could be done but not only saving the HTML of the source page but to take a screenshot of what it looked at the time of crawling. There are many services now offers API to allow your application to take screenshots of website. But there is a simpler easier solution;

Introducing Firefox Headless Mode

Headless mode is a very useful way to run Firefox. Just as it might sound, Firefox is run as normal, minus any visible UI components visible. Though not so useful for surfing the web, it comes into its own with automated testing.

Firefox Headless mode allows to run Firefox as a background service. The headless mode as a

-screenshot
parameter which allows the browser to take a full page screenshot of a given URL. The following simple command would take a screenshot in Firefox:

/path/to/firefox -headless -screenshot https://developer.mozilla.com
You can omit the -headless flag when using the -screenshot flag as follow
/path/to/firefox -screenshot https://developer.mozilla.com
Please see Firefox Dev page for a list of all available options.

Take a screenshot from Java

A prerequisite is to make sure that Firefox is installed and accessible from your Java application.

The java code will execute the external Firefox process. I’m using Java 11 and the ProcessBuilder API but you can do the same with the Runtime API. The ProcessBuilder API gives you more control over the executed process.

It is important to remember that you can only have a single instance of Firefox running on your system. In a multi-threaded environment, you will control access to this process.

Here is a simple example of running Firefox in headless mode and taking a screenshot from Java; this is run from a MacOS.


package com.kiktronik.websitescreengraber;

import java.io.IOException;

/**
 *
 * @author Armel Nene
 */
public class Screenshot {
    
    public static void main(String[] args) throws IOException{
        Process p = new ProcessBuilder("/Applications/Firefox.app/Contents/MacOS/firefox", "--headless", "--screenshot", "armelnene.png", "https://www.armelnene.co.uk").start();
        System.out.println("Process ID: " + p.info());
    }
    
	}
	
I hope this would help you in your next project. Happy coding.

Liferay and AngularJS Made Simple: Connecting AngularJS to a Backend with REST and JSON

1st October 2014 in Coding

Introduction

Liferay is the leading Open Source Enterprise Portal. One may asked what an enterprise portal is and this question is very valid as it has been asked on every single Liferay project that I have worked. This blog post is not about defining what an enterprise portal is but it wouldn’t be a crime if we provided a brief definition:

An enterprise portal is a web application which provides services required by an enterprise such as: user management, authentication and authorisation services, ability to connect to third party applications and provide a single point of access to multiple applications, hence the “portal”.

The above is my own definition and it could be extended to encompass web content management, content management system (CMS) and single sign on (SSO). This post is about Liferay and the use of its web content management system (WCMS) to create single page applications using AngularJS. The motivation to create portlets using AngularJS instead of Java is as:

  • Portlet development using Java is very expensive
  • Not many Java developers with portlet experience
  • Java portlets development requires heavy duty tools such as build tools, IDE and JVM
  • Portlet developers need to be familiar with the Portlet API, lifecycle and framework

We will focus on Liferay available RESTful web services API but do remember that you can create your own custom web services using Liferay service builder SDK.

Liferay RESTFul API and Security

Liferay ReSTFul and SOAP API implement the same security as the core library:

  • API can be secured so that only authenticated users can access them (AUTHENTICATION)
  • API can be secured so that only users with the right roles can executed certain API calls (AUTHORIZATION)

When creating your own custom API, Liferay Services Builder will create the necessary permission for the web services API.
For a list of API available in Liferay, point your browser to the following

http://<your-server-address>:<your-server-port>/api/jsonws

Liferay will provide a means of testing the services calls when the above URL is loaded. Most services execution will require authentication or a secured token to be passed on with the calls. This level of a security is required in an enterprise environment. It is possible to stop Liferay from checking for the secured token in portal-ext.properties as

Auth.token.check.enabled=false

Software developments should promote code reuse, therefore by separating the business logic from the portlet code, developers can share the business logic with third party applications.

Why Use AngularJS to Create Web Applications (Not Portlets)?

This is not a tutorial on AngularJS. Developers should use the same approach for developing any AngularJS application to developing Liferay web applications.
AngularJS is a popular JavaScript framework promoting Object Oriented Development (OOD) and Model View Controller (MVC) to the JavaScript community. Java developers are already custom with the methodology through the use of Spring MVC and JSF for front end developments. Developers familiar with Google Web toolkit (GWT) should find themselves in familiar territory. Now to answer the question of why use AngularJS to create web applications on Liferay?
AngularJS is JavaScript and therefore can be executed in the browser without recompilation and redeployment. Liferay Web Content Management System (WCMS) provides an HTML editor and content versioning. Liferay JSONWS API runs on the same server and can be accessed through the JavaScript written in the WCMS. AngularJS modules can be written in a third party editor such Notepad++ and uploaded to Liferay Content Management System (CMS). The Liferay CMS provides a link to the latest version of the file which can be referenced in the HTML/ JavaScript code. By creating the web services in Java through Liferay Services Builder, the java developer can focus on the business logic – including testing. The front end developer can utilise his skills in HTML and JavaScript to develop the user interfaces and any necessary interactions with the backend through the ReSTFul services. There is a clear separation of work and accountability. The learning curve for the Java developers to create the services will be minimal. To preview the live code, the frontend developer only has to save the content (WCMS) and refresh the page to see the latest changes.
Here is a quick example:

 

 <div ng-app="" ng-controller="companiesController">  
   <ul>  
    <li ng-repeat="x in data">{{'title: ' +x.title + ', group Id: ' + x.groupId }}</li>  
   </ul>  
 </div>  
 <script>  
   function companiesController($scope,$http) {  
    $http.post("http://localhost:8080/api/jsonws/assetentry/get-company-entries/company-id/10157/start/0/end/5?p_auth=cbSXanJ2")  
    .success(function(response) {$scope.data = response;});  
   }  
   companiesController.$inject = ['$scope', '$http'];  
 </script><script src="http://ajax.googleapis.com/ajax/libs/angularjs/1.2.15/angular.min.js"></script>  

You can copy the above in a new web content article and see the result displayed on the page. Make sure to change red bold values to your system specific:

  • Company id: for ease of testing, you can retrieve that value from the control panel
  • P_auth: this value can be looked up programmatically as it will change every time the user logs into the portal

Traditionally, the simple code above would require a JavaServer page or JSF application which is slow for rapid prototyping. In software testing, portlets controller are one of the most complex components to test. By creating a clear separation between components, testers and automated tools can test each components individually. The sample code pulls information from Liferay REST web services and displays a list of registered companies on the page.

Conclusion

Liferay has a rich set of features which allows developers to create enterprise components and applications. When working with ReSTFul services, Liferay Web Content Management editor can act as an Integrated Development Environment in the browser. AngularJS is approaching maturity and it is very popular with web developers. Liferay 7 (next release as of writing ) will be introducing single page portlets but this is already possible with AngularJS and ReSTFul web services API. Needless to say that you can use any web browser to create content in Liferay CMS and debug your code in real time using tools such as Firebug.

Hadoop vs Java Batch Processing JSR 352

21st August 2013 in Coding, Uncategorized
Hadoop has become synonymous to Big Data. Oracle has release the latest standard to Java EE stack: Batch Processing JSR 352. Batch processing has been around for decades and there are many Java framework already available such Spring Batch. This talks provides a perspective about Hadoop and JSR352. Knowing when to use or the other or both together.