Blog

Coding Conventions in Java

1. Avoid using printStackTrace() to log exceptionThis is because with printStackTrace(), the trace is written to System.err and its hard to route it elsewhere and even filtering is difficult. The end user has little control in the way the messages are shown.

Solution: Use Logger to log the exception. The reason is as follows:

a) With Logger, exception can be written to different locations (console, file) based on user preference.

b) Log statements can be filtered by severity (error, warning, info, debug etc) and origin (namely class or package based).

Example: Prefer this:

catch(Exception e) {

String message = String.format(“Exception occurred because of divide by zero %s” + e, reader);

LOGGER.warn(message);

}

over this:

catch(Exception e) {

e.printStackTrace();

}

2. Avoid using catch clause with Throwable or Error

Throwable is the superclass of all errors and exceptions in Java while Error is the      superclass of all errors and ideally these should not be caught by applications. Catching either throwable or error will also catch OutOfMemoryError and InternalError from which an application actually should not attempt to recover.

Throwable catches really everything even Thread Death which gets thrown by default to stop a thread from the now deprecated Thread.stop() method. So by catching throwable you can be sure that you’ll never leave the try block  and you should be prepared to also handle OutOfMemoryError and InternalError or StackOverflowError.

Hence, the best practice would be:

Example: Prefer this:

catch(Exception e)

over this:

catch(Throwable e) unless the code demands.

3. Define Constants instead of repeating String literals 

Constant will throw a warning if a mistake is done in constant name but during repeated usage of String literal, error might go unnoticed.

Another advantage, coding style is more consistent if we use constants.

Example: Prefer this

Final String RAW_BYTES = “RawBytes”;

map.put(RAW_BYTES, 123);

map.put(RAW_BYTES, 234);

map.put(RAW_BYTES, 546);

map.put(RAW, 345);       // Compiler will immediately report an error as RAW is not defined. So using constants prevent us to use wrong literals.

over this:

map.put(“RawBytes”, 123);

map.put(“RawBytes”, 234);

map.put(“RawBytes”, 546);

map.put(“Raw”, 345);   // This wont be detected as an error although we might have to use RawBytes as the name but there is no way compiler can detect this.

 

Apache Flink: Stream Processing

Apache Flink is a framework for distributed stream processing and in a very high level description, Flink is made up of the following:

Screen Shot 2017-08-06 at 13.55.26.png

where Data Source: is the input data to flink for processing.

Transformation: is the processing stage where different algorithms may be applied.

Data Sink: is the stage where flink sends the processed data. This could be kafka queue, cassandra etc.

Flink’s capability to compute accurate results on unbounded data sets is based on the fact that it has the following features:

  1. Exactly Once semantics for stateful computations: where stateful means  application can maintain a summary of the data that has been processed and checkpointing mechanism of Flink ensures exactly once semantics in the case of a failure. In other words, checkpointing allow Flink to recover state and positions in the stream in case of a failure.
  2. Flink supports stream processing with event time semantics where event time refers to the time at which each individual event occurred in the device. The event time semantics makes it easy to compute accurate results over streams when the events arrive out of order or with delay. So, the time at which the event occurred will be present in every event implies it will be easy to group and process the events by assigning them to their corresponding hour window. An hourly event time window will contain all records that carry an event timestamp that falls into that hour, regardless of when the records arrive, and in what order they arrive.

Screen Shot 2017-08-06 at 17.30.09

   3. Flink supports flexible windowing option where the windowing can be done  based on time, session or counts. Apache Flink supports different types of window such as tumbling window, sliding window, global window and session window. Time based window is created as soon as the first event corresponding to this window arrives and window is removed when the time (event time or processing time) passes its end timestamp + user specified allowed lateness.

Screen Shot 2017-08-06 at 18.42.30

    4. Flink’s save point is a mechanism to update the application or reprocess historic data with minimum downtime. Savepoints are externally stored checkpoints that can be used to update the Flink program. It uses Flinks checkpointing mechanism to create a snapshot of the state of the streaming program and write the checkpoint metadata to an external file system.

Screen Shot 2017-08-06 at 22.25.37.png

Example:

A simple word count algorithm using Apache Flink DataSet Api can be found at the github project Apache Flink Git Hub Project

Cassandra-Unit

Cassandra-Unit is a Java Utility test tool that helps us write isolated Junit Tests without having to mock or connecting to real cassandra. There are several ways to use the Cassandra-Unit in a Java project. Before to start the entire project is available at my Git Hub account: Cassandra-Unit Git Hub

This blog will describe how using Junit4 rule annotation (@Rule) cassandra-unit can be used.

  1. Create a CQL file describing the table that needs to be created and the data that needs to be inserted.
CREATE TABLE IF NOT EXISTS EMPLOYEE(
 id int,
 name text,

PRIMARY KEY (id)
);

INSERT INTO employee (id, name) VALUES(1, 'Lakshay');
INSERT INTO employee (id, name) VALUES(2, 'George');
INSERT INTO employee (id, name) VALUES(3, 'Andy');
INSERT INTO employee (id, name) VALUES(4, 'Nicole');

   2. Create Junit Test class using CassandraCQLUnit Rule annotation by mentioning the name of the CQL file created in Step 1 and the Cassandra Keyspace name that you prefer. This in itself sets up and starts an embedded Cassandra. This Cassandra unit active instance should be passed to the DAO/Repository class so that the cassandra session can be used to query the started database.

@Rule
public CassandraCQLUnit cassandraCQLUnit = new CassandraCQLUnit(new ClassPathCQLDataSet("cql/employee.cql", "emp_keyspace"));

private EmpRepository empRepository;

@Before
 public void setUp() throws Exception {
     // pass the cassandra session to the class.
     empRepository = new EmpRepository(cassandraCQLUnit.session);
 }

@Test
 public void testFindEmployeeById() throws Exception {
     EmpDetails empExpected = new EmpDetails(1, "Lakshay");
     EmpDetails empGenerated = empRepository.findEmployeeById(1);
     assertEquals(empExpected, empGenerated);
 }

3. The Java class that need to be tested is the following:

public class EmpRepository {

   private Session session;

   public EmpRepository(Session session) {
      this.session = session;
   }


   public EmpDetails findEmployeeById(int id) {
     EmpDetails empDetails = null;
     ResultSet resultSet = session.execute("select * from employee where id="+ id);
     
     for (Row row: resultSet) {
      empDetails = new EmpDetails(row.getInt("id"), row.getString("name"));
       }
    return empDetails;
 }

 

 

Rest Assured Testing

Introduction:

Unit Testing is done in order to test the individual Java classes in an application but there is also a need to carry out Functional Test in order to test the application end to end apart from the Integration Test which focuses on testing the modules. Rest assured testing thus enables us to test a RESTful application by connecting to the HTTP end points supported by the application and imitate the role of another client/browser. In other words, REST-assured is a way to automate testing of a REST Api.

Structure of a Rest Assured Test

The way Rest Assured Tests are written is in given(), when(), then() format.

1. A sample test code for a testing a GET end point in order to get users:


@Test
public void getUser() {
given()
      .contentType(ContentType.JSON) 
      .port(port). // HTTP port that got allocated at runtime. 
when()
      .get("/users/"). // GET endpoint in order to get the users
then()
      .statusCode(200) // RESPONSE expected
      .body("id", equalTo(128));
}

2. A sample test code for a testing a POST end point in order to add a user:


@Test
public void testAddUser() {
given()
      .contentType(ContentType.JSON)
      .port(port)
      .queryParam("empName", "Andy Murray") // parameters 1
      .queryParam("salary", 50000) // parameters 2 
      .queryParam("Id", 128). // parameters 3 
when()
      .post("/users/add"). //POST endpoint for adding user
then()
      .statusCode(200);
}

3. A sample test code for a testing a DELETE end point in order to remove a user with a specific Id:


@Test
public void testRemoveUserById() {

given()
.contentType(ContentType.JSON)
.port(port).
when()
.delete("/users/remove/{id}", 128). // DELETE end point
then()
.statusCode(200);
}

OR

given()
.contentType(ContentType.JSON)
.port(port)
.pathParam("id", 128). // path parameters specified here
when()
.delete("/users/remove/{id}"). // DELETE end point
then()
.statusCode(200);
}

Why Rest Assured?

  1. Easy HTTP Request Building and Execution: REST-assured allows us to easily define many things such as headers, query parameters, path parameters, request body etc.
  2. Easy Response Checking: REST-assured easily allows us to parse responses by providing constructs for making assertions on response body, response header etc.
  3. Ability to write clean code: REST-assured with its given() – when() – then() style helps to write and understand a code very easily. Predetermined things are under given(), condition under test is specified under when(), and the verification is done under then().

Complete Code Base:

Complete Example can be seen at my git repository: Rest Assured Test Complete Example

SOLID: 5 Object Oriented Design Principle for Software Development

There are 5 principles that any software developer should take into account while programming. One of the biggest advantage, if we follow these principles, in my opinion is that there will be reduced code smells. Here are the principles:

Single Responsibility Principle (S)

This principle tells us that a class should have a particular objective and it should not be created to serve more than 1 objective.

Bad Example:

public class Employee {
     private String empId;
   
     public void generateAnnualReport() {
       // .. generates the report
     }
     
     public void addNewEmployees(Employee e) {  
         // .. adds a new employee
     }
 }

The class above is not having a single responsibility but rather class has more than 1 purpose. Hence, according to the Single Responsibility principle

Good Example:

 public class Employee {
   
   private String empId;
   // getter and setters
 }

public class EmployeeReport {
   // class for generating the employee annual report
}

public class AddNewEmployee {
   // class for adding new employee
}

Open Closed Principle (0)

This principle means that software code (classes, methods, modules etc) should be open for extension but closed for modification.

√ This mean new behaviour can be added in order to meet the new requirements.

 ×This does not mean that new behaviour can be added by modifying the existing code.

Continuing with the previous example, Imagine there are several types of Employee Annual Reports as shown in the example below.

Bad Example:

 public class TravelExpenseReport {}
 public class MealExpenseReport {}

 public class ExpenseReport {
    
   public void getExpenseReport(Object typeOfReport) {
      if (typeOfReport instanceOf TravelExpenseReport) {
         // call the expense method of Travel Expense Report
      } else if (typeOfReport instanceOf MealExpenseReport) { 
 // call the expense method of Meal Expense Report }
 else { 
 // not supported exception
 }
}

 

The above example is bad because imagine for any new report addition, the if else structure will need to be modified thus Open Close Principle is violated.

Good Example:

public abstract class ExpenseBase {
     abstract void expense();
} 

public class TravelExpenseReport extends ExpenseBase {

     @Override
     public void expense() {
        // define the method
     }
}

public class MealExpenseReport extends ExpenseBase {
  
      @Override
 public void expense() { 
 // define the method
 }
}

public class ExpenseReport { 
 public void getExpenseReport(ExpenseBase expenseObj) {
 expenseObj.expense();
 }
 }

The above example is good because an addition of another expense report will not modify any of these classes itself but rather it would be an extension (as another class would be added).

Liskov Substitution Principle (L)

It states objects in a program should be replaceable with instances of their subtypes without altering the correctness of the program.  In other words, it states derived classes should extend the base classes without changing their behaviour. It is an extension of Open Closed principle.

Interface Segregation Principle (I)

It states that a class should not implement an interface if its not intended for its use. If not, a class might have methods that it is forced to implement but were not needed.

Bad Design:

public interface class ExpenseBase {
     void expense();
     void expenseMonthly();
     void expenseMealQuarterly();
} 

public class TravelExpenseReport implements ExpenseBase {

     @Override
     public void expense() {
       // define method
     }

     @Override
     public void expenseMonthly() {
      // define method
     }

     @Override
     public void expenseMealQuarterly() {
      // Do nothing as this method is not intended for class.
      // This class has been forced to implement this method
      }

}

public class MealExpenseReport implements ExpenseBase {

      @Override
 public void expense() { 
 // define method
 }
 
 @Override
 public void expenseMonthly() {
 // define method
 }

 @Override
 public void expenseMealQuarterly() {
 // define method
 }
 }

The above example is bad because Travel Expense Report class is forced to implement the method expenseMealQuarterly() even though its intended for Meal Expense Report class.

Good Design:

public interface class ExpenseBase {
     void expense();
     void expenseMonthly();
} 

public interface class ExpenseMealBase {

     void expenseMealQuarterly();
}

public class TravelExpenseReport implements ExpenseBase {

     @Override
     public void expense() {
       // define method
     }

     @Override
     public void expenseMonthly() {
      // define method
     }
}

public class MealExpenseReport implements ExpenseBase, ExpenseMealBase {

       @Override
 public void expense() { 
 // define method
 } 
 @Override
 public void expenseMonthly() { 
 // define method
 }
 @Override
 public void expenseMealQuarterly() { 
 // define method
 }
 }

The above example is good because interfaces have been segregated now and the class which does not need a specific interface will not implement and hence will not be forced to implement any method that it does not need it.

Dependency Inversion Principle (D)

This principle states that High Level modules should not depend on low level modules. Both of them should depend on abstractions. This mean a particular class should not depend on directly on another class but rather on an abstraction (or interface) of this class.

BAD DESIGN:

public class Employee {
   private EmployeeTeam empTeam;

   public String getEmpData(EmployeeTeam empTeam) {
      //... 
      return empTeam;
   }  
   //.....
}

This is a bad design because Employee and EmployeeTeam are two individual modules. Employee class (High Level) depends on EmployeeTeam (Low Level) class directly for the operations and hence it violates the principle of Dependency Inversion.

GOOD DESIGN:

public inteface EmpData {
    String getEmpData();
     //...
} 
public class EmployeeTeam implements EmpData {
 
   @Override
   public String getEmpData() {
   //...
   return ..
   }
}
public class Employee {
 private EmployeeTeam empTeam;

 public String getData(EmpData empData) {
    // .. return the data
 } 
 //.....
}

This design is considered good because we do not couple the classes together. EmployeeTeam class implements an interface which has a method getEmpData. Employee class will use this interface to call the EmployeeTeam method thus both High Level (Employee) and low level (EmployeeTeam) depend on abstraction now.


Aspect Oriented Programming with Spring

What is Aspect Oriented Programming (AOP)?

Aspect oriented programming (AOP) is a horizontal programming paradigm, where same type of behaviour is applied to several classes. In AOP, programmers implement these cross cutting concerns by writing an aspect then applying it conditionally based on a join point. This is referred to as applying advice. There are several examples of aspects like logging, auditing, transactions, security, caching, etc.

When AOP should be used?

Imagine there is class containing several methods and there is a need to perform logging action (common functionality) before the method call, after the method call and when the method returns.

Normally the way one would be adopt is to log before the method call, log after the method call, and log when the method returns for every method in the class like its shown here Non Aspect Oriented. This technique would involve a lot of CODE DUPLICATION. This is where Aspect Oriented Programming comes into play.

Basic terms related with AOP

Aspect: is the functionality that you want to implement. For instance, logging before every method call or after etc. Here Logging is an aspect.

Pointcut: is the expression that determines what method needs to be invoked where the functionality needs to be implemented. For instance, every method inside Addition class should have the functionality implemented would be written as

@Pointcut("execution(*com.project.MathOperation.Addition.*(..))")

Advice: When a particular pointcut expression is met, then the code that needs to be executed is called an advice.

JoinPoint: Execution of the method when the condition is met is called JoinPoint.

Weaving: The entire process of getting the method executed when the condition is met is called weaving

Practical Example of AOP

The following link to my github account shows how aspect oriented programming can be implemented with Spring and how it differs from non aspect oriented programming – Aspect Oriented Programming

Different ways of Dependency Injection

I explained in my previous post on Spring Framework [Dependency Injection] what is dependency injection? how spring finds the dependency? how does spring create and inject the service? etc. In this continued post, I will try to explain the three different ways of Dependency Injection.

  1. Field  Injection: Using autowired annotation on a field.
  2. Method Injection: Using autowired annotation on a setter method.
  3. Constructor Injection: Using autowired annotation on a constructor.

The order in which the injection takes place is constructor followed by field and then method.

Autowired annotation on a field is explained here in my git hub code [Autowired Dependency] and in the previous Dependency Injection post. In this post, we will cover the setter and constructor dependency injection.

Setter Based Dependency Injection:

It is done by container calling the setter method on your bean after invoking a no argument constructor or a no argument static factory method to instantiate your bean. Example of setter based dependency injection is shown here on my git hub code: Setter DI Example

Constructor Based Dependency Injection:

It is done by container calling the parameterised constructor of the class with each parameter representing the dependency on another class. Example of constructor based dependency injection is shown here on my git hub code: Constructor DI Example Code and Constructor DI Example Test

 

 

Spring Tutorial: Autowiring in Detail

How to autowire an interface when there are several implementations of that interface?

Problem: Consider an example where we have an interface defined as Math Interface which is implemented by two classes Add and Multiply . Now, the question is how to autowire the two classes? Autowiring like below on interfaces implemented by several classes will fail!

@Autowired

MathOperation mathOperation;

Somewhere in the class there is a call mathOperation.performOperation(); Spring is in trouble now as it does not know which performOperation() it should call? the one in Add or the one in Multiply class? The error that it gives is as follows: ERROR: Trying to find a bean but found two!

There are two solutions for this problem

  • Solution1: Use annotation qualifier specifying the instance of the class that will allow spring to identify which particular class to be invoked.

@Autowired

@Qualifier(value = addOperation)

MathOperations mathOperations1;

So now a call to mathOperations1.performOperation(); will call the method defined in Add class.

Similarly,

@Autowired

@Qualifier(value = “multiplyOperation)

MathOperations mathOperations2;

So now a call to mathOperations2.performOperation(); will call the method defined in Multiply class.

  • Solution2: Use the name of that specific implemented class object when auto wiring instead of using generic interface instance name. For example

@Autowired

MathOperations addOperation;

So a call  addOperation.performOperation(); will call the method defined in Add class.

Similarly,

@Autowired

MathOperations multiplyOperation;

So a call multiplyOperation.performOperation(); will call the method defined in Multiply class.

How to autowire an interface when there is only one implementation of that interface.

Imagine there exists this interface called ComplexMathOperation defined here: ComplexMathOperationInterface and the only implementation of this interface called DerivativeOperation defined here: DerivativeOperationClass . How to autowire in this scenario?

There are two simple ways to autowire:

  • Using the interface name@AutowiredComplexMathOperation complexMathOperation;

    With this a call to performComplexMathOperations() will have no issues.

  • Using the class name

@Autowired

DerivativeOperation derivativeOperation;

With this a call to performComplexMathOperations() will also have no issues     but this defeats the purpose of using interface for auto wiring as it directly uses the class name.

 

Spring Dependency Injection Tutorial with Code Example

What is dependency?

It is a technique where an object supplies the dependency of another object.

a) A dependency is an object that can be used as a service.

b) Injection is the passing of a dependency to  a dependent object (receiving class) that would use it.

How does Spring finds the dependency?

a) Spring creates the service that needs to be injected

b) Spring injects the service as a dependency to another class.

These things are handled by Spring Inversion of Control Container.

In other words, imagine this example

private Addition add = new Addition(); Here we create the instance using new Addition() and then assign it to add object so that add object can then be used later.

How do you tell Spring to create that service?

a) In order for spring to be able to “create the service“, an annotation called @Component needs to be added on that class which needs to be injected as a dependency. Component is basically a bean class.

b) In order for spring to be able to “inject the service” as a dependency, an annotation called @Autowired needs to be added above the service.

Who is creating Spring?

a) The way we create Spring is basically using the annotation @RunWith(SpringJUnit4ClassRunner.class) It instantiates the Spring framework and it needs an application context in order to instantiate.

b) Application context can be provided via the annotation @ContextConfiguration(locations={“/applicationContext.xml”})

How does Spring know where to search for a Component/Bean? Or What is there in the applicationContext.xml

a) Inside the application context xml file the aim is to specify the path where the spring should look for in order to find the components/bean that needs to be created as a service and then later injected.

b) The way the path is specified in the application context xml is using the format:

<context:component-scan base-package="----location to the package where the bean is present---"/>

This field component-scan tells the spring –  the location (package) where it should look for in order to find the component/bean. Spring then goes to that location and finds the component in order to create the service.

NOTE:

Refer to my github project in order to understand this tutorial: Spring Dependency Injection Example

 

Machine Learning: Supervised Vs Unsupervised Learning

“Supervised Learning”

As mentioned in my previous blog: Machine Learning Basics, we have a data set given and we know how our output should look like.

It can be classified into:

Regression Problems: The input variables are mapped to a continuous output function.

Classification Problems: The input variables are mapped to a discret output function.

Example 1:

Given – The size of house data and the

Objective 1: Predict the price based on size of the house.

Objective 2:  Whether the house sells for more or less than 20k$?

Objective 1 is based on the regression problem whereas the Objective 2 is based on the classification problem.

Example 2:

Given – A data set of patients. Objective: Determine if patient has diabetes or not?

This is an example of classification problem.

 

“Unsupervised Learning”

In this the datasets does not have any labels or might have some. The objective is to find some structure in the data. It might be used to find that data lies in 2 or x different clusters.

Example 1: Google News

It goes and looks at thousands of new stories on web and groups them into cohesive new stories. So for instance a news as shown below: The news title (Hindu Marriage Bill) is one but it has clustered all the news stories (India Today, Business Standard, Radio Pakistan, Pakistan Today) referencing this story. So this is what unsupervised learning is, trying to find structures in the data. Once found, similar set of the data form a cluster.

Screen Shot 2017-03-19 at 23.11.42.png

Example 2: Imagine we have 10K different genes and aim is to find a way to group the genes that are somehow related by some variable or similar. In other words, aim is to form clusters based on some factor such as lifespan etc.

Few other Examples:

a) Stock Market: Predict the price of a share tomorrow?  REGRESSION SUPERVISED

b) Weather Prediction: Whether it will rain tomorrow at 10Pm ? CLASSIFICATION SUPERVISED

c) Determine amount of rain that will occur tomorrow? REGRESSION SUPERVISED

d) Predict if a person will have diabetes in 3 years or no? Given the sample of DNA of a person. CLASSIFICATION SUPERVISED

e) Given an email, classify if its spam or not? CLASSIFICATION SUPERVISED

f) Given a database of customer data, discover the market segments. UNSUPERVISED

 

Machine Learning Basics

What is Machine Learning?

It is the science of getting computers to act without being explicitly programmed. In simple words, Machine Learning constructs the models and algorithms which will not only predict on data but also learn from it.

It involves but not limited to this:

a) trying to detect an output based on some relationship between input and output.

b) trying to identify some structures in the data and then predicting.

Real Life Examples

Here are the few examples where there is machine learning behind the scenes but probably you never got to know!

a) Facebook Image Detection Feature:- When you upload an image on facebook, it detects the objects by pointing out. This is nothing other than machine learning usage.

b) Email Spam Classification:- Distinguish an email from spam vs non spam is done using machine learning. This is done by classifying the words in a list as spam against others being non spam. Some famous algorithms for this are Support Vector Machine Algorithm (SVM), Bayesian Algorithm etc.

Standard Definition

As I was mentioning, ML is the act of getting computers to act but how will the computers act? A computer program is said to learn from

  • Experience E
  • with respect to some class of tasks T &
  • performance measure P

“if performance at tasks in T -> as measured by P ->  improves with experience E”

Example: E: experience of playing many games of checkers.

T: tasks of playing checkers.

P: probability that program will win next game.

Machine Learning Classification

Machine Learning can be broadly classified into two categories Supervised and Unsupervised Learning.

In supervised learning: There is an input set of data and the aim is to predict the output as we have an idea about the relationship between the input and output.

In un-supervised learning: Aim is to find some structure in data. It allows us to approach problems with little or no idea what our results should look like.