The many types of technology clouds

If you remember from school, there are many different types of clouds. You have stratus, cumulus and nimbus just to name a few. In technology, the cloud is no different. The cloud has a different meaning depending on who you talk to and they are all mostly right but they can be referring to totally different things.

Cloud Architecture

Let’s look at the Wiki definition of a cloud

Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over anetwork (typically the Internet). The name comes from the use of a cloud-shaped symbol as an abstraction for the complex infrastructure it contains in system diagrams. Cloud computing entrusts remote services with a user’s data, software and computation.

There are many types of public cloud computing:[1]

In the strictest sense, cloud computing is not merely just moving your software or services onto a server outside your firewall, but it’s enlisting a service with “cloud” properties like flexible horizontal scalability through elasticity. Cloud architecture is not at all uniform as well. It also means different things to the vendors that provide the services and it they implement it how they choose. Amazon, VMWare Cloud Foundry, Stackato and Azure are some of the hottest cloud architecture services out there today.

Consumer Cloud

From a consumers perspective, the cloud means that your data is stored on the internet somewhere. For example, the iCloud product from Apple keeps all your data in sync between computers and iOS devices. The iTunes Match service stores your music on servers and the same goes for the Google Music service.

Outsourced Cloud

To many IT professionals, the cloud is just a way to offload services they currently have in house to a server outside of their infrastructure managed by someone else to varying levels. Take for example that many organizations have shifted having internal mail servers to migrating to Google for Business, where every aspect of the infrastructure is managed by Google. Another example, might be moving your internal Microsoft Exchange mail server to a hosting provider. The end result, again to varying degrees, is greater reliability and less burden and reliance on internal IT resources. JFrog Artifactory for dependency management has a cloud service or a server you can download. If I purchase this service, can I say that my dependency management is in the cloud regardless of how JFrof implements their cloud service? The answer to this question is yes.

The True Cloud

While the cloud may mean different things to different people, preparing an application for a cloud architecture is not as simple as one might think. There are all kinds of decisions that need to be made and coding practices that should be adhered to. Typically, if you follow Java conventions, then you won’t have much of an issue with cloud deployments. Let’s look at an example Java web application and some of the concerns that need to be addressed prior to cloud deployment to a service such as Cloud Foundry.

  • Your application must be bundled as a war file
  • You application must not access the file system. (Utilize the class loader to load files from within the war, but don’t write to files)
  • Resources/Services should be looked up via JNDI
  • If you cache, utilize caching solutions like EHCache that propagate automatically across instances
  • Persist data to a database such as MongoDB, MySQL or PostgreSQL to guarantee access

For the most part, if you follow the J2EE web application conventions regarding deployments, you won’t have an issue with a deployment to a true cloud environment.

Cloud Architecture Dynamic Scaling vs. Autoscaling

One of the advantages to a cloud architecture is the ability to scale up instances of your application and only pay for what you utilize from your provider. There are two types of scaling, the first is Auto-scaling where a tool is utilized to measure load and will automatically increase instances of your application to accommodate the need and the second if dynamic scaling where you control the scale based on a load forecast. An example of dynamic scaling would be if you normally have a load of 1000 concurrent users and you have an event which will ramp up your load to 10,000. In a more traditional setting, you would have purchased servers and hardware to accommodate the worst case scenario which leaves your infrastructure largely utilized most of the time causing inefficiency and waste. In a cloud scenario,you might have 10 instances to service your normal load and for the 24 hour period of your event, you scale to 100 instances and when the smoke clears, you return to 10 instances. All this can be done with a single click of a button in a cloud infrastructure.

Automatic scaling is provided by some cloud providers in the form of a tool. Amazon provides such a tool that will monitor the load and allow you to set triggers to determine how many instances to ramp and when. Automatic scaling is very useful as it handles the unforeseen, but it should not be used in lieu of true capacity planning. It can also be dangerous and advanced intelligence needs to be built in to determine the different between junk and legitimate traffic. For example, think of what would happen if a DoS attack was performed against your application. Remember you pay based on the number of servers you ramp up.

Under the hood, scaling works by issuing commands to the server through an API. For example, with Cloud Foundry, the vmc command will let you monitor load and add instances which in turn creates a JSON request that gets sent to the server to give it instruction. You can also use a third party tool that interfaces with it in the same fashion or you can also build in the scaling intelligence into your own application by hitting the server API yourself. Using the latter technique, your application can control itself, making it very intelligent.

Conclusion

Regardless of how you see the cloud and how you utilize it, the endgame for your organization should be to offload the burden from your internal staff, decrease your expenses, provide greater uptime and flexibility and give you the ability to scale dynamically.

 

If you are using Java, you should be using Spring

I spend a fair amount of time evangelizing the Spring Framework, and with good reason. Spring is not only a great lightweight container framework that provides IoC (Inversion of Control) and Dependency Injection, but it pretty much has a tool, component for every task that you can think of when dealing with the day to day ins and outs of programming.

You probably have a need for Spring and you don’t even know it if you haven’t used it before. Most developers at one time or another have created frameworks to accomplish tasks like remoting, JMS, MVC, database interactions, batch work, etc., so I would label Spring as “The framework” for such tasks instead of succumbing to the “roll your own” urge. I started using Spring back in 2006 and I have not, in over 50 Java projects since then, neglected to utilize it to some extent. It has reduced the amount of code I have to write, allowed me to dynamically wire dependencies together at runtime and even provided tools for tasks that I thought I was going to have to write something custom to accomplish.

Spring was born as a solution to the heavy-weight EJB and J2EE container environments. It reduces the overhead of J2EE, allows the usage of containers that are not J2EE compliant like Tomcat and Jetty and provides a consistent API that most developers these days are familiar with. Here are some example of what Spring can do:

  • Dependency Injection (e.g. create a database pool object factory in XML and inject that into objects at runtime)
  • Eliminates the need to write specific code for singleton patterns
  • Allows you to turn a POJO into a service with a mere annotation
  • With Aspects, it allows you to inject values into classes that are not managed by Spring
  • Spring has an abstraction on top of over 100 different frameworks in Java
  • Spring MVC is the most concise and robust MVC framework
  • Spring provides JPA, Hibernate and DataNucleus support and will allow transaction demarkation
  • Spring provides AOP capabilities to allow method interception and point cuts
  • Exposing POJO methods as web services is as simple as adding Apache CXF to the mix
  • Annotation support is richer than any other framework
  • Spring is the most widely used Java framework
  • Property file loading and substitution in XML

Spring is not only a Java tool, in fact, Spring.NET is available for the .NET platform. It is usually a little bit behind the Java version but it is out there.

What are these new concepts AOP, IoC and Dependency Injection?

Usually a discussion of Spring always amounts to explaining the concepts that are at the core of the framework. Let’s take a look at each of them and what they give you. IoC and Dependency Injection go hand in hand. IoC is the concept and Dependency Injection is the mechanism. For example, you create a service class on your own and now you need to manage that class by ensuring it only has one instance, you also need to get that class reference to other classes to utilize so you create a mechanism for that. Now you need transaction support so you write that in, but you also need to dynamically read in properties that are for the environment you are running in and it goes on and on. As you can see, it not only gets complicated, but that is a lot of code you are writing and maintaining yourself. Spring provides it all and through XML or annotations (preferrable the latter), you can with one simple Plain Old Java Object (POJO) accomplish all of this through Spring conventions and inject the values of the objects into your service or inject your service into any other class by simply this.


//Service class Spring Bean
@Service
@Transactional
public MyService implements IService {

public void doThis();

}

//MVC Controller Class

@Controller
public MyController {

@Autowired
MyService myService

public Report doThat() {

myService.doThat();

}

}

In just a few lines of code, we created a singleton service that is transactional and we created a controller to call that service. There are way more complex things we could do here. For example, using the OpenSessionInView Pattern, we could get the Controller to control opening the transaction and closing it to allow for multiple service calls to use the same transactional context. We could also change the isolation level of the transaction. The point here is that we used Dependency Injection to demonstrate what IoC can do.

AOP or Aspect Oriented Programming is an advanced concept in the Spring world. AOP is merely the separation of cross-cutting concerns. The Transactional support of Spring is similar to AOP. The ability of Spring and AspectJ to allow you to inject objects into other objects that are not managed by Spring is another great example. Transactions, Security, Logging and anything other than the business at hand is a cross cutting concern. The goal of AOP is to separate those away from the code. If you didn’t use AOP, then you would have to actually control these elements your self. Take for example that you want to check that a user is valid before method calls. Without AOP, you would have to write some checkUserIsValid() method and call it at the beginning of each method. Using AOP, you could merely mark with an annotation or Aspects that each method of a certain class call another method on another class as an interceptor.

Spring is also for simple projects

You may be thinking Spring is too heavy weight for the task at hand… nonsense. I will guarantee that Spring, used properly, will reduce the amount of code in your project by at least 25%. That is 25% less code for your to maintain or write in the first place. Also, Spring provides even tools to accomplish the small tasks such as the following:

  • Finding resources on the classpath or file system (ResourceLocator)
  • Finding classes with a certain annotation
  • Generating JSON/XML from objects and vice versa (Jackson support)
  • load multiple property files and substitute variables inside of your spring XML files (Useful when promoting to different environments)
  • Ability to treat a JNDI resource as just another bean
  • Ability to treat a web service as just another bean
  • JDBCTemplate for issuing database queries and batch framework for batch operations
  • Spring Data for NOSQL support with Mongo
  • MVC Content negotiation to convert a POJO to JSON,XML, PDF (iText), Excel (POI) and more
  • Security integration that supports Windows, Active Directory, Kerberos, Basic, Digest, etc.
  • Robust Testing adapters for Junit and TestNG

I could spent a week delivering Spring training to a group of developers and only scratch the surface of what is there. Without a doubt though, when I have a tough problem to solve or a simple Java project, I always utilize parts of Spring. Conveniently, Spring is broken up into modules so that you can only include the ones that have the functionality you need to avoid causing any project bloat.

Conclusion

With Spring being the #1 Java framework, I highly recommend spending some time getting familiar with it and I recommend getting some training as well from someone like myself who is an expert with the framework who can show you everything it has to offer before you start utilizing it. You can also get training directly from vmWare, the company that owns SpringSource.

Why you should be using MongoDB/GridFS and Spring Data…

I recently delved into MongoDB for the first time, and albeit I was skeptical at first, I now believe it is my preference to use a NOSQL database over a traditional RDBMS. I rarely just fall in love with a new technology but the flexibility, ease of use, scalability and versatility of Mongo are good reasons to give it a chance. Here are some of the advantages of MongoDB.

  • NOSQL – A more object oriented way to access your data and no complex SQL  command to learn or remember
  • File Storage – Mongo is a master of storing flat files. Relational databases have never been good at this.
  • No DBA – The requirement of database administration in greatly minimized with NOSQL solutions
  • No schema, complex structures or normalization. This can be a good thing and also bad. Inevitably everyone has worked on a project that has been over normalized and hated it.
  • No complex join logic

Spring Data for Mongo

My first stop when coding against Mongo was to figure out how Spring supported it and without fail, I was not disappointed. Spring Data provides a MongoTemplate and a GridFSTemplate for dealing with Mongo. GridFs is the Mongo file storage mechanism that allows you to store whole files into Mongo. The Mongo NOSQL database utilizes a JSON-like object storage technique and GridFS uses BSON (Binary JSON) to store file data.

As the name implies, a NOSQL database doesn’t use any SQL statements for data manipulation, but it does have a robust mechanism to accomplish the same ends. Before we start interacting with Mongo, let’s look at some of the components I used to accomplish the examples I am going to show you.

  • Spring 3.1.0.RELEASE
  • Spring Data for MongoDB 1.1.0.M2
  • Mongo Java Driver 2.8.0
  • AspectJ (Optional) 1.7.0
  • Maven (Optional) LATEST

The very first thing we need to configure is our context.xml file. I always start a project with one of these but I use Spring annotations as much as possible to keep the file clean.

 <?xml version="1.0"?>
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns="http://www.springframework.org/schema/beans" 
        xmlns:context="http://www.springframework.org/schema/context"
	xmlns:mongo="http://www.springframework.org/schema/data/mongo"
	xmlns:tx="http://www.springframework.org/schema/tx"
        xsi:schemaLocation="http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-beans.xsd


http://www.springframework.org/schema/data/mongo

        http://www.springframework.org/schema/data/mongo/spring-mongo.xsd">

	<!-- Connection to MongoDB server -->
	<mongo:db-factory host="localhost" port="27017"
		dbname="MongoSpring" />
	<mongo:mapping-converter id="converter"
		db-factory-ref="mongoDbFactory" />

	<!-- MongoDB GridFS Template -->
	<bean id="gridTemplate" class="org.springframework.data.mongodb.gridfs.GridFsTemplate">
		<constructor-arg ref="mongoDbFactory" />
		<constructor-arg ref="converter" />
	</bean>

	<mongo:mongo host="localhost" port="27017" />

	<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
		<constructor-arg ref="mongoDbFactory" />

	</bean>

	<context:annotation-config />
        <context:component-scan base-package="com.doozer" />
	<context:spring-configured />

</beans>

In short, the context file is setting up a few things.

  • The database factory that the templates will use to get a connection
  • The MongoTemplate and GridFSTemplate
  • Annotation support
  • Annotation @Configuration support if needed (Optional)

Let’s take a look at my App class that is the main entry point for this Java application.

...
@Configurable
public class App
{

@Autowired
public MongoOperations mongoOperation;
@Autowired
public StorageService storageService;

ApplicationContext ctx;
public App() {

ctx = new GenericXmlApplicationContext("mongo-config.xml");
...

I am using AspectJ to weave my dependencies at inject them at compile or load time. If you are not using AspectJ, you need to lookup the MongoOperation and StorageService from the Context itself. The Storage Service is a simple @Service bean that provides an abstraction on top of the GridFsTemplate.

...

@Service("storageService")
public class StorageServiceImpl implements StorageService {

@Autowired
private GridFsOperations gridOperation;

@Override
public String save(InputStream inputStream, String contentType, String filename) {

DBObject metaData = new BasicDBObject();
metaData.put("meta1", filename);
metaData.put("meta2", contentType);

GridFSFile file = gridOperation.store(inputStream, filename, metaData);

return file.getId().toString();
}

@Override
public GridFSDBFile get(String id) {

System.out.println("Finding by ID: " + id);
return gridOperation.findOne(new Query(Criteria.where("_id").is(new ObjectId(id))));
}

@Override
public List listFiles() {

return gridOperation.find(null);
}

@Override
public GridFSDBFile getByFilename(String filename) {
return gridOperation.findOne(new Query(Criteria.where("filename").is(filename)));
}
}

...

Our StorageServiceImpl is merely making calls to the GridOperations object and simplifying calls. This class is not strictly necessary since you can inject the GridOperations object into any class, but if you are planning on keeping a good separation to be able to extract Mongo/GridFS later to go with something else, this makes sense.

Mongo Template

Now, we are ready to interact with Mongo. First lets deal with creating and saving some textual data. The operations below show a few examples of interacting with data from the Mongo database by using the MongoTemplate.

User user = new User("1", "Joe", "Coffee", 30);
//save
mongoOperation.save(user);
//find
User savedUser = mongoOperation.findOne(new Query(Criteria.where("id").is("1")), User.class);
System.out.println("savedUser : " + savedUser);
//update
mongoOperation.updateFirst(new Query(Criteria.where("firstname").is("Joe")),
Update.update("lastname", "Java"), User.class);
//find
User updatedUser = mongoOperation.findOne(new Query(Criteria.where("id").is("1")), User.class);
System.out.println("updatedUser : " + updatedUser);
//delete
// mongoOperation.remove(
//      new Query(Criteria.where("id").is("1")),
//  User.class);
//List
List<User> listUser =
mongoOperation.findAll(User.class);
System.out.println("Number of user = " + listUser.size());

As you can see, it is fairly easy to interact with Mongo using Spring and a simple User object. The user object is just a POJO as well with no special annotations. Now, let’s interact with the files using our StorageService abstraction over GridFs.

//StorageService storageService = (StorageService)ctx.getBean("storageService"); //if not using AspectJ Weaving
String id = storageService.save(App.class.getClassLoader().getResourceAsStream("test.doc"), "doc", "test.doc");
GridFSDBFile file1 = storageService.get(id);
System.out.println(file1.getMetaData());
GridFSDBFile file = storageService.getByFilename("test.doc");
System.out.println(file.getMetaData());
List files = storageService.listFiles();

for (GridFSDBFile file2: files) {
System.out.println(file2);
}

The great thing about Mongo is that you can store metadata about the file itself. Let’s look at the output of our file as printed by the code above.

{ "_id" : { "$oid" : "502a61f6c2e662074ea64e52"} , "chunkSize" : 262144 , "length" : 1627645 , "md5" : "da5cb016718d5366d29925fa6a2bd350" , "filename" : "test.doc" , "contentType" : null , "uploadDate" : { "$date" : "2012-08-14T14:34:30.071Z"} , "aliases" : null , "metadata" : { "meta1" : "test.doc" , "meta2" : "doc"}}

Using Mongo, you can associate any metadata with your file you wish and retrieve the file by that data at a later time. Spring support for GridFS is in its infancy, but I fully expect it to only grow as all Spring projects do.

Query Metadata

The power of Mongo also lies in the metadata concepts that I mentioned earlier and relational databases just don’t have this concept. Mongo stored implicit metadata about the files and it also allowed me to attach any data I wish onto a metadata layer. You can query this data in the same fashion you would query Mongo directly by using the . notation.

gridOperation.findOne(new Query(Criteria.where("metadata.meta1").is("test.doc")));

Map Reduce

Mongo offers MapReduce, a powerful searching algorithm for batch processing and aggregations that is somewhat similar to SQL’s group by. The MapReduce algorithm breaks a big task into two smaller steps. The map function is designed to take a large input and divide it into smaller pieces, then hand that data off to a reduce function, which distills the individual answers from the map function into one final output. This can be quite a challenge to get your head around when you first look at it as it requires embedding scripting. I highly recommend reading the Spring Data for Mongo documentation regarding Map Reduce before attempting writing any map reduce code.

Full-Text Search

MongoDB has no inherent mechanisms to be able to search the text stored in the GridFS files, however, this isn’t a unique limitation as most relational databases also have problems with this or require very expensive addons to get this functionality. There are a few mechanisms that could be used as a start to writing this type of mechanism if you are using the Java language. The first would be to just simply take the text and attach it as metadata on the file object. That is a really messy solution and screams of inefficiency, but for smaller files is a possibility. A more ideal solution would be to use Lucene and create an searchable index of the file content and store that index along with the files.

Scaling with Sharding

While very difficult to say in mixed company, Sharding describes MongoDB’s ability to scale horizontally automatically. Some of the benefits of this process as described by the Mongo web site are:

  • Automatic balancing for changes in load and data distribution
  • Easy addition of new machines without down time
  • Scaling to one thousand nodes
  • No single points of failure
  • Automatic failover

Configuration

  • One to 1000 shards. Shards are partitions of data. Each shard consists of one or more mongod processes which store the data for that shard. When multiple mongod‘s are in a single shard, they are each storing the same data – that is, they are replicating to each other.
  • Either one or three config server processes. For production systems use three.
  • One or more mongos routing processes.

For testing purposes, it’s possible to start all the required processes on a single server, whereas in a production situation, a number of server configurations are possible.

Once the shards (mongod‘s), config servers, and mongos processes are running, configuration is simply a matter of issuing a series of commands to establish the various shards as being part of the cluster. Once the cluster has been established, you can begin sharding individual collections.

Import, Export and Backup

Getting data in and out of Mongo is very simple and straight forward. Mongo has the following commands that allow you to accomplish these tasks:

  • mongoimport
  • mongoexport
  • mongodump
  • mongorestore

You can even delve into the data at hand to export pieces and parts of collections by specifying them in the commands and mixing in . notation or you can choose to dump data by using a query.

$ ./mongodump --db blog --collection posts --out - > blogposts.bson

$ ./mongodump --db blog --collection posts
    -q '{"created_at" : { "$gte" : {"$date" : 1293868800000},
                          "$lt"  : {"$date" : 1296460800000}
                        }
        }'

Mongodump even takes an argument –oplog to get point in time backups. Mongo’s backup and restoration utilities are as robust as any relational database.

Limitations of MongoDB

Mongo has a few limitations. In some ways, a few of these limitations can be seen as benefits as well.

  • No Joining across collections
  • No transactional support
  • No referential integrity support
  • No full text search for GridFS files built in
  • Traditional SQL-driven reporting tools like Crystal Reports and business intelligence tools are useless with Mongo

Conclusions

The advantages of MongoDB as a database far outweigh the disadvantages. I would recommend a Mongo NOSQL database for any project regardless of what the programming language you are using. Mongo has drivers for everything. I do however think that if you are in a certain scenarios where you are dealing with rapid, realtime OLTP transactions, MongoDB may fall short of competing with a high performance RDBMS such as Oracle, for example. For the average IT project, I believe Mongo is well-suited. If you still aren’t sold on Mongo by now, (I would be pretty shocked if you weren’t), then feast your eyes on the high-profile sites that are using MongoDB as their backend database today.

  • FourSquare
  • Bit.ly
  • github
  • Eventbrite
  • Grooveshark
  • Craigslist
  • Intuit

The list goes on and on… There are also several other NOSQL solutions out there that enjoy popularity.

  • CouchDB
  • RavenDB
  • CouchBase

Optional Components

I used several optional components for my exercises. I wanted to address these for the folks who may not be familiar with them.

AspectJ and @Configurable

Many folks would ask why I chose to use Aspect Weaving instead of just looking up the objects from the context in the App object. @Configurable allows you to use the @Autowired annotation on a class that is not managed by the Spring context. This process requires load-time or compile-time weaving to work. For the purposes of Eclipse, I use the ADJT plugin and for Maven, I use the AspectJ plugin to achieve this. The weaving process just looks for certain aspects and then weaves the dependencies into the byte code. It does solve a lot of chicken and egg problems when dealing with Spring.

Maven

If you are using Maven and you want all of the dependencies I used for the examples, here is the pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.doozer</groupId>
	<artifactId>MongoSpring</artifactId>
	<packaging>jar</packaging>
	<version>1.0</version>
	<name>MongoSpring</name>
	<url>http://maven.apache.org</url>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<spring.version>3.1.0.RELEASE</spring.version>

	</properties>

	<dependencies>
		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>4.8.2</version>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-api</artifactId>
			<version>1.6.6</version>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>jcl-over-slf4j</artifactId>
			<version>1.6.6</version>
			<exclusions>
				<exclusion>
					<artifactId>slf4j-api</artifactId>
					<groupId>org.slf4j</groupId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-log4j12</artifactId>
			<version>1.6.6</version>
			<exclusions>
				<exclusion>
					<artifactId>slf4j-api</artifactId>
					<groupId>org.slf4j</groupId>
				</exclusion>
			</exclusions>
		</dependency>

		<!-- Spring framework -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-context</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-aop</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-aspects</artifactId>
			<version>${spring.version}</version>
		</dependency>

		<!-- mongodb java driver -->
		<dependency>
			<groupId>org.mongodb</groupId>
			<artifactId>mongo-java-driver</artifactId>
			<version>2.8.0</version>
		</dependency>

		<dependency>
			<groupId>org.aspectj</groupId>
			<artifactId>aspectjweaver</artifactId>
			<version>1.7.0</version>
		</dependency>

		<dependency>
			<groupId>org.aspectj</groupId>
			<artifactId>aspectjrt</artifactId>
			<version>1.7.0</version>
		</dependency>
        <dependency>
			<groupId>org.springframework.data</groupId>
			<artifactId>spring-data-mongodb</artifactId>
			<version>1.1.0.M2</version>
		</dependency>

		<dependency>
			<groupId>cglib</groupId>
			<artifactId>cglib</artifactId>
			<version>2.2</version>
		</dependency>

		<dependency>
			<groupId>javax.persistence</groupId>
			<artifactId>persistence-api</artifactId>
			<version>1.0</version>
			<scope>provided</scope>
		</dependency>

	</dependencies>

	<build>
		<plugins>
			<plugin>
				<artifactId>maven-compiler-plugin</artifactId>
				<configuration>
					<source>1.6</source>
					<target>1.6</target>
				</configuration>
			</plugin>

			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-dependency-plugin</artifactId>
				<executions>
					<execution>
						<id>copy-dependencies</id>
						<phase>prepare-package</phase>
						<goals>
							<goal>copy-dependencies</goal>
						</goals>
						<configuration>
							<outputDirectory>${project.build.directory}/lib</outputDirectory>
							<overWriteReleases>false</overWriteReleases>
							<overWriteSnapshots>false</overWriteSnapshots>
							<overWriteIfNewer>true</overWriteIfNewer>
						</configuration>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-jar-plugin</artifactId>
				<configuration>
					<archive>
						<manifest>
							<addClasspath>true</addClasspath>
							<classpathPrefix>lib/</classpathPrefix>
							<mainClass>com.doozer.mongospring.core.App</mainClass>
						</manifest>
					</archive>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>aspectj-maven-plugin</artifactId>
				<configuration>
					<complianceLevel>1.6</complianceLevel>
					<aspectLibraries>
						<aspectLibrary>
							<groupId>org.springframework</groupId>
							<artifactId>spring-aspects</artifactId>
						</aspectLibrary>
					</aspectLibraries>
				</configuration>
				<executions>
					<execution>
						<goals>
							<goal>compile</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>

Looking for a unicorn: HR and IT recruiters are overlooking potential candidates

Over the past few years, I’ve been watching more and more IT candidates get overlooked for jobs for various reasons, despite that they are fully capable and qualified. A person can be disqualified for a job position by a recruiter or human resources representative for a number of valid reasons.

  • Not enough experience
  • Not enough experience in the right field or the right technology set
  • Not enough academic education background or lack of a degree
  • Poorly worded resume or one that just doesn’t advertise their skills
  • An overwhelming resume that is too long and contains too much info

By far the most common element I have seen over the past few years is that the HR personnel and recruiters are trying to match requirements from managers verbatim with a candidate’s resume, resulting in completely qualified candidates being overlooked because the lacked a certain keyword. The phrase “looking for a unicorn” has been used to describe the process of looking for someone to fit the exact set of requirements you have in hand. I even saw one example where a company was looking to fill a position and disqualified every candidate because the last title held didn’t match the title of the position they had an opening for. The twist here is that the title of the position that was open was unique to that company and only one person in that company had ever held it. This is like saying I need to drive a nail and I have to have a framing hammer and no other hammer will do. I don’t want a claw hammer, ball-peen hammer, sledge hammer or a rock hammer (which all of these will drive a nail, some better than others I might add) but I just have to have a framing hammer. Why you ask? Because someone said framing hammer at some point so I inferred only a framing hammer would do.

A near fit is still a fit

In IT, I see examples of overlooked candidates every day. For example, a client wants a developer with Dojo  (A Javascript Framework) experience… now being a developer for many years, I understand that looking for someone with experience with Dojo is going to be near impossible, especially if I narrow my search to local candidates only. Also, as an IT professional, I understand I could fill this position with any developer who has extensive Javascript experience or ExtJS, Prototype, NodeJS, Scriptaculous or YUI Javascript frameworks. Many times though, the client is an HR representative who was given the requirement from a software development manager so they want to see Dojo experience on the resume and won’t waiver in the requirement. This results in some recruiters just tempting the developer to just put Dojo on the resume and try to pass them off. This would seem like a reasonable thing to do if the developer is confident they can just acquire the new framework quickly, but it could get you into trouble when the client interviews the candidate and starts asking about specific Dojo questions. It hurts the recruiter’s credibility and the person interviewing or the job. Honesty is always the best policy so we have to address the problem by training people to either learn a little more about technologies and how they relate or get them to lean on a technical resource to aide in screening the candidates.

No one seems willing to train anymore

10 years ago, people understood that after hiring a candidate there is a certain amount of training that is necessary, in fact, most companies were perfectly allright with hiring someone who didn’t match at all so long as they felt they could train them to do the job and had confidence that they would stay with the job. A lot of the problem  lies not only with the recruiters and human resources personnel and how they match requirements to candidates but with the economy. Employers think they can afford to be really picky and not pay premium rates for IT talent, plus they don’t have the training budgets they once had back in the good ol’ days. IT is suffering the greatest shortage of talent it has ever had and if you go by the laws of supply and demand, the wages should be the highest they have ever been for skilled IT talent, but this is not the case. Wages in IT have stagnated for years after they dropped off following the .COM bust. The poor economy has affected almost every other field out there in a dramatic way, but IT hasn’t suffered as much because of mass attrition. This is due to several factors;

  • Overseas outsourcing created job losses and scared a lot of young students from pursuing IT careers and scared others out of the field
  • The .COM bomb years ago also scared a lot of people out of the field and from going into it
  • The initial hit of the recession hit IT pretty hard at first and weeded people out of the field
  • The Facebook/CSI generation are graduating to find out that IT isn’t all hacking and getting rich so they are not staying in the field
  • Larger companies have grabbed up all the IT people and paid them well, creating a shallow pool for smaller companies to choose from
  • People are staying put in this economy…IT professionals barely even poke their heads up to look around, as a matter of fact, most developers are ignorant to the fact that there is a shortage of their skills.

With such a shortage of talent, why haven’t the people that hire woken up and started to smell the coffee. Well, this is part of the disconnect; the same people who are trying to match skill for skill are also under the impression that IT is just like the other positions they are trying to fill and that there is a greater pool of unemployed out there to choose from. I don’t see the light at the end of the tunnel really at this point either. These problems might just persist until the economy recovers although I hope not.

How can we fix the issues?

I’ve come up with a shortlist of recommendations that might help hiring managers, human resources and recruiters be able to increase their success with placements.

  • Look at cross-cutting skills when trying to fill a position. A good example is that Java and C# have a lot in common. In most cases a Java developer can fill a C# position with minimal effort. This doesn’t always work the other way around very well unfortunately because Java’s learning curve is steeper and it’s been around longer.
  • Use a technical resource to help screen candidates. My company is very good at this since we are a custom software company as well as a staffing company. I conduct interviews and technical screenings all week. Not only do I weed out candidates who don’t fit, I can provide candidates who are near fits that a non-technical person might overlook.
  • Don’t automatically disqualify candidates without degrees, some of the best IT people I have had the pleasure of working with didn’t have a degree or had a degree in an unrelated field. For example, I know a guy who has a doctorate in physics who has always been a talented programmer.
  • Don’t be picky in this economy when choosing a candidate. Don’t compromise, but don’t be complacent in thinking that you are going to find anyone close to perfection.
  • Don’t forget that aside from you interviewing the candidate, the candidate is interviewing you. You need to show them how happy they will be and enjoy working for you as well as try to ascertain whether or not they are a good fit for the position.
  • Don’t let archaic rules like dress codes get in the way. Look at the candidate and determine, within reason, if they attempted to dress for the interview. So what if they failed to wear a tie or jacket? The average interview these days in conducted in a polo shirt and slacks and increasingly this is changing to be jeans.
  • Do ask the candidate what they have heard about your company candidly and inform them that in no way will it affect their chances of getting hired. Intel is very useful and a person who is interviewing with you most likely has spent a bit of time researching you before stepping in the door.
  • Look at the salary landscape and take that into consideration with the lack of people available.
  • Be willing to train a capable person

OK, so I colored outside the lines a bit from my original premise, but it is definitely useful to get out in the open some of these issues affecting. In the future, I hope that more IT professionals get the jobs that make them happy and fulfill their career aspirations. I also have high hopes that the folks who are hiring get a wake up call and maybe they can learn that it is ok to train qualified candidates again and stop looking for that unicorn.

An Apple a day does keep the doctor away

How medical devices sensors and mobile devices are advancing healthcare

Healthcare has been hot for mobile over the past few years, even more so than any other industry. Hospitals, medical device companies and software providers have been clamoring to have an increased presence in a booming industry. The latest trend in healthcare is integrating small sensors to be able to stream live data from a patient back to another system and drive that information back to the doctor or the family of the patient. For example, you may have a family member in home health services or in a nursing home. The sensors would wirelessly feed data such as pulse, respiration, oxygen saturation and temperature back to a small device in the patient’s room or a bed that is connected to the network. From that point the data can be viewed by the doctor via their mobile device. Aside from internal medicine, mobile devices and sensors are being used to monitor your fitness, sleep and many other aspects of overall well being.

Personal Healthcare Sensors

There are already several excellent peripheral devices in the medical field for the iPhone. Withings is a leader in the consumer space. They provide a blood pressure cuff, scale and a baby monitor that you can connect to your iPhone or iPad directly and their app allows you to keep and graph historical data. The advances in bluetooth technology and Near Field Communication (NFC), in conjunction with mobile devices, has allowed these technologies to be taken one step further.

As with any emerging technology, engineers are trying to figure out ways to make sensors and monitoring work for all modes of healthcare. In some instances, the patient themselves carry a mobile phone or medical device that transmits status of sensors attached to them back to another system via wifi or cell network. If the patient falls, the accelerometer on the device itself registers the fall and an alert is sent out. This is extremely helpful in situations where you might have a parent or loved one living alone and you need to manage aspects of their healthcare.

Among popular options for the consumer are the sleep monitoring apps that attempt to determine how you are resting at night. The Sleep Cycle Alarm Clock  for your mobile device allows you to monitor patterns in your sleep to determine if you are getting that quality rest that we all need by waking you up only when you are in a higher level of sleep instead of a normal alarm clock that might wake you up abruptly from a very deep sleep.

Institutional Settings

In another way mobile sensors are making a splash in healthcare is the field is sleep. Sleep clinics are making cash hand over fist in an age where most folks are overweight and those extra pounds are increasing the likelihood of apnea and other sleeping disorders. Although the people who actually have a sleep disorder is an order of magnitude larger than the people who actually go to a sleep clinic to get tested since it usually means at least two nights of very uncomfortable sleep with sensors attached to your legs, chest and head. Again mobile sensing devices are solving the problem. Some cutting edge sleep technicians are hooking you up at night with the sensors and those sensors are transmitting to a mobile device and then the device is transmitting the data back to a server where it is monitored by the technician in another location. So imagine, you get hooked up at the clinic and then go home to a familiar environment where you can sleep better or the sleep technician comes to your home to hook you up and then leaves. The net effect is more people are comfortable seeking testing and less is being spent on sleep clinic suites. Sleep Group Solutions is one such company that offers such technology with an app called mSleepTest.

In the 1960s, when Star Trek debuted on television, the audience was amazed at how Bones could monitor his patient just by them lying on the bed or how he could use his tricorder to instantly scan the health of a patient. Although, even as advanced a concept as this way, the doctor had to be in the vicinity of the patient. The doctors of today can do much more than he dreamed of by monitoring the patient from afar using their iPhone. AirStrip is a company that specializes in monitoring vitals and cardiological condition via mobile devices. A doctor can monitor the stats of any patient with near real time accuracy from any location they choose by reading the waveforms from a patient’s sensors. At Vanderbilt University in Tennessee, similar technology is being used to alert anesthesiologists when their patients are in certain states without having to visit the operating room to check themselves. Also, at Vanderbilt, mobile apps allow for the viewing of a patient’s room by their physicians. Stryker, a medical supply company that manufactures hospital beds, is already launching an iBed that provides information about whether or not the patient is in the bed and if they need to be moved if they cannot do so themselves.

Personal Fitness

Mobile has not only taken on healthcare but the health and fitness market as well.  A new way of heart-rate monitors supporting bluetooth technology have emerged and Polar is one such company providing one. No longer do you have to buy a watch when you can buy just the bluetooth chest strap. These heart rate monitors have more applications than tracking your workouts as well. By combining this technology with mobile, the data that was once limited to the watch or your computer now can be transferred wirelessly back to another location monitor the purposes of alerting. Imagine using this technology to monitor a loved one with a heart condition. Nike has been a forerunner in the mobile World since the early stages of the iPod by integrating it with their Nike+ product to track your fitness and workouts.

What’s next?

In just 3 short years, healthcare has taken a dramatic leap forward in terms of care, but the advances haven’t changed some of the things that have been constants for many years. You still have to make an appointment, wait for the doctor, wait for laboratory work, wait for an X-Ray and then wait for the doctor again. After that you usually have to head to the pharmacy and wait some more. In the near future, eventually this process will change due to sensors. Imagine a first- come-first-serve sensor  KIOSK where you step inside and get an analysis that is recorded to your mobile device and then take that to your doctor to speed the process or a device that you purchase that measures hundreds of points about your healthcare status by just attaching the device. Think about it like the OnBoard Diagnostics tool that your dealer uses when he services your car and the error codes it throws when it finds a problem. Healthcare wish lists aside, we are already heading into a fantastic beginning to healthcare technology.

Leadership qualities are still important in business

Sun Tzu said that if a man fears you, he’ll fight for you, but if I man respects you, he’ll die for you. The Art of War has been used for decades as a guidebook on how business leadership should function. Though many people scoff at the idea of juxtaposing battle command tactics with running a business, a lot could be learned from just the principles of the book. Sun Tzu meant for a leader to inspire troops and demoralize the enemy into retreat to minimize the losses on both sides. Is this not in essence what we seek to do in business management.

There are many types of leaders out there. I’d have to say that most are ineffective at “leading” because leadership training is something that is rarely sought after and few consider it is important as raw office politics. Despite the fall of Rome, most Roman conquerors kept their perspective as leaders until politics became too much for them to control. Behind the Roman conquerer stood a man who would whisper messages to the Cesar as he was cheer and worshipped by the masses. He would whisper, “You are only human and all glory is fleeting.” This was a tactic to keep the leader grounded in reality. Today, no one wants to hear about the poor leadership job they are doing, in fact, most are scared to lose their jobs by criticizing their leadership, even if it is in a constructive way.

I’ve had the pleasure and displeasure of serving many masters over the years. Out of the 40 or so leaders I have encountered, only 3 standout in a crowd, but what made them effective? What made their people choose to fall on their own sword if they had ever been asked to do so? Over the years, I’ve tried to wrap my head around the qualities they had that I believe are the key to success. You will be surprised at some of them.

  • A leader leads by example and is not afraid to get their hands dirty. A good business leader has been where you have stood, has moved up through the ranks, attained their position. For example, a Warrant Officer in the Marine Corps is an enlisted man who became an officer. A WO automatically gets more respect from men than a straight officer rank.
  • A leader doesn’t micromanage and trusts people to do their jobs and only intervenes when absolutely has to. A good leader surrounds himself with people who can get the job done without them. I know this seems counter-intuitive for some people to comprehend and it leads to the thinking that if your people can move the machine without you, then why are you needed. A leader’s job is to provide cohesion but not be the actual glue. A leader will choose the right people, cull the wrong people and cultivate an environment that fosters autonomy.
  • A leader doesn’t criticize, berate, belittle or otherwise demoralize his people, nor does a leader criticize his superiors in front of his people. Tom Hanks character in Saving Private Ryan said, “I don’t complain to you, complaints go up the chain of command… not down.”
  • A leader never puts himself in a position to take anything from his people. For example, the main protagonist in Band of Brothers reprimands a fellow officer for gambling with the enlisted men because he was concerned about what would happen if he won.
  • A leader knows his people. He should know their names, where they live, what their wife’s name is and their children. He should never struggle to remember anything about them and always be wary if they seem different or distracted. How can you know what is out of the ordinary if you do not know what is ordinary.
  • A leader should be decisive but also be open to advice. You should know who your subject matter experts are and lean on them when you need counsel, but when you make a decision, make it and do not appear like you have any doubt. Doubt and uncertainty are deadly adversaries to a leader and they are contagious and easily transmitted down to your people. Also, remember that pride is also your enemy
  • Anger should be used cautiously. Anger can rarely be a motivational tool and most of the time anger conveys a lack of control over yourself and the situation which leads to doubt and remember what I said about doubt. Also, anger induces fear in most people. Remember we want respect … not fear.
  • A leader doesn’t surround himself with toadies, brown-nosers and yes men. A leader wants people to ground him. Remember the guy who whispered in to the conqueror’s ear… A leader hires people who he feels are better men than he is. A leader does not hire people who are incapable to make himself feel more secure.
  • A good leader never throws his people to the wolves. Failure on the part of your people is a failure on your part always. It is a leader’s responsibility to shield, not to push them into the path of destruction to save his own neck.
  • Know when you can push your people and know when you can’t. Know when you need to reward them. Know when the need encouragement or when they feel demoralized. Know when you can help to take some of the burden they carry. Know them and know thyself.
  • Do not ask anything of your people you are not willing to do yourself. In the Marine Corps, leaders are required to participate in any physical activity that their subordinates are doing and lead by example by doing it better.
  • A leader should be strong, confident, fit, clean. Again, you lead by example. Set the example. You want your people to respect you. The people in my life I always respected were the people I couldn’t beat in a run at weightlifting or at figuring out difficult technical problems.

By no means is this an exhaustive list of leadership qualities that are keys to success, but it is a good start. Unfortunately, the people that possess these qualities rare had to work to achieve them, they just come naturally. The people who do not have these qualities are usually the ones who are arrogant enough to think they possess them already. I am fortunate that when I chose Doozer Software as a place to settle down, I recognized the qualities in the leadership of the company and that was one of the many things that influenced my decision.

I’m not advocating that you immediately go out and read Sun Tzu, but I am going to recommend that if you are in a position of leadership, see about getting a professional to come in an give a class or think about taking leadership classes yearly. I do believe that it is a worthy investment and that it will improve the bottom line of your business.

Your mobile strategy should facilitate business over advertising it

Everyone is thinking about what they want to do with their mobile strategy these days. Most of the bleeding edge strategy in the arena is coming from the advertising/marketing side of the corporate house though instead of IT. Marketing gurus consider a mobile strategy to be akin to a web site. Just another way to expand your business and get your name out there to the masses. While they are correct, the mistake comes when the comparison is made between the effort to create a static content web site as opposed to the effort to create a functioning mobile application.

Most web sites are just static HTML pages. I like to say “marketing fluff”. They serve a great purpose by legitimizing your business and showing the general public that you are more than a few guys at a kitchen table dreaming up ideas. From a programming perspective, static web content is simple. Most programmers don’t even consider it actual code. A mobile app differs from a static web site in that it should provide users with functionality instead of eye candy. For example, you may accept orders through your mobile app, do estimations or load data from a server to keep the content fresh. All of this is functionality that requires deeper programming expertise than static HTML content. For example, on iPhone and iPads, you need an Objective C programmer and in the Android world you need a Java programmer. Objective C and Java are far more complex than HTML and are true programming languages. So, while the cutting edge thinking in mobile is done in marketing, the implementation of the app fall more into the domain of IT.

Some companies have more than static web sites out there. Take Wal-Mart for example, Wal-Mart is a traditional brick and mortar institution but has a web site that allows you to shop at home. This type of functional web site might have been designed by a marketing department but the functionality was delegated to programmers at some point. The same principal applied to the Wal-Mart web site applies to their mobile app. It facilitate their business first and advertises it second. But if you download the app, you’ll notice that it simplifies the functionality that the web site offers. The main functionality is there to shop but some of the more detailed functions are not present. Another principle of mobile development is to keep it extremely simple. Do not try to put all of your bells and whistles from the web site into the mobile app.

Usability and design are extremely important in a functional web site and they are equally important in your mobile app. It’s probably a good thing if your marketing department is leading the charge for your mobile app. This means that usually graphic designers and usability experts are working on the design and intend to hand it off to programmers for implementation. This strategy works very well and usually results in a fantastic mobile experience for the end user.

Now that I have given you a little background on the differences between a mobile app and a static web site, I would be remiss not to mention cost. Cost is the biggest issue I see when someone starts talking a mobile app. In their mind, they equate the mobile app back to marketing expenses and the cost of their “static” web site. “I paid $10,000 for my web site, so I was going to budget 75% of that for my mobile app” is something I hear quite often. Now, let’s look back at what I have said before. The static content web site took one designer to implement doing just straight HTML. Now you need a designer plus a programmer and let’s face it, designers are no less talented than programmers but programmers are more expensive, usually by another 50-100% more depending on the experience level. So based on this, you can logically deduce that your expense for your mobile app should be around 150% more than it cost to do your web site. So if you spent $10,000 for your web site, I would off the cuff estimate that your mobile app is going to be in the ballpark of $20-25,000. Now keep in mind that there are a lot of variables at play here and take my estimate with a grain of salt. I’m not saying you can’t do a mobile app for 75% of your web site costs but I am going to say with 100% accuracy that if you go that route, you won’t be happy with what you get from a functionality perspective.

I want to throw out another thing I see people rationalizing in their heads. It’s a smartphone, therefore it’s smaller, so it must be cheaper to develop for and take less code. It takes the same amount of code to do a task on the server as it does doing the same task on a smartphone 99.9% of the time. Another mistake I see people make with relation to cost is forgetting the cost of the backend support. You want your mobile app to update from the server to get fresh content and you also want it to store that data and work if the user is without access to an internet connection. This adds great amounts of complexity and multiplies the cost of the project. For one, it takes a greater level of programming expertise to integrate with services and cache/store data for offline use than just simply creating a self contained mobile app. This expertise comes at an increased cost in terms of the resource and the time of the project.

All, this being said, I want to just put a few tips out there to help you evaluate your mobile strategy so that you can make an informed decision when you get to that point.

  • Don’t do a mobile app for the sake of doing a mobile app
  • Make sure that you can’t just get by with a mobile web site strategy instead native mobile functionality
  • Remember the cost will be greater than you expect always
  • Remember that you need a programmer and a designer at least
  • Keep the functionality simpler than your web site
  • Try to plan to feed the app from the same data sources as your web site
  • Do not use an advertising firm to write your mobile app. This is the greatest advice I can give. By all means, contract them to design it and create mockups, but have it reviewed and implemented by an experienced mobile developer or architect. Most advertising firms outsource the implementation to a contractor anyway.
  • Do not exclude your IT organization from your strategy. IT and marketing should collaborate on mobile strategy

 

 

 

 

Follow

Get every new post delivered to your Inbox.