Investing in the Python Language

python-logoOver the years, I’ve had to learn many programming languages in the course of my career; Perl, JavaScript, shell, Java, C#, Objective C to name the most prominent. Several years ago, I wrote a little Python for a small project and I hated the language for some of the syntactical eccentricities that makes it unique from other languages out there. Lately, I had to pick up Python again, just to be pleasantly surprised that I can actually enjoy it now.

I am notorious about code formatting, in fact, it is the worst thing about having to read my code if you are another developer with a little OCD about formatting, indention and spacing. The python language doesn’t have the brace constructs most other languages use to delineate blocks flow blocks in code, it uses indention. For example,  if you want a series of operations to occur following the execution of an if block, you would merely nest them the same indention level under the if itself. This is a fundamental concept in Python but I have found that it actually has helped my formatting as I write other languages and I no longer torture other developers as they read through my code.

Python is pretty versatile. It allows for access to a very robust API without a lot of overhead and a very shallow learning curve. I would argue that it is the easiest of all the languages I write on a normal basis by far. I recommend Python for any novice programmer. I wouldn’t write enterprise level applications with Python, but I would use it as a support language no matter what platform I am running on. short scripting tasks of tasks or routine maintenance tasks that are a too complicated for more rudimentary scripting languages are where Python rules. In fact Python is an integral part of every linux distribution.

Overall, spending a little time in Python isn’t a waste of time at all and I highly recommend it.

Home automation is booming

I love being right. I predicted that Nest would prove to be a technology that would be of considerable value to consumers and to other companies as well. The $3.2 billion acquisition of the company by Google pretty much says it all. I’m betting Honeywell, and the other companies out there who lagged behind several years in advancing their line of products, is really kicking themselves now. I was an early adopter of Nest and I love the device. I loved it so much, I proceeded to automate other parts of my home. At present, I also have Chamberlain MyQ light switches and garage door openers, Belkin Wemo plugs and a couple of switches, Foscam security cameras and other odds and ends that make up a pretty advanced smart home. I like to tinker and my next project at home is to tie a Raspberry Pi into my home automation center to control and monitor my security system.

It saves you money

The fiscal conservatives and the green energy folks love these connected devices. For example, my Nest has already paid for itself in the few years I have had it just by it’s “smart” reactions to my habits and the way I like my house to be heated and cooled. I no longer leave my outdoor lights on all night to deter prowlers since the motion sensors and automatic switches take care of it all. I even replaced every incandescent bulb in my house with energy efficient LEDs. It was an enormous up front cost, but if a light is left on, it only uses a fraction of the power of the older bulbs. I thought I’d just compile some of the advantages of a “smart home” that I have experienced. These are by no means all inclusive and the extent of how you automate your home is all a matter of your imagination.

  • Chamberlain MyQ notifies me if I leave my garage door opens or it stays open too long. I can close it remotely if I drive away and forget
  • Chamberlain MyQ switches allow me to turn on all my outdoor lights at the touch of a button. I do this if I hear dogs barking at night
  • Foscam Cameras monitor my entrance ways and send me notifications of movement
  • Belkin Wemo devices monitor energy usage of my lights and appliances and allow me to control them and automate their operation
  • HDMI-CEC (built into most TVs now) allows external control of devices like a receiver so I don’t need multiple remote and I don’t need to turn off the power to devices when I turn off the TV. It does it for me.
  • Chromecast, Samsung and Apple TV allow me to broadcast video and audio from any device to another
  • Green initiatives: Solar film on windows, enhanced attic insulation, and other energy saving modifications

Other areas of automation include wifi/bluetooth enabled door locks, the new Nest Protect smoke and carbon monoxide alarms and much more. All of these are available on the Internet but I will warn you that most of these devices are not cheap. The cost of the Nest Protect alone has been prohibiting me from diving into that technology.

All my excitement and gushing aside, I think this is the year for a home automation boom. The acquisition of Nest signals to other companies that the race is on. I’m pretty excited about what is headed our way this year myself and I look forward to my own projects as well.

What is responsive web design and how do I get there.

mashable-responsive-designA few short years ago, I could have almost guaranteed that you would be reading this on a Windows PC with Internet Explorer (IE) and a few short years after that, I could reasonably conclude you would be reading this on the aforementioned PC with IE, Firefox or Chrome but you could also be on a Mac with Safari, Firefox or Chrome. If we look back to two years ago, I could add that you would most likely be reading this on an iOS device such as an iPhone or iPad with Safari. Today, I would have to add the plethora or Android phones and tablets to that cacophony of mediums. All that being said, I can no longer reasonably guarantee with a degree of certainty how you will be reading this content. This is where a “responsive user interface” comes to the rescue.

The rise of the smartphone and tablet has made the old debate of simply browser and platform responsiveness a purely academic one. Today, we must add to our consideration screen size, device type, browser, mobile, etc. Does anyone remember the old days of trying to write code to determine a user’s browser in Javascript and simply tailoring the response? Doing that very thing today is more of an art instead of a science.

More and more, the phrase “responsive user interface (UI)” or “responsive web design” are thrown around without much consideration to the preponderance of effort that is required to make it happen. Being “responsive” is about detecting the screen size, browser, platform, and device type of the user and providing an aesthetically pleasing and functional interface to said user. In essence, as the user’s resolution changes, so does the interface to accommodate that resolution.

“It’s not a single piece of technology, but rather, a set of techniques and ideas that form a whole.” as stated by a fellow blogger.

Just to get you started as a primer, let’s look at some terminology you’ll hear with regards to responsive design.  Keep in mind that none of this jargon is new to designers as these concepts have been around for many years.

  • Fixed layout – a Web Design that is defined by exact pixel widths. Typically the banner image or header provide the maximum inner width for collapse
  • Fluid layout – Nothing new here, but a fluid design is merely providing a percentage width instead of a fixed width, for example, your header may be defined as 100% of the browser width.
  • Elastic – Adds the best of both fixed and fluid but adapts to the users preference in font size. In this design font are expressed using em or emphasis.
  • Adaptive – Adaptive is merely responding to the users device with a template based on their device type or screen resolution with a canned template. This differs from pure responsive in that typically it involves a redirect.

Now that we have the jargon out of the way, let start talking about being responsive. I’d never get into a discussion about how to paint a car without knowing what kind of car we are painting. We wouldn’t paint a Ford Pinto hot rod red would we? It isn’t just a matter of providing a UI layer with chameleon like properties, but it also means having an architecture to match.

Responsive Design design has given birth to RESS or Responsive Design with Server Side components. Some responsive design purists will claim that being responsive is merely about responding to the screen resolution. This is all fine and dandy so long as the user is going to sit there on their desktop browser and resize it to ohh and ahh at your responsive design. (Come on… you know you have done it.) RESS is merely leveraging server side detection to target the client more specifically, e.g. a template for mobile iPhone users or a specific stylesheet for an Android tablet. The net effect of RESS is that you can provide a better experience with a smaller footprint. This is because less code has to be downloaded for the UI to be responsive. In a pure responsive UI, the images for multiple resolutions are often downloaded all at the same time. This technique wreaks havoc on users with limited bandwidth or a limited data plan. If you load up your favorite responsive site on your mobile phone via wifi, you’ll no doubt notice that it looks really nice. Now turn off your wifi and attempt to load it if you have 3G service and you’ll notice it is probably slow. It is trying to unnecessarily load all those resources to respond to all UI possibilities, including a desktop. The responsive solution here is to conditionally load elements and carefully tailor your design.

The easiest way to get your feet wet with responsive design is to start with a simple web site built with a responsive technology. For example, bootstrap is a good start for a responsive UI layer.


Since I cannot guarantee how you will be read this, I can with a degree of certainty say that you are most likely on a mobile device of some kind. The latest statistics show that people who read news and blogs prefer to do so over the mobile web and not through a native app. From my own experience and research, I have seen this to be true. Although, I will add that most users do prefer a native mobile experience when they are doing tasks that require more interaction, e.g. shopping apps, productivity, music. The reasons for these are fairly simple in that they are local to the device and respond faster and are more intuitive that using a browser. The key here really lies in offline ability of an app. Think about it this way … when you request a page over the mobile web it may error out and not load, but if you request something in a native app, I can attempt the connection in the background and catch the error and retry over without you knowing it and also, I could give you cached data that I stored on the device if the connection is down. Which this isn’t entirely out of the question with HTML5 since you can use local storage. The key concept here is allowing you to work regardless of your moving around, traveling or being completely out of service. All that being said, the mobile web is the ideal way to view static content like news, social media, etc.

Being responsive is about blending technologies if you are considering a “roll your own” approach, e.g. HTML5, Javascript, CSS Media Queries, fluid grids, flexible images and JSON. It is not a simple task by any stretch of the imagination. Alternatively, you can choose to utilize a product or reach out to a partner to assist in your responsive needs. Keep in mind that just because you are responsive to screen resolution, doesn’t mean that you don’t have to consider certain nuances of the device or browser you are running on and respond accordingly. This is why most responsive sites have special stylesheets for Android, iPhone, etc.

Here are some of the top framework projects that provide out of the box responsive design.

Overall, I’d say to follow these rules when attempting a responsive UI.

  • If your site is content-based, consumer facing or fairly static, a responsive UI is definitely the way to go
  • If your site is more an application with lots of user interactions, you may be better off choosing to build something more adaptive than responsive.
  • Choose a responsive UI framework like the ones mentioned above… Don’t attempt the role your own approach

This should give the rest of you folks out there a better understanding of responsive design. The key to being effective and getting the most bang for the buck with regards to responsive design is simply in being there when your users need you to be.

15 recommendations to make you a better programmer

I often give the same advice over and over to programmers that I work with. Most of the time I give advice on things to do with regards to coding and that works out pretty well, but “don’ts” are just as important as the dos. Let’s look at the 15 things a programmer should watch for. Most of these are geared toward object-oriented languages, but some can apply to procedural,functional languages as well.

  1. Don’t forget the tenets of object-oriented programming, Inheritance, Encapsulation and Polymorphism.
  2. Don’t overuse inheritance. Inheritance should make sense. Ask yourself about the relationship. If “is a”, then you should inherit, if “has a” then it should be a relationship off the owner.
  3. Watch your method/function/subroutine lengths. If a method or function gets longer than 5-10 lines, you are probably missing an opportunity to abstract or extract functionality. The longer a method is, the more complex it will become exponentially.
  4. Before you start to “roll your own”, spend some time looking for open source solutions or blog articles where someone solved this problem before. There is nothing wrong with leveraging someone else’s hard work. Chances are someone else will take over your work at some point and it is easier for them if it is a solution that they can find support for by a Google search. Also, think about the testing, maintenance time with regards to rolling your own solution. Aside from that the chances that one person can produce a solution better than a community project is unlikely no matter how good you think you are.
  5. Don’t hack. There is a lot to be said for code written during a time crunch, but more often than not, programmers will use this excuse to shortcut a solution and not take time to do it the way they know it should be done. You want the next programmer to look at your code and pat you on the back for your solution, not curse you in disgust.
  6. Don’t forget about reusability. Think about every line of code you write. Ask yourself if what you are doing is going to be repetitive by you or someone else. If it is, then abstract it to a utility class and reuse it. Never just copy code from one place to another when you could extract it to a utility or utilize polymorphism to address the need.
  7. Don’t use obscure variable names. It should be very clear what data a variable contains when another person looks at your code.
  8. Don’t forget to ask for a code review or design review. No one is perfect. You should always walk through your code with a peer with both of you sitting side by side. Explain your rational, what techniques you used and ask the reviewer for recommendations. Two heads are better than one. Also, you should do this early and often. Don’t wait until you finish a project to ask for a review because by then, it may be too late to fix.
  9. Don’t use global or member variables when a local one would suffice. I’ve seen this a few times before. A junior programmer will scope their variables as wide as possible. This not only causes confusion when others look at the code, but it can cause unintended consequences in your applications.
  10. Don’t forget about threading and thread-safety. Threading is a difficult concept for unseasoned programmers. It can bite you really quick if you don’t think about it. Complex applications may have many threads accessing the same resources and if you’re not concentrating on managing this, then you can get junk data, crashes and unexpected results. And DO NOT synchronize everything as a solution to thread safety, else performance will suffer.
  11. Don’t code first and ask questions later. You should understand the problem domain and the goals you want to accomplish before you even write one line of code. Ideally, you will design the application and run through your mental sanity checks in your head well ahead of actually putting code onto a screen.
  12. Don’t forget about unit testing. Unless you just enjoy spending hours and hours testing your application or sitting with your QA resource, you should unit test your code at the lowest levels and run these tests as regression tests along with your builds. Remember that a unit test is for very small bits of code. A unit test does not take the place of actual functional testing of your application, but it sure makes it easier.
  13. Don’t forget to comment and don’t over comment. If you want to provide yourself a hint, reminder or give that hint to another programmer, use a comment to make the point. Don’t over comment your code either as too many comments are an indication of complexity and you need to revisit your code for simplification or refactoring.
  14. Don’t forget to refactor your code as you go. If you see areas of your code you need to revise, do so as early an opportunity as possible. If you wait, the problem can be compounded by other code that utilizes it. Never wait until the end of a project to refactor, you will never get the chance and by then, it is a daunting task.
  15. Don’t forget to layer and loosely couple. Do not forget to keep your code as loosely coupled as possible. A good strategy is to layer your code, e.g. DAO layer, service layer, integration layer, Controllers, UI layer, etc. For example, a UI Layer should never access classes directly from the DAO Layer, but should utilize the controllers to access data, which in turn access service layer and so on.

While this is not an all inclusive list, it does give a programmer a great advantage. Being a programmer is definitely about working smarter and not harder.




Integrating Spring 3.1 and Lucene 4

Integrating Spring 3.1 and Lucene 4 is a fairly trivial matter, but I didn’t want to use the XML configuration so utilizing the @Configuration annotation, I was able to configure the Lucene indexer, analyzer and queryParser.  Here is the configuration code:



public class AppConfig {

    private @Value("#{appProperties['index.location']}") String indexLocation;
    private @Value("#{appProperties['index.source']}") String indexSource;
    public Analyzer getAnalyzer() {

        return new StandardAnalyzer(Version.LUCENE_40);

    public FSDirectory getFSDirectory() throws IOException {

         File location = new File(indexLocation);

        if (!location.exists() || !location.canRead()) {
              System.out.println("Creating directory: '" +location.getAbsolutePath()+ "'");

        return, new NativeFSLockFactory() );

    public IndexWriter getIndexWriter() throws IOException {
        IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_40, getAnalyzer());
         // Optional: for better indexing performance, if you
           // are indexing many documents, increase the RAM
             // buffer.  But if you do this, increase the max heap
           // size to the JVM (eg add -Xmx512m or -Xmx1g):
               // iwc.setRAMBufferSizeMB(256.0);

        // NOTE: if you want to maximize search performance,
        // you can optionally call forceMerge here.  This can be
          // a terribly costly operation, so generally it's only
        // worth it when your index is relatively static (ie
           // you're done adding documents to it):
          // writer.forceMerge(1);

        IndexWriter writer = null;
     try {
        writer = new IndexWriter(getFSDirectory(), iwc);
        indexDocs(writer,  new File(indexSource));

     } catch (Throwable t) {

         System.out.println("Unable to create IndexWriter!: " + t.getMessage());


        return writer;

    public IndexSearcher getIndexSearcher() throws IOException {

        return new IndexSearcher(;

    public StandardQueryParser getQueryParser() throws IOException {

        return new StandardQueryParser(getAnalyzer());

    static void indexDocs(IndexWriter writer, File file)  throws IOException {
      // do not try to index files that cannot be read
       if (file.canRead()) {
          if (file.isDirectory()) {
            String[] files = file.list();
            // an IO error could occur
             if (files != null) {
              for (int i = 0; i < files.length; i++) {
                indexDocs(writer, new File(file, files[i]));
          } else {

            FileInputStream fis;
           try {
              fis = new FileInputStream(file);
            } catch (FileNotFoundException fnfe) {
              // at least on windows, some temporary files raise this exception with an "access denied" message
              // checking if the file can be read doesn't help

           try {

              // make a new, empty document
              Document doc = new Document();

              // Add the path of the file as a field named "path".  Use a
              // field that is indexed (i.e. searchable), but don't tokenize
              // the field into separate words and don't index term frequency
              // or positional information:
              Field pathField = new StringField("path", file.getPath(), Field.Store.YES);

              // Add the last modified date of the file a field named "modified".
              // Use a LongField that is indexed (i.e. efficiently filterable with
              // NumericRangeFilter).  This indexes to milli-second resolution, which
              // is often too fine.  You could instead create a number based on
              // year/month/day/hour/minutes/seconds, down the resolution you require.
              // For example the long value 2011021714 would mean
              // February 17, 2011, 2-3 PM.
              doc.add(new LongField("modified", file.lastModified(), Field.Store.NO));

              // Add the contents of the file to a field named "contents".  Specify a Reader,
              // so that the text of the file is tokenized and indexed, but not stored.
              // Note that FileReader expects the file to be in UTF-8 encoding.
              // If that's not the case searching for special characters will fail.
              doc.add(new TextField("contents", new BufferedReader(new InputStreamReader(fis, "UTF-8"))));

              if (writer.getConfig().getOpenMode() == OpenMode.CREATE) {
                // New index, so we just add the document (no old document can be there):
                System.out.println("adding " + file);
              } else {
                // Existing index (an old copy of this document may have been indexed) so
                // we use updateDocument instead to replace the old one matching the exact
                // path, if present:
                System.out.println("updating " + file);
                writer.updateDocument(new Term("path", file.getPath()), doc);

            } finally {


public class App

    public MongoOperations mongoOperation;
    public StorageService storageService;

    ApplicationContext ctx;

    public App() {

      ctx =  new AnnotationConfigApplicationContext(AppConfig.class);

Let’s get fancy with @Configuration with Spring

Spring has changed a lot over the years to make things more flexible and convenient for developers. Annotations in Spring 3 really hit home, but recently, Spring has added features that almost completely eliminate the need to XML all together. In the past, you still needed an XML configuration file if you wanted to utilize third-party code as Spring beans but you could use annotations to demarcate your own code. With the latest Spring code, you can use a class for your configuration. Let’s see how it works.

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.ImportResource;
import org.springframework.context.annotation.PropertySource;

public class AppConfig {

    private @Value("#{appProperties['index.location']}") String indexLocation;

    public String getIndexlocation() {

         return indexLocation;



//App.class main
ApplicationContext ctx =  new AnnotationConfigApplicationContext(AppConfig.class);

There is a lot going on here, but it may not be apparent by the small amount of code we have written. This code does the following:

  • Maps a class as the configuration for Spring
  • Loads an XML Property file (There are still some things I prefer to do in the XML)
  • Creates a String bean of type String and returns the definition of a property found in the Property file

While the property example is not necessarily useful in this example, you can see the flexibility of the properties using Spring expressions to access them. The first question you might ask is why am I still loading an XML file since the @Configuration annotation eliminates the need for it. If you declare a Bean in the class, you need to inject properties into it in most cases so this is a little extra work and on top of that you are writing some code that needs to be maintained. Using the XML declaration, you can use property substitution as parameters to an existing class and no code needs to be placed your configuration class.

So how do you determine when to put class in the XML and when to declare it as a bean? Here are my general rules:

  • If you create a class, the demarcate it with a Spring stereotype (@Component, @Service, @Repository, @Controller, @Configurable, etc.)
  • If the class is a class from a third-party jar, then place the configuration in the XML
  • If the class is from a third party but you want finer grain control over the events of instantiation and circumstances, then create the Bean using the @Bean annotation in the class containing the @Configuration annotation

Pretty simple rules to follow…

There are several other annotations that can be used in the class containing the configuration as well such as @Depends-On and @Value.

Java,.NET Caching with Spring

If you want a major performance boost out of your application, then caching should be a part of your strategy. No doubt you have experienced moments in coding where you needed to store sets of data so that you don’t have to go back to the source to get them every time. The simplest form of caching is lazy loading where you actually create the objects the first time in memory and from there on out, you access them from memory. In reality, caching gets a lot more difficult and has many considerations.

  • How do I cache in a distributed environment
  • How do I expire items in the cache
  • How do I prevent my cache from overrunning memory
  • How do I make my cache thread-safe and segment it

All of these are concerns that you will have if you “roll your own” solution to caching. Let’s just leave the heavy lifting to the Spring Framework and we can go back to concerning ourselves with solving the complex problems of our domain.

Spring has a caching mechanism/abstraction for both Java and .NET, although the Java version is far more robust. Caching in Spring is accomplished through AOP or Aspect Oriented Programming. A caching annotation (Java) or attribute (.NET) can be placed on a method or a class to indicate that it should be cached, which cache should be used and how long to keep the resources before eviction.

Java Spring Cache with EHCache

In Java, caching with Spring couldn’t be easier. Spring supports several different caching implementations but EHCache is the default and by far my favorite. EHCache is robust, configurable and handles a distributed environment with ease. Let’s look at how we can add the ability to cache to a Spring project.

<beans xmlns="" xmlns:xsi=""
  <cache:annotation-driven />

<!-- Ehcache library setup -->
<bean id="ehcache" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean" p:config-location="ehcache.xml"/>

Now that we have our cache setup, we can start to utilize it.

@Cacheable(name="records", key="recordList") //Cache the output of records, return it if already cached
public Collection findRecords(RecordId recordId) {...}

@Cacheable(name="records", key="recordsList", condition="recordType == 2") //Only cache if record type is 2
public Collection findRecords(int recordType) {...}

@CacheEvict(value = "records", allEntries=true) //Reload the cache, evict all entries
public void loadAllRecords() {...}

In the above example, we specify through the annotation that the records collection will be stored in a cache named “records” and the key to access the collection will be called “recordList”. The key parameter is optional. We also displayed an example of using Spring Expression language to process the cache conditionally. Remember that caches are defined either dynamically like above or in the ehcache.xml. For most complex caching scenarios, you will want to define the cache in the ehcache.xml with eviction and distribution rules and Spring will find it by the name parameter in the annotation.

What about .NET?

In .NET, you have a very similar mechanism to managing a cache.

<!-- Apply aspects to DAOs -->

[CacheResult("MyCache", "'Record.RecordId=' + #id", TimeToLive = "0:1:0")]
public Collection GetRecord(long RecordId)
   // implementation not shown...

Remember that with the .NET cache, you provide the caching implementation in the XML just as you do in the Java version. In this example, we have used the provided AspNetCache.

What about controlling the cache yourself, querying the cache and more complex operations? Well even that is simple by merely autowiring the cache class or retrieving it from the context.

EhcacheCacheManager cacheManager;

EHCache cache = cacheManager.getCache("records");
Collection records = cache.get("recordList");

Built-in .NET Caching

Fortunately for .NET users, there is also a built in caching framework already in .NET. This technique is used mostly in the MVC and ASP world and I am not particularly fond of it since it is specifically geared for the web side of an application. I would prefer it to be more generic like the Spring solution, but it also has some advantages such as you can configure the caches in the web.config, you can also create custom caching implementations that utilize different mechanisms for cache management. Here is a great example utilizing MongoDB. The .NET version of the cache works much the same way as the Spring one does. Here are some examples with configuration.

[OutputCache(CacheProfile="RecordList", Duration=20, VaryByParam="recordType")]
Public ActionResult GetRecordList(string recordType) {


Now the configuration in the web.config…


          <add name="RecordList"  duration="3600" />

If we are deploying our application to the cloud in Azure, we can use the AppFabric cache as demonstrated here.

Hibernate, Data Nucleus, JPA Result Caching

Another thing to keep in mind is that when you are using tools such as Hibernate, caching is built in with these solutions. There is a built-in second level cache implementation that is provided when you configure Hibernate and I tend to use EHCache for this as well. You must remember to add the Hibernate Cache annotation onto your objects at the class level that you want to cache as well. A properly setup ORM solution with Spring and Hibernate with a properly configured second-level cache is very hard to beat in performance and maintainability.


We have really done a lot with very little thanks to Spring and caching. While caching is powerful and will help improve the performance of your application, if it is overdone, it can cause problems that are difficult to diagnose. If you are caching, make sure you always understand the implications of the data and what will happen throughout the caches lifecycle.


Get every new post delivered to your Inbox.