Programming is like Writing

Everyone heard that programming is very similar to writing books. “It has many similar traits” programmers think. There is even a thing called “Literate programming” invented by famous Donald Knuth.

But, are there really much similarities between those two activities? Well, except that both require starring at screen and typing a lot of text.

One of my goals for 2017 is to improve at writing texts. I found that there is an increasing need for me to write good texts both at work and for personal needs (like this blog). At the same time, writing is not something that goes for me as easily as coding. To achieve my goal, I’ve started with picking and reading a few books which give advices on how to write texts correctly.

One of such books is “On Writing” by Stephen King. My friend was reading some fiction book written by Stephen King, so I’ve also decided to check what are some interesting books by this author. I must say, that the only book I found the most interesting for me at that moment was “On Writing”. I could have done 2 things in one: finally read some Stephen King book and make a step towards my yearly goal.

I must confess that book was an easy read. It was like a fictional book, but wasn’t. A great example of helpful book that is easy to read. This is due to 2 important parts:

1. Author gave background stories from his live. First 1/3 of the book is more like a biography of Stephen before he became a famous author. Maybe, it’s because I love to read biographies of famous people, but I’ve finished that part on a single breath.
2. Every advice author adds also comes with some background story and intuitive explanation. You not only learn good advices, but have a background that helps you remember it once you’ve closed the book.

Among the advices author gave, I have highlighted a few that were, as I felt, most important to me. I’ll share them with you a few moments later. I use some of the advices every time I write a text, like simplification, drafting, and 2nd draft rule to some degree as well.

Somewhere in the middle of the book I had an enlightenment that the writing’s general best practices from Stephen King are very similar to those we follow in programming.

Simplify and remove the clutter

Although some junior engineers love to create overly complicated (they say beautiful, flexible and extendable) designs, over-engineering in architecture and code, experienced software engineers know that simplicity is the only true thing we all must aim.

Software engineers should focus on keeping things simple, both in code and in design. And that’s what author of the book also recommends. Remove unimportant parts that don’t add up to the story, remove overly complicated and unnecessary descriptions, leave as low as needed for the reader to feel the story. Otherwise, 90% of readers would just give up on this boring book.

Not everyone is ready to read 10 pages about the colors of sunset of 20 pages about the architecture of the city. Same in code: not everyone ready to get through 5 page methods, and not everyone is ready to dig deep into the layers and layers of your code.

Avoid passive verbs

The analogy here is very simple:

1. use verbs for functions/methods names,
2. put methods into the objects that they related to.

It is not the best sentence “Pizza was delivered to my doors.” Once stop using passive verbs, you get “Courier delivered pizza to my doors.” Way better!

Same in the code: not “pizza.deliveredTo(myDoors)” but “courier.deliverPizza(myDoors)”. That makes modeling object-oriented relationships easier.

Practice Continuously

To become a good author, one should practice continuously, and write something almost every day. For example, Stephen King writes every day. He starts in the morning, and then writes until he reaches his goal.

Same is with coding. You just can’t become a great programmer, if you don’t practice.

---

Story from my life

Many years ago, I was a kid who wanted to learn programming. The only issue, I didn’t have a computer. But I didn’t get upset that easy, I’ve bought a book about programing on Turbo Pascal 7. That was a book with cover of green and white colors, published by Sankt-Petersburg publisher and covered TP7 from basics to writing code that draws 3D objects and generates audio. I loved that book. It was my first book on programming.

I spent a couple months going through this book: learnt about data types, arrays, pointers, files and many other things. I followed author recommendations and wrote many programs. As I mentioned, I didn’t have a computer yet, so all my programs were on pieces of papers.

But one day, my parents bought a computer. I started porting my programs from paper to Borland Turbo Pascal 7 environment. Only to find that, lo and behold, none of them worked. They even didn’t compile. Boy, was I upset!

I had to spend a few more days to fix some of the programs and got them working. Since then, there were not many periods of my live where I didn’t code for a long time. Because, I quickly realized that practice is the most important part.

For years after that, I still thought the problem was with me: I’m just a “practice” person, not a “theory” man. I learn better and faster from practicing not from reading books. That was a wrong-thinking. It’s not my problem. We all are like that!

---

If you want to be good at something, practice at it continuously.

Have a place for writing

Authors should have a place where they can hide from everyone and focus on the book. Place, where nobody and nothing would distract you, where things motivates you to do your work the best.

But, to be honest, not only writers need such place. Artists also need a place where they could focus on paining. Designers need such a place too. Software engineers also want to have it in their life; and, ideally, this place shouldn’t be in the open office space.

Sometimes, I imagine my ideal place for programming. It is a large room, with high ceiling, 2 walls are covered completely by bookshelves, another wall has a large glass whiteboard on it. Room has a large desk with large displays on top, and comfortable super expensive chair next to it. There is also a comfy wingback chair for reading with a standing lamp close to it. Right, my ideal “office” is both programming office and small library.

Have a toolbox

Simple here, authors have dictionaries and vocabularies, favorite software to write a book or type of paper and pen they can’t live without.

Programmers have their IDEs, programming languages, version control tools, programs for reading documentation and many many more.

Always do 1 or 2 drafts before final version

Review results of your own work, no matter if it’s code, design or documentation. See if you could improve it, whether it is bug-free and covered with tests. And, once you are fine with the version you have, hand it over for a peer review.

Similar with books: author writes first draft, reviews and modifies it a few times, and once author is ready, book is passed to editor.

Write about something you like

I don’t think any reader would be happy to read what author wrote about a topic author doesn’t like. Author would either make it extremely boring or obviously incorrect. Reader would inevitably feel discomfort reading such work.

Code, that programmer hated to write, would look like… code that written in hatred.

If you want to be successful at what you do, you need to love it. Either it is writing, programming, paining, designing, crafting or counting money.

Read Continuously

Stephen King reads a lot of books. He loves that. And he recommends it to other wanna-be authors as well. Read many different books, see what works good and what doesn’t, learn elements of style from others, and improve your own.

But that advice is exactly same as all programmers receive from their mentors all around the world: read code written by others, study design approaches created by other more experienced software engineers etc.

Summary

Writing good is hard. Coding good is hard.

But there are a few best practices we all can use. There is no magic behind them. They are universal as work for writing, coding, and almost anything else.

These best practices are:

• keep simple and remove the clutter,
• practice continuously,
• learn from more experienced peers,
• make your work be more comfortable for you to do,
• love what you do.

Simple Multiple BloomFilters data structure implementation on Java

As a follow up on my previous post about using Multiple Bloom Filters data structure to identify hot values, I decided to write a simple dumb implementation on Java. And open source it.

The project can be found on GitHub and is proudly named multi-bloom-filter.

It comes with basic little class called MultiBloomFilter which does most of the job. It accepts the number of enclosed bloom filters, capacity of each BF and duration before the the head BF will be reset. One can also specify what hash function is used and how many times hash function should be applied for each value.

Simple example:

This short example shows that MBF will reset only one of the internal BFs. Means, whenever reset happens, it will remove only part of the data, and whenever the hot key is added again, it would be identified as such.

Once again, MBF is a great solution if you need to find a set of hot values for some period of time. In particular, this helps to put only hot values into the cache. If we have many hosts that use a single distributed cache service, then using MBF might save from redundant traffic of putting cold data into the cache, where they would be evicted pretty fast. Also, as hot keys are in MBF, means there is a high chance they are in the distributed cache as well. Thus application has some kind of "bloom filter" to check what is the chance that value could be found in the cache for specified key.

There are much more use cases for the MBF data structure. Being able to work in concurrent scalable environment is another "feature" that I love about BloomFilters and MultiBloomFilter in particular. For me, good implementation of BloomFilter, that is able to grow and scale correctly, has different mechanism to evict data and fight the false positives, sounds as a very useful service.

Twister: A Simple Way to Manage Your Scripts

Imagine an average project that has many scripts, each written using different practices, uses different argument names, different namings, does something similar to other script but a bit different etc. Sometimes there are so many scripts, that it's hard to find the one you really need at this very moment. Moreover, scripts have no standard location, often put in semi-random directories, so it's really hard to find them. And even more, many developers have similar scripts for different projects. Some scripts are in the PATH, others are relative to the project directory. The one in the PATH are named in odd manner, because different versions used for different projects. And some scripts are written using bash and ruby and python etc.

Oh, so many troubles just because of the scripts. That's the reason why Twister was created. Twister is a simple framework/tool that allows to create and manage project scripts. I use it in a few of my home projects, and find it very helpful. About a year ago, I open sourced Twister. It's a python project, so it makes it simple to create scripts and execute them on any popular operating system.

Memory v0.1 is released

A few weeks ago I decided to create a simple library that emulates a virtual memory but with some extra features. My primary interest is to play with memory (de)allocation algorithms, replication algorithms, improve my knowledge in concurrency programming etc.

As for now, I have implemented a virtual memory with allocation mechanism, added concurrency and a basic transaction support.

Today I'm releasing a first version of this library, ie v0.1. This library can't do much yet, and if I'll have some time, I will start working on next version soon.

Library is open sourced, I'm hosting it on GitHub https://github.com/rkhmelyuk/memory. Documentation can be found in project's Wiki.

First memory should b allocated. There a few different allocators for this:

• FixedMemoryAllocator - to allocate a memory of fixed size
• DynamicMemoryAllocator - to allocate a memory of dynamic size (with initial and max size specified)
• FileMemoryAllocator - to allocate a memory mapped to file, supports both fixed and dynamic allocation

After memory is allocated, it can be used in application. The common workflow is to allocate spaces in this memory. Space represents a block of memory. This block will be not available for allocation till a space is freed. A space is like a window to specified block of memory: it can be used to read and write data at this block.

Spaces are thread-safe. This means that locking is used when read/write operation happens. For a single write operation there is no change other thread will read a corrupted data. For a multiple sequence read and write operation it's possible to use a transactional space currently. In the next version I plan to add a lock/unlock operation, so single client can lock and use space without a chance a space to be changed by other thread.

A simple example for the end of this post:

// allocate a memory with a fixed size 20KB
FixedMemoryAllocator allocator = new FixedMemoryAllocator();
Memory memory = allocator.allocate(20 * Memory.MB);

// allocate a space in the memory with a size 10KB
Space space = memory.allocate(10 * Memory.KB);

// write a string to space memory
space.write("Hi world");

// create a transactional space, and start a transaction
TransactionalSpace transactional = space.transactional();
transactional.start();
try {
    transactional.write("Hello");
    transactional.write(transactional.readString() + " world!");

    // in transaction - original space is not changed
    assert space.readString().equals("Hi World");

    // but transactional does
    assert transactional.readString().equals("Hello world!");

    // commit transaction
    transactional.commit();
}
catch (Exception e) {
    // rollback transaction
    transactional.rollback();
}

// check transaction was committed correctly
assert space.readString().equals("Hello world!");

// it's safe to use in multiple threads
executor1.processData(space.getInputStream());
executor2.processData(space.getInputStream());

Mobile website with jQuery Mobile

In my previous post I've described how to add support of mobile version of website using django. In this post I'm going to stop on front-end side of mobile website.

From the beginning, I wanted to create a simple mobile version from scratch and use a similar to original website look. But, after some investigation, I've changed my plans. I decided to use some framework to create a mobile website. So I found two popular such frameworks: Sencha Touch and jQuery Mobile. Whilst Sencha Touch looks awesome and production ready, while jQuery Mobile is still in release candidate stage, I stopped on the second one. This choice was very simple for me: jQuery products usually are qualitative, there is a strong community behind them and they introduce new versions very quick. Nevertheless Sencha Touch is a good product too.

I'm not going to describe how I used jQuery Mobile step by step, but only list of some facts and tips. First of all, I was using jQuery Mobile version 1.0 RC2, and that means, it wasn't a production-ready version yet. Even so, I found a few bugs or maybe just a things that need an improvement.

I just love such structures as lists. So will list a fact using a list =).

• You should read a manual first and follow a standard layout mechanism, like declare header, content and footer sections. This will save you a lot of time.
  
  
  

  <div data-role="page">
             <div data-role="header">
                 <a href="{% url m_home %}" data-icon="home" class="ui-btn-left">Home</a>
                 <a href="{% url m_help %}" data-icon="help" class="ui-btn-right">Help</a>
                {% block header %} <h1>[App Name]</h1>{% endblock %}
             </div>
  
             <div data-role="content">
              {% block content %}{% endblock %}
             </div>
  
             <div id="footer" data-role="footer">  Copyright © 2011 ... </div>
         </div>
  

  
  It's a good practice to add Home/Back and Help buttons to the header or footer panel. The navigation bar can be also added to the header, where it looks very natural:
  
  
  

  <div data-role="header">
     <a href="{% url m_home %}" data-icon="home" class="ui-btn-left">Home</a>
     <a href="{% url m_help %}" data-icon="help" class="ui-btn-right">Help</a>
     {% block header %}<h1>[AppName]</h1>{% endblock %}
  
     <div data-role="navbar">
        <ul>
           <li><a href="{% url m_download %}">Download</a></li>
           <li><a href="{% url m_contact %}">Contact</a></li>
           ...
        </ul>
     </div>
  </div>
  

• Fixed position of header and footer isn't working as it's expected. Well, expected it to be fixed always, while content is scrolling. But currently it just disappears when user starts scrolling and shows back after a second or so scrolling is ended.
• Hash anchor links (#something) aren't working yet.
• Can't show an image as a page, instead shows an "undefined" text. If you have a links that reference images, you must specify rel="external" for them.
• Use header to specify page title, rather than do it in the h1 element in the content section.
• Use collapsible blocks for long pages. For instance, I've found them very helpful to organize a help page with many different topics.
  
  You may also want to embrace collapsible blocks using collapsible set.
• For large image, don't show original version of it on page, but prefer thumbnail image as a link to the original version of image.
• Use list view where possible. For instance, a main help page contains a list of articles which are references to the appropriate pages.
• Buttons should be with icons where possible. You can easily add icon to the button with the data-icon="[icon]" attribute. The list of available icons is on jQuery Mobile website.
• Make website as simple as possible: structure should be simple, navigation bar should contain only the most used links. Remember, that it's still hard to work fully with websites on mobiles.

Well, that's all as for now. Hope that would be helpful.

Mobile website with Django

Although I'm not a web guy, but more specialized on backend logic, I had a very interesting experience today creating a website version for mobiles. I had a simple website with a few pages, minimum server side logic and a slice of documentation there. This website perfectly works for desktop and tablet browsers, but is not that good for mobiles. I use to call this website as "normal" version throughout this post. It's important to have a working mobile version of website, as website is about Android application.

A little bit about technologies and task. Website is running on Django as well as mobile version should be too. Mobile version of website should be much simpler than normal version, not so beautiful, but must do own job well: allow users to know about the product, download and access to the documentation.

I decided to use the same django application for both normal and mobile website versions. I found a few different ways to add mobile website, for example: redirect to another django application, replace TEMPLATE_DIRS setting with a version that points to mobile templates if user is using mobile browser or just check and use different logic/templates in every view method. As I wanted to have a different structure for mobile website, I decided to go by my own way.

Mobile website path should be prefixed with /m/ rather than be the same or use a different domain name. So, if normal website download page is http://example.com/download/, then mobile download page should be http://example.com/m/download/. Django should check every user request whether it's from mobile browser or not using a User-Agent metadata, and if so, redirect to the mobile version of the page. In other words, if user opens a /download/ page from mobile browser, then she will be redirected to the /m/download/ page. But if she opens a /m/download page, then no redirects happen - just a mobile version of page is returned to user.

I've created different templates and views for mobile version of website. Normal version of website has references only to the normal pages and mobile version of website has references only to the mobile pages. This was done to avoid not necessary redirects that could slow down a work with website. Thus, I had a different version of URL patterns for website in project's urls.py:

urlpatterns = patterns('website.views',
    url(r'^$', 'home', name='home'),
    url(r'^download/$', 'download', name='download'),
    url(r'^screenshots/$', 'screenshots', name='screenshots'),
    url(r'^contact/$', 'contact', name='contact'),
    ... 
)

urlpatterns += patterns('website.mobile',
    url(r'^m/$', 'home', name='m_home'),
    url(r'^m/download/$', 'download', name='m_download'),
    url(r'^m/screenshots/$', 'screenshots', name='m_screenshots'),
    url(r'^m/contact/$', 'contact', name='m_contact'),
    ...
)

As you see, website.views are responsible for handling normal website pages, while website.mobile for handling mobile website pages.

Now need to redirect mobile browser users to the mobile version of website. For this purpose, I wrote a custom middleware called MobileWebsiteMiddleware. The task of this middleware is to check if user is from a popular mobile platform and needs to be redirected to the mobile version of website. The other responsibility is to redirect users from mobile to the normal version of website in case if request isn't from mobile browser. This is an extract option, that should be disabled while testing.

class MobileWebsiteMiddleware(object):

    MOBILE_PREFIX = '/m/'
    MOBI_REG = re.compile(
        '(iphone|windows ce|mobile|phone|symbian|mini|pda|ipod|mobi|blackberry|playbook|vodafone|kindle)',
        re.IGNORECASE)

    def process_request(self, request):

        if 'HTTP_USER_AGENT' in request.META:
            userAgent = request.META.get('HTTP_USER_AGENT')
            matches = self.MOBI_REG.search(userAgent)
            path = request.path_info

            if matches:
                # from mobile browser, check if need to redirect to mobile
                if not path.startswith(self.MOBILE_PREFIX):
                    # need to redirect to mobile version
                    return HttpResponseRedirect('/m' + path)
            elif path.startswith(self.MOBILE_PREFIX):
                    # need to redirect to normal version
                    return HttpResponseRedirect(path[2:])

        return None

If you'll check the list list of user agent patterns in MOBI_REG you wouldn't find neither iPad nor Android. I want to show a "normal" website for tablets. "Android" keyword is used in User Agent for both mobiles and tablets. While Mobile Android browsers pass a User Agent with a "mobile" keyword, but Tablet Android browsers don't do that. Thus, I will have a normal version for desktop and tablets browsers, but mobile version for mobiles and Kindle.

Now I have a simple website, that has two different version: one for desktop and tablet browsers and another for mobile browsers. User will be automatically redirected to the right version of website which depends on used device.

In my next post I'm going to tell more about using jQuery Mobile to build a mobile version of website.

Django: use global variables on rendering templates

Problem

Some settings or global value is used in every template or, for example, in base layout template. This value should be passed as context variable to each template. I call such value a global context variable in this post.

For instance, it can be a Google Analytics Code, which should be used in production, but not in staging or development environments. In this case, it’s fine to move GA code to the settings file (of course, if different settings used for different environments). Though need to find a way to get the value from the settings and pass it to the template context before rendering.

Solutions

There are a few simple solutions that solve the described problem:

1. Add global variable to the template context in every view. But according to DRY and good sense this is a bad solution.
2. Write a tag to fetch setting value by name and output it. Nice solution and flexible enough in case we need to show different settings in different pages. The only disadvantage is fetching setting value each time it’s accessed. But in most cases this is a minor.
3. Write a context processor. In the documentation said that context processor is a method that takes a request object as argument and returns a dictionary of items to be merged into the context. In other words, context processor returns a dictionary of items that will be available in template on render phase. In our case, we can get setting value and return from our own context processor method.
   
   

   from django.conf import settings
   
   def global_vars(request):
       return {'GA_CODE': settings.GA_CODE}
   

   Also, need to declare our context processor in settings.TEMPLATE_CONTEXT_PROCESSORS:
   
   

   TEMPLATE_CONTEXT_PROCESSORS = (
      . . . 
      'synctab.website.context_processors.global_vars',
   )
   

   From now, GA_CODE context variable can be used in any template, and will be processed correctly.
   

In my opinion, the best decision for fixed list of globally accessed template context variables is a custom context provider. It's also a right way, if most or all templates use same global variables. Otherwise, it's good to write a few tags to cover access to the global state, like settings.

Performance Optimization for Android

Here is the list of the performance optimization tips when developing Android applications:

• Avoid creating objects if you don’t need them.
• Avoid memory allocation if you can work without it.
• Array of ints is preferred than array of Integers.
• Two parallel arrays of ints are a lot more efficient than an array of (int,int) objects. Two parallel arrays of Foo and Bar are a lot more efficient than an array of (Foo,Bar) objects.
• Avoid creating short-term temporary objects if you can.
• Make your method static. Invocations will be about 15%-20% faster. It's also good practice, because you can tell from the method signature that calling the method can't alter the object's state.
• It's reasonable to follow common object-oriented programming practices and have getters and setters in the public interface, but within a class you should always access fields directly.
• Direct field access is about 7x faster than invoking a trivial getter.
• Use static final for constants.
• Prefer for-each loop.
• Declare fields and methods accessed by inner classes to have package access, rather than private access.

Read Designing for Performance article from Android Dev Guide to find more details and samples.

List also available as gist.

Neverending Groovy: compareTo() simple as it can be with <=>

Groovy introduced a new operator to compare two values: <=>.

When you use this operator for 2 values a <=> b, Groovy translates this to code something like that a.compareTo(b). Well that's not the true, as it also can handle null values correctly. So you wouldn't get NPE, when you compare value with null or vice versa. For instance, next script:

a = null
b = 12

println "a.compareTo(b) = " + (a <=> b)
println "a.compareTo(a) = " + (a <=> a)
println "b.compareTo(b) = " + (b <=> b)

outputs

  a.compareTo(b) = -1
  a.compareTo(a) = 0
  b.compareTo(b) = 0

where we can see, that null values are handled correctly, without any runtime exceptions.

With such operator we could write really nice and simple code to allow multiple comparisons for object values.

Lets we have a class Person with firstName, lastName and age data:

class Person {
    String firstName
    String lastName
    int age
}

And we need to sort a list of Persons by age, than by name (first by last name, than by first name). Assume, this is standard sorting mode, so we may prefer to implement Comparable interface:

class Person implements Comparable {
    String firstName
    String lastName
    int age

    int compareTo(Person other) {
      ...
    }
}

Method compareTo() isn't implemented yet. So lets try to implement in Java first. It would look something like this:

int compareTo(Person other) {
        if (other == null) {
            return 1;
        }

        int result = 0;
        if (this.age > other.age) {
            result = 1;
        }
        else if (this.age < other.age) {
            result = -1;
        }
        if (result == 0) {
            if (this.lastName != null && other.lastName != null) {
                result = this.lastName.compareTo(other.lastName);
                if (result == 0) {
                    if (this.firstName != null && other.firstName != null) {
                        result = this.firstName.compareTo(other.firstName);
                    }
                    else {
                        result = this.firstName != null ? 1 : -1;
                    }
                }
            }
            else {
                result = this.lastName != null ? 1 : -1;
            }
        }
        return result;
    }

Hm... Looks ugly! Too much if's, too much checks, too much 0, 1 and -1. Hard to see the logic behind the code ((.

And here how this can be written with Groovy's <=> operator:

  int compareTo(Person other) {
        age <=> other.age ?: lastName <=> other.lastName ?: firstName <=> other.firstName
    }

And a simple test:

def john = new Person(firstName: 'John', lastName: 'Doe', age: 25)
def mark = new Person(firstName: 'Mark', lastName: 'White', age: 25)

print "john <=> mark = " + (john <=> mark)

prints

  john <=> mark = -1

As for me, it looks wonderful.

Neverending Groovy: map with default values

More and more Groovy openings for me every day!

Just today, I found that there is a way to setup default value for map entry, if absent. In some cases it may be very handy.

For example, I need to prepare a map with words counts. This simple task should be coded simple too, like:

def map = [:]
words.each { word -> map[word]++ }

But it's not working and fails with NPE exception. Why? Because it's hard to increment null.

Although, there is simple way to avoid this problem even using Groovy API:

def map = [:]
words.each { word -> map[word] = map.get('word', 0) + 1

Here 2nd parameter of map.get() method is the default value, returned if there is no value for key 'word'. And now it's working fine!
Still there is another way, and, as for me, it helps to make code clean and safe. Safe because default logic can be used elsewhere we want to use our map object.

def map = [:].withDefault { 0 }
words.each { word -> map[word]++ }


I bet there is a better way and I'm going to find it...

Code Generation at Java

Some time ago I found how the code generation techniques are used at Java.
Java NIO exposes a set of Buffer interfaces with package level implementations that are returned while creating buffers. If you will look at those implementation classes (HeapByteBuffer or DirectByteBuffer) you will notice that they are generated. Look at the file header comments - and you will see notion that it's generated and the name of the template. You may find templates with description of it's parameters here.

Code Generation

We are writing the code and this is our job. That's the art that we make. We put our thoughts, our patient, our soul to make it ideal or near to ideal. Sometimes we just write the code asap as there is no time or code is simple or we wrote similar code hundreds of times. This small article is about writing similar code again and again.

When we are writing similar code second or fourth time it's not an art and it is tedious. But that's useful to polish previous code: we can find and fix bugs, add needed logging, rewrite with better performance, make the code clean and handy to read and change etc. Often we are changing previously written code to make it better as well.
But what if you need to write similar code tens or hundreds of times? Will you change previous 20 classes because now you find the best algorithm or fixed the annoying problem with performance? Will you be glad to fix 50 classes over the project because just now you find the bug? (so it must be fixed, right?)
That will take your time, that you could spend on coding something interesting or studying some new popular technology or have a beer with your friends. And what about customer for whom this software is a business.
That's why I'm writing about code generation, the thing we all know and use often. You remember, when you're creating new project or adding new class with your favorite IDE, you got initial code so you can write your important part rather that spend time on writing the same code again and again.
While reviewing code generation tools we can divide them into external and internal. External are provided by IDE or other providers (remember famous xdoclet toolkit?). Internal is written by your team and used withing some one or few projects.
Internal code generators could be simple enough to generate only basic template source code or UI that will be changed and extended. It also can be large and complex to generate layers (domain types, DAOs and repositories etc) of your software.

Lets review next types of code generators:
1. template code generators
2. partial code generators

Template code generators run once at the begin. They are responsible for generating source code that will be rewritten or extended by developer. It's important to have clean generated code with comments for generated code and for placements where to put your code. It's good to use such generators if you have already ideal code to be generated. Ideal means that you will not need ever to change template and regenerate the same code again and again. Regenerating code will remove code written by developers with hands.

Partial code generators could be run as much as you will need. They are separated from code written by developers. For C# that would be good to use partial keyword (as I remember partial class declaration was added to have distinct generated and written by developer code for WinForms and ASP.NET); for Java extending could be used as well. We use generator to generate only similar code and put in another file, e.g. partial class definition or base abstract class. The specific code is written in derived class. While we find new bugs and improvements we need just to change templates or generator configuration and regenerate the code again.

Although code is generated it also must be easy to read it. Document your code, as you need to do it only once - in templates, so don't be lazy and stingy! For partial code generator or template code generator that is twice as important, as you are working directly with generated code.
Don't forget to add comment with notion that code is generated automatically and can be regenerated again, so other developers will think twice before changing it by hands!

Also remember that it's okay to add code generation to project step-by-step. For example, I'm the only who is using the new generator for now. I need to do so to find and fix defects, find the parts of code that could and must be generated, improve configuration and way of use. In this case the generated base code (with partial code generator) is commited too.

Use the best tools to write templates. For example I'm using Freemarker template engine to describe templates and generate source code. There are few code generators that read configuration file and generate appropriate code. For such tools performance is not as important as flexibility of templates and configuration.

Document the generator. Share source code within team. Put your generator into source code repository near to the project, but not into it. Also make and tag latest stable version as runnable scripts or program, so other developers can update and use it without spending on it additional time.

Spring MVC and RESTful web service.

This article is a result of my experience on building small RESTful web service using SpringMVC 2.5. Web service is simple, it supports only few operations, most of them just retrieves data and sends back it in XML.

Well, the task is simple enough:

1. build secured web service for mostly getting and sometimes posting data;
   
2. REST interface is required
   
   • but SOAP could be added later (I think that if you build SOAP web service you should wait that you'll need to provide REST API for this web service too and vise versa).

As web service is build with REST philosophy, than we need to support processing of GET, POST (well, for me it's enough) requests and return data as XML or JSON. Spring MVC as well others REST web service frameworks and API help with this too. So, what's next?

As application is build with Spring 2.5, it was the best to have some framework for building REST web service easily and easily to integrate it with Spring. Not a secret, that Spring MVC integrates with Spring better that any other web framework or REST web service framework (hm, maybe it's because it is build on Spring). Well, this is cons for using Spring MVC and it is good, but truly - this not enough yet.

Yet one requirement: SOAP web service could be added later. And if it could be added than (in most cases) it will be added. This is important, very important, because we want to build easy-to-extend software. And in case of SOAP WS I prefer to use Spring WS, that is well integrated with Spring MVC. Need also to take into consideration that some REST web service frameworks hardly integrates with Spring MVC or wanted to process all requests. That is a great pros for using Spring MVC.

You may understand what I'm implying. Yes, Spring MVC will be used to build RESTful web service.

Architecture

The main goals of architecture are:

• be scalable (not to much, but enough to process thousands of requests per hour)
  
• be extendable (SOAP must be easy to add)
  
• do what is required and nothing more
  

Next decisions were made:

• request should be stateless
  
• each request should be processed as quick as possible
  
• there will be 2 levels of caching: one cache for entities and queries (used by web service backend service), and another cache for rendered XML response (used by web service endpoint).
  
• everything that could be changed is configured through simple XML configuration (i.e. cache configuration, security configuration, business logic configuration etc.)
  

The main result of made decisions & rules is request processing lifecycle, that consists of next few phases:

1. Authentification & Authorization (security filter)
   
2. Getting request parameters. Web service endpoint (in our case it is SpringMVC controller) parses request parameters and validates them.
   
3. If everything is okay than it tries to get XML response from cache and sends it as response to the client. This step is optional and is processed only if cache is enabled and client does not specify request as such that ignores cache.
   
4. Endpoint determines the appropriate WebService backend service and run it and get results.
   
5. Endpoint converts results into XML, prepare response and stores it at cache (if cache is enabled). After the XML response is send back to client.
   

So, if the request response is cached in 2nd or 1st cache levels, than response will be generated without hitting database. In my development environment caching helps to increase performance up to 1500%. In production this number is less - just 300-500%. Still these is very well.

This lifecycle helps to build a scheme of responsibilities and modules. Later each responsibility could be processed by own server or a set of servers.

Simple implemenation

Spring 2.5 gave us possibility to use annotations as for declaring services as for declaring SpringMVC controllers. Spring's annotations Controller, RequestMapping, RequestParam and flexible politic on request handle method parameters gives us brilliant instrument for building web service endpoint.
Ofcourse, hardcoding of URLs in annotations looks as not the best way for building flexible and configurable application. But, this is quite pardonable, because we will use UrlRewriteFilter. UrlRewriteFilter is very useful tool, as it will helps to rewrite URLs, get parameters from URL and do other interesting things. But for our the most important are next 2 features:

1. flexibility while mapping external URLs (used by users) to internal URLS (processed by endpoint controllers)
   
2. possibility to catch parameters from URL, e.g. http://example.com/book/edit/12 ->
   http://exampl.com/book/edit?boodId=12 etc.
   

Lets look how we could use this features:
Let we have RESTful web service that returns XML with information with specified by id book. Web service clients could get this information by requesting url
http://ourservice.com/ws/rest/book/{bookId} (e.g. http://ourservice.com/ws/rest/book/12). Our application should handle this call and the information about book wit specified id.

Here is our web service controller:

@Controller
public class WebServiceEndpoint {

     @Autowired
     private BookService bookService;

     @RequestMapping(value = "webservice/book", methods = RequestMethod.GET)
     public ModelAndView getBook(@RequestParam(name="bookID", required=true)int bookId, Map<String, Object> model) {
         model.put("book", bookService.getBookById(bookId));
         return new ModelAndView("template/ws/bookInfo", model);
     }
}

Code is very simple, but it's good as for sample.

So, we have controller that processes all GET requests for 'webservice/book' resource, that should have request parameter of type integer with name 'bookID'. Endpoint just call backend service, gets information about book, put it to model map and return the ModelAndView instance with simple template that will be rendered and returned to client.

This endpoint controller can't be default process web service request. To redirect web service client request to our endpoint we will use UrlRewriteFilter

     <rule>
        <from>^/ws/rest/book/[0-9]+$</from>
        <to>/webservice/book?bookID=$1</to>
     </rule>

so request for resource http://ourservice.com/ws/rest/book/12 will be rewrited by filter to http://ourservice.com/webservice/book?bookID=12.


That's about all for now. The story looks to long. Most of us do not have enough time to read long articles :)

You and your's sofware expansibility.

For a long time one of the scores of software I'm writing is expansibility. That's mean how fast, with minimum of changes, I can add new features or extend existed one.
I've made few notes and use them as often as possible. Here are they:
- Never or almost never requirements you are working on are complete. Even if customer told you that this is finish step and we stop work on it - that mean only - we stop the work for now and we may continue it after some time. And of course, that time may vary from few weeks to few month, so you may and you will forget the code. That's why always write comments to the code even if it is not yours!
- If you can extend the code you've write before - do it! If you need to get the list of some entities - use search engine if it exists. If search engine is not enough - extend it and use it. Be DRY principal follower!
- Write your code with one think: I will use it tomorrow too! That's mean that you should write quality code, well commented code and code that make everything you have now and a little bit more. Sure, this "a little bit more" should be result of your experience. If you don't have enough experience - use this too! Why? You'll get need experience much faster!
- Do code review! Often. Why? Because it increases the quality of your code and as result of software. And also you'll find your and yours colleagues weak and that is 50% of new experience. I like Atlassian experience with code reviewing for many tasks they do.

Simple, but important.