Lazy loading on demand in CMS

Context:
Application is responsible for generating pages from templates. User can edit such templates to show data from storage. Data shown on the page are defined by template, each template can show separate set of data (for instance, news, tasks, articles etc.)

Problem:
How to render a user defined template correctly, so any provided by user or application data can be included into template in runtime and have a good performance of rendering?

Solution:
Lets view the problem from MVC point:
View is a template defined by user, and can use any data entered by user (news, blog articles, tasks etc.)
Model is data that should be used by a template engine to render a View.
Controller is a place where user can prepare the Model, that will be sent to a template engine to generate specified View.

As we can’t predict what kind of data are used in any particular View (it could be news, blog articles etc.), the simplest implementation is to fetch all data from storage and push them as a Model to the template engine. In this case template engine will render a View correctly, as all required data are available from the Model. The bad side of this approach is a very slow performance as we need to fetch all data and put them as Model again and again for each request. The more data and different types of data we have to use to populate a Model, the more time this phase will take.

Another approach is to analyze template on save or before render, to fetch all types of data we may need to render it right. For example, before saving a template, we found that it contains a references to blog articles and latest news. Although, we may need to run such operation only once and populate the model only with specified type of data, there is a large number of case not covered by this method: using predefined macros, including other templates and access to data that will be need in runtime, for instance: a single article with specified id, a list of latest 5 articles or anything else that need to be fetched dynamically.

If we think more about the last case, where we need to fetch data dynamically, we will find that it's much easier and better to populate Model not with data, but with data access objects. Of course, in this case we will need to create these data access objects on each request, to pass a context to theme and use them while rendering, it’s still much better (and faster), then populate Model with all data.

When I say a data access object, I don’t mean this object should get the data directly from database or any other storage. But I mean that this object can store request context and can access a restricted set of data, e.g. the list of news posted by current user only or the list of tasks that belongs to current user.

OK, hope your engine template supports methods call and you can get the list of 5 latest news just as simple as saying ${news.latest(5)} or get current user information just with ${user.profile()}.

But here I would like to propose another way to access those less dynamical data. For instance, we need to show user profile information or list of active tasks. Of course, this can be done with appropriate DAO from Model, but there is another way.

This another way helps to save templates clean, - without calling all those DAO methods, but also without prefetching data on the Controller side. This way is based upon a famous Lazy Loading technique.

Although, this may look a bit harder to implement, it brings us a few valuable benefits. First of all, as I said before, code looks clean and easier for user to work with. It's much pleasant to write ${news.latest} than ${news.latest()} and ${user.profile.firstName} than ${user.profile().firstName}.

Second, and it's related to the lazy loading too, we load data once and only when it's needed. The emphasis in on the word 'once'. When we hit our object from the Model first time, it's loading data, which then are available since and to the end of rendering phase. So no matter, from what part of template you access it, it will use already loaded data without touching a storage again. When I say "any part of template", I mean it can be anything available in template engine: macro, function or included template. In this case, you don't need to use a variable or local cache to store loaded data, as you may need to do with data access objects.

Third, lazy loaded data can be complex structures. For instance, we load a current user profile. Profile object is a structure that contains user names, characteristics, address, contact information and a list of followings profiles. Here, address and contact information are embedded objects, which are stored in separate tables/collections and list of followings is a reference to another a few profile objects. If you need to output user full name, you just don't want to load address information, contact information and list of followings. That's where we can also use lazy loading to save user time and server powers. We load address information when need, we load the list of followings when need, we load contact information of some following only when need.


In my next article I’m going to show a simple implementation of lazy loading technique for FreeMarker template engine, which is my favourite one for many years.

No comments: