memory: metrics graphs with gnuplot

So, at my spare time, I work a bit on my pet projects called memory. This is a simple virtual memory written on Java. User can allocate a large space of memory and then use it to allocate and free some blocks of it for personal use.

My main interest was to implement some memory allocation algorithms that could be working in concurrent environment with minimal of locks. I'm still on my way with it however, have some progress.

One of the problems with such projects is the way you can measure performance and quality. And what is more important, how you can assess you changes in these fields. Without a good set of performance tests, you can't know if your changes are for good. But having tests is not enough. You need to have metrics and gather them, and represent them in some charts for better visibility. Visualization matters. And then, when you have a way to look at those charts you can examine your's changes much much easier and better.

So that's what I did. First, wrote a simple metric gathering framework for internal use. Then added
metrics and used CodaHale Metrics to gather metrics data and save into CSV file.

SyncTab is open sourced

As I wrote yesterday on my twitter, SyncTab had been open sourced. About an year ago I started working on the project that would simplify my life. I used a lot my new smartphone to browse internet and read RSS, but the most interesting articles I wanted to read in my browser on laptop. And I can be pretty lazy and its easy for me to forget things, so I wanted to be sure that I won't leave this links without attention. The only way I could think about was just open the link in browser as soon as possible.

Structure101

I've found today my post on project wiki from Sept 1, 2008 about Structure 101. Structure101 is really a useful piece of software. Here is a post, which was written to introduce this program to other team members.

Structure101 is used to visualize the structure of application in graph of modules/packages/classes or in dependency matrix presentation. It is also hepls to build the architecture diagram with violations check support, find the tangles, fat packages, classes and methos in code-base.
Structure101 also provides web application that helps to find the information about current status of codebase, watch for changes in code-base, code complexity and dependencies. The web application shows information in informative tables and charts, helps to watch the changes from previouses versions etc.
Structure101 provides the plugins for IDE (Eclipse and IntellijIdea. Plugins can synchronize with public repository (it can be configured to use web application to get data from repository), so any updates to repository are available to developers and developers are always updated with the last architecture desicions.
Plugin can check for architecture violations while IDE builds the project and show the messages as warnings or errors (depends on configuration).

Using Structure101 were created few diagrams (Dao and Services, Functional Presentation, Layers Diagram, Current architecture diagram] that helps to see the architecture almost-complete-view.
In picture you may see the screenshot of Structure101 with architecture Layers Diagram:




IntellijIdea and Structure101
Developer's IDE (IntellijIdea) with installed Structure101 plugin can help to prevent not permitted dependencies within code:




Structure101 web application
Structure101 has also web application that can be used to view the project code-base status using WEB. Web application gives to the architect and project manager complete information about dependencies, changes after previous version, tangles, XS etc.

Pricing
- Structure101 is payable. See more http://www.headwaysoftware.com/products/purchase.php.
+ Structure101 web application as plugins for Eclipse and IntellijIdea are free of charge.

Tips: developing for Android

Here I want to post a list of some interesting facts about developing applications for Android. Although, my experience with it isn't huge, I believe this information will be useful for starters who want to create an Android application.

Lets get started.

1. To publish your application to Android market you'll need to register and pay registration fee - $25. It's paid only once and is not returned.
2. Name your application simple. If you application isn't part of your brand (like Facebook, Gmail or Twitter), it's the best to title application with some description of it. For example, Send Links and Phone 2 Chrome are good names, but SyncTab is not. At least, call it like 'SyncTab - send links easily'. The name should help user to understand what's your application for.
3. Don't hard code values. Move them to appropriate files like strings.xml, integers.xml, arrays.xml, colors.xml etc. This will save your time to localize application and hone application for different devices.
4. Use styles and themes.
5. Prefer to use Content Providers to work with database.
6. Use different forums to find first users and beta testers.
7. Release often. This will help to get new users faster. Each time you release a new version of application, it's added to the Just Released feed. Prefer to make such releases before or on weekends.
8. Make a nice but simple icon. Follow Android's style guide.
9. Remember that you'll need a promotional and feature graphics.
10. Use different techniques to cleanup the code. Like Android annotations, intent helpers etc.
11. Optimize!
12. Investigate Android applications of other developers, like iosched application from Google.
13. You can use Google Analytics to track not only usage of application, but also errors
14. Save resources: memory, cpu, network traffic.

SyncTab architecture. Part 2. Android Application

This post is a continuation for recent post with description of SyncTab server application. In this post I'm going to tell about SyncTab Android application.

As android application is very simple, I will make a very short description.

SyncTab application has a few activities only: the main activity is a list of shared links. When user first opens an application, she can either login into or register a new account. There is also a way to reset a password. User can only reset password, as original password is not stored; only salted hash of the password is stored on server. After user is signed in, she can see the list of links, edit settings and tags etc.

When user is authenticated and device is online, application starts two services: SyncService and RefreshService.

RefreshService connects to the server and checks if there are any new links to open on this device. Each device has an associated tag and opens only links with specified tag. Android device by default checks for the links with Android tag, and opens them if any. By default, RefreshService checks the server every minute if connection is up. But it is possible to change a period time or disable this service at all (if user doesn't want to receive links on device).

SyncService is used to synchronize android application status with server. Application allows to send links, when device is offline. In this case, link can't be send, but is added to the sync queue. Well, there are other operations that can be proceeded offline, like logout, remove link, re-send link, add/edit/remove tags. If device is offline, such operations are added to the sync queue. When network is app and device is online, SyncService checks the sync queue and executes each task in it. Sync queue tasks are persistent and stored in the database.

No hard coded values, except preferences names. Everything moved to the appropriate files like strings.xml, array.xml, integers.xml, colors.xml etc. Values vary not only for different localization, but for different devices (e.g. phone and tablet).

Documentation is available only online from synctab website. Website has two versions: for normal clients and for mobile clients. While, my phone will be redirected to the mobile version, a tablet will open a normal version of website.

There are two managers for each entity, a regular manager and remote manager. Application works mostly with regular manager, which itself calls remote manager if need. For example, there are TagManager and RemoteTagManager. To call an operation, activities use SyncTabFacade, while SyncTabFacade calls managers.

Activities started using special IntentHelper class, that provides a static functions like showLoginActivity(), showAboutActivity(), browseDocumentation() etc.

Links' favicons are cached as files in the cache directory. When a list of new links is retrieved, a favicon preloader thread starts. Thus, favicons are downloaded in background. If device is offline, and favicons can't be loaded, than an appropriate task is added to the sync queue for each favicon (and next time device is online, the favicons will be downloaded).

Well, looks like nothing interesting I can tell more about SyncTab android application.

So I decided to write a list of helpful facts about developing for Android as a separate post – "Tips: developing for Android".

Memory v0.1 is released

A few weeks ago I decided to create a simple library that emulates a virtual memory but with some extra features. My primary interest is to play with memory (de)allocation algorithms, replication algorithms, improve my knowledge in concurrency programming etc.

As for now, I have implemented a virtual memory with allocation mechanism, added concurrency and a basic transaction support.

Today I'm releasing a first version of this library, ie v0.1. This library can't do much yet, and if I'll have some time, I will start working on next version soon.

Library is open sourced, I'm hosting it on GitHub https://github.com/rkhmelyuk/memory. Documentation can be found in project's Wiki.

First memory should b allocated. There a few different allocators for this:

• FixedMemoryAllocator - to allocate a memory of fixed size
• DynamicMemoryAllocator - to allocate a memory of dynamic size (with initial and max size specified)
• FileMemoryAllocator - to allocate a memory mapped to file, supports both fixed and dynamic allocation

After memory is allocated, it can be used in application. The common workflow is to allocate spaces in this memory. Space represents a block of memory. This block will be not available for allocation till a space is freed. A space is like a window to specified block of memory: it can be used to read and write data at this block.

Spaces are thread-safe. This means that locking is used when read/write operation happens. For a single write operation there is no change other thread will read a corrupted data. For a multiple sequence read and write operation it's possible to use a transactional space currently. In the next version I plan to add a lock/unlock operation, so single client can lock and use space without a chance a space to be changed by other thread.

A simple example for the end of this post:

// allocate a memory with a fixed size 20KB
FixedMemoryAllocator allocator = new FixedMemoryAllocator();
Memory memory = allocator.allocate(20 * Memory.MB);

// allocate a space in the memory with a size 10KB
Space space = memory.allocate(10 * Memory.KB);

// write a string to space memory
space.write("Hi world");

// create a transactional space, and start a transaction
TransactionalSpace transactional = space.transactional();
transactional.start();
try {
    transactional.write("Hello");
    transactional.write(transactional.readString() + " world!");

    // in transaction - original space is not changed
    assert space.readString().equals("Hi World");

    // but transactional does
    assert transactional.readString().equals("Hello world!");

    // commit transaction
    transactional.commit();
}
catch (Exception e) {
    // rollback transaction
    transactional.rollback();
}

// check transaction was committed correctly
assert space.readString().equals("Hello world!");

// it's safe to use in multiple threads
executor1.processData(space.getInputStream());
executor2.processData(space.getInputStream());

SyncTab architecture. Part 1. Server.

This post is a continuation for my previous post with introduction to SyncTab. I'm going to describe SyncTab server application in this post.

Well, I had a desire to write a server application on python, but in result I wrote it on Grails. Why Grails? Because I was actively using it for more than a year to creating different web applications, including web API. And I knew that would be simple and fast to create such app with Grails and publish some sort of web service.

So server side is running on a single Amazon EC2 instance. Application is running under Tomcat. Nginx plays a role of proxy. MongoDB is used as a database, and mongodb grails plugin used to add MongoDB.

API is very simple. It's based on GET/POST requests and JSON result with a status flag and other values, like auth token, list of tags and links.

Some code sample:

/**
* Returns the list of last available tabs.
* Client can pass a "tabId" parameter with an id of the tab to get tags for.
*
* Returns:
*  - status: "true" if was removed, otherwise "false".
*  - tabs: an array of found tabs.
*/
def getLastTabs = {
if (request.method != 'GET') {
response.sendError 405
return
}

User user = session.user
Tag tag = getTagFromRequest()
RecentTabConditions conditions = new RecentTabConditions(user, tag, LAST_TABS_NUM)
List tabs = sharedTabService.getRecentSharedTabs(conditions)

render([status: true, tabs: prepareTabs(tabs)] as JSON)
}

An authentication token system is used in SyncTab. Each time user enters his username/password to login, a new authentication token is created and assigned to user. Auth token is invalidated after two weeks.

As both Android application and Chrome extension should support not only login but also registration, it is a part of API too. Actually, user related API contains only 4 functions: register, authenticate, logout and resetPassword.

There are more API functions related to sending/receiving links and view/edit the list of tags.

A short about tags. Tags is just a way to name the device. For example, user can create a tag for his home computer and another tag for work computer, and tags for phone and tablet. Each device/browser is subscribed only to one tag, and will receive and open only links sent with specified tag.

Server not only responsible for storing and fetching links by client applications, but also for preparing links for user. Usually, when user sends a link, information about it contains only name of the device, tag id and the link URL. Server is responsible to fetch the link title, description, favicon link and the final URL. The final URL may differ from sent URL. For example, user could send a link like http://bit.ly/xNPUeo, but user should see this link as http://blog.khmelyuk.com.

Well, looks like there is nothing interesting left that I would like to tell about SyncTab server application.

Ah, source code is not public, and I'm not planning to open source it soon. As VCS I'm using Git and BitBucket as hosting service.

SyncTab architecture. Part 0.

A few months ago I was willing to write my own android application. And I also had a need - a way to send easily links from my phone to my browser and vice versa. Well, my phone is under Android, and my favourite browser is Chrome. So I decided to create the app to satisfy my desire to create and release Android app and make my life easier.

And as result of this, I have made a set of applications that forms a SyncTab. Obviously, those applications are Android application, server-side application and Chrome extension.

This post is like a preface, and next few posts will include a short technical description of each application.

Well, the work of application is very simple: android app and chrome extensions are both clients for the server. There is no web interface for application. User can get the list of all sent links and manage them using Android application.

In most functionality, Android app and Chrome extension are similar. But the only way to see the list of sent links is using Android application.


Some links:

• SyncTab website
• SyncTab Android application is on Android Market
• SyncTab Chrome extension is at Chrome Web Store.