memory: metrics graphs with gnuplot

So, at my spare time, I work a bit on my pet projects called memory. This is a simple virtual memory written on Java. User can allocate a large space of memory and then use it to allocate and free some blocks of it for personal use.

My main interest was to implement some memory allocation algorithms that could be working in concurrent environment with minimal of locks. I'm still on my way with it however, have some progress.

One of the problems with such projects is the way you can measure performance and quality. And what is more important, how you can assess you changes in these fields. Without a good set of performance tests, you can't know if your changes are for good. But having tests is not enough. You need to have metrics and gather them, and represent them in some charts for better visibility. Visualization matters. And then, when you have a way to look at those charts you can examine your's changes much much easier and better.

So that's what I did. First, wrote a simple metric gathering framework for internal use. Then added
metrics and used CodaHale Metrics to gather metrics data and save into CSV file.

Memory v0.1 is released

A few weeks ago I decided to create a simple library that emulates a virtual memory but with some extra features. My primary interest is to play with memory (de)allocation algorithms, replication algorithms, improve my knowledge in concurrency programming etc.

As for now, I have implemented a virtual memory with allocation mechanism, added concurrency and a basic transaction support.

Today I'm releasing a first version of this library, ie v0.1. This library can't do much yet, and if I'll have some time, I will start working on next version soon.

Library is open sourced, I'm hosting it on GitHub https://github.com/rkhmelyuk/memory. Documentation can be found in project's Wiki.

First memory should b allocated. There a few different allocators for this:

• FixedMemoryAllocator - to allocate a memory of fixed size
• DynamicMemoryAllocator - to allocate a memory of dynamic size (with initial and max size specified)
• FileMemoryAllocator - to allocate a memory mapped to file, supports both fixed and dynamic allocation

After memory is allocated, it can be used in application. The common workflow is to allocate spaces in this memory. Space represents a block of memory. This block will be not available for allocation till a space is freed. A space is like a window to specified block of memory: it can be used to read and write data at this block.

Spaces are thread-safe. This means that locking is used when read/write operation happens. For a single write operation there is no change other thread will read a corrupted data. For a multiple sequence read and write operation it's possible to use a transactional space currently. In the next version I plan to add a lock/unlock operation, so single client can lock and use space without a chance a space to be changed by other thread.

A simple example for the end of this post:

// allocate a memory with a fixed size 20KB
FixedMemoryAllocator allocator = new FixedMemoryAllocator();
Memory memory = allocator.allocate(20 * Memory.MB);

// allocate a space in the memory with a size 10KB
Space space = memory.allocate(10 * Memory.KB);

// write a string to space memory
space.write("Hi world");

// create a transactional space, and start a transaction
TransactionalSpace transactional = space.transactional();
transactional.start();
try {
    transactional.write("Hello");
    transactional.write(transactional.readString() + " world!");

    // in transaction - original space is not changed
    assert space.readString().equals("Hi World");

    // but transactional does
    assert transactional.readString().equals("Hello world!");

    // commit transaction
    transactional.commit();
}
catch (Exception e) {
    // rollback transaction
    transactional.rollback();
}

// check transaction was committed correctly
assert space.readString().equals("Hello world!");

// it's safe to use in multiple threads
executor1.processData(space.getInputStream());
executor2.processData(space.getInputStream());