Python is slooow.. Rust is fast.

I love using Python to play with data, solutioning or just prototyping. If I need to come up with some tricky algorithm, I often prototype in Python. Python is great for it, especially with Jupyter added. No compilation time, easy scripting, lots of libraries, especially those backend by native code written in C/C++. Using numpy and similar libraries makes things pretty fast comparing to just raw Python.

But then, any time you need to do lots of processing in Python itself, especially looping through amounts of data, you get hit by a performance issues, that make it inefficient to use Python codein production. Just recently, I needed to do some math and processing of 50MM-100MM elements in 2D array, and without numpy, that would take many hours if not days. Numpy helped to get it to 10-20 minutes. Significant reduction, but still too slow for me, if I want to make similar processing for tens of thousands of times.

I tried to re-implement this in Rust. Took me sometime given I'm pretty new in Rust, but it was a huge satisfaction to see that processing time dropped to 3-4 minutes, and after a few basic optimizations to 2-2.5 minutes. That sounds much better. Then I realized, I'm running this in debug mode. Switched to release mode, which added a bunch of own optimizations, and the time droped to 20-25 seconds. Wow!

But, I think, I still can do better. Can I use CUDA?..

Twister: A Simple Way to Manage Your Scripts

Imagine an average project that has many scripts, each written using different practices, uses different argument names, different namings, does something similar to other script but a bit different etc. Sometimes there are so many scripts, that it's hard to find the one you really need at this very moment. Moreover, scripts have no standard location, often put in semi-random directories, so it's really hard to find them. And even more, many developers have similar scripts for different projects. Some scripts are in the PATH, others are relative to the project directory. The one in the PATH are named in odd manner, because different versions used for different projects. And some scripts are written using bash and ruby and python etc.

Oh, so many troubles just because of the scripts. That's the reason why Twister was created. Twister is a simple framework/tool that allows to create and manage project scripts. I use it in a few of my home projects, and find it very helpful. About a year ago, I open sourced Twister. It's a python project, so it makes it simple to create scripts and execute them on any popular operating system.

Django: use global variables on rendering templates

Problem

Some settings or global value is used in every template or, for example, in base layout template. This value should be passed as context variable to each template. I call such value a global context variable in this post.

For instance, it can be a Google Analytics Code, which should be used in production, but not in staging or development environments. In this case, it’s fine to move GA code to the settings file (of course, if different settings used for different environments). Though need to find a way to get the value from the settings and pass it to the template context before rendering.

Solutions

There are a few simple solutions that solve the described problem:

1. Add global variable to the template context in every view. But according to DRY and good sense this is a bad solution.
2. Write a tag to fetch setting value by name and output it. Nice solution and flexible enough in case we need to show different settings in different pages. The only disadvantage is fetching setting value each time it’s accessed. But in most cases this is a minor.
3. Write a context processor. In the documentation said that context processor is a method that takes a request object as argument and returns a dictionary of items to be merged into the context. In other words, context processor returns a dictionary of items that will be available in template on render phase. In our case, we can get setting value and return from our own context processor method.
   
   

   from django.conf import settings
   
   def global_vars(request):
       return {'GA_CODE': settings.GA_CODE}
   

   Also, need to declare our context processor in settings.TEMPLATE_CONTEXT_PROCESSORS:
   
   

   TEMPLATE_CONTEXT_PROCESSORS = (
      . . . 
      'synctab.website.context_processors.global_vars',
   )
   

   From now, GA_CODE context variable can be used in any template, and will be processed correctly.
   

In my opinion, the best decision for fixed list of globally accessed template context variables is a custom context provider. It's also a right way, if most or all templates use same global variables. Otherwise, it's good to write a few tags to cover access to the global state, like settings.