Tuesday, 24 December 2013

What is going on with 1.0-alpha3?

Time between alphas in 1.0 release cycle is really long. Alpha1 was released 1.5 year after 0.12, alpha2 - more than month after alpha1. Another month passes but there is no alpha3 in sight. There are several reasons for this.

First, I'm the only active developer of Kadu now (beevvy is porting it to Qt5, but he is doing it in secret, so it does not count).

Second, I'm helping my friend with home overhaul.

Third, these are alphas for 1.0 release. I want them to be as solid as possible. Each alpha is dedicated to fixing one specific big issue (big in terms of required code changes). First alpha was done as mark that OTR is ready for generic use. Second one fixed issues #2621 and #2646. Now I'm working on better plugins handling (#2706). It requires a lot of work: splitting classes, writing unit tests, implementing graph algorithms. Most of the work is already done, but there is still GUI to be worked on.

Last big issue (that will make alpha4) is duplicating messages in chat view. After that, just little bugs will remain and we will be happy to switch to beta cycle.

Sunday, 1 December 2013

Source code structure evolution


Kadu source code evolved over years. Directory structure evolved along with it. It all stared with single code directory (named kadu), single autotools one (named admin) and a few with libgadu library sources. Above chart shows this evolutions as number of directoriesd use for different categories during last 12000 commits.

Lets start with boring ones: varia line shows how many icon and sound themes we had. During development some were added (like modern Adium themes) and some removed (because of licence issues or pure ugliness).

Modules and plugins lines shows how many code was kept our of core source for modularity. First modules emerged just after a few years: autoresponse, voice, sms... They were responsible for most of new directories for a very long time. Big switch from modules to plugins shows that we didn't think that all of them were worth the transition (there is subtle technical difference between module and plugin in Kadu terminology).

Count of classes in each files and files in kadu (now kadu-core), where core source is kept, directory was growing very fast from the beginning to the point where code was almost unmanageable. So the Big Split era was started. Each "business" object family got it own directory (buddies, contacts, chats, accounts, avatars, status and so on...). GUI code was split between widgets and windows and separate directories for configuration were created. This is all represented by core line on above chart. Lately speed of expanding this structure increased - some widgets got whole directories instead of just files. Number of support classes for handling ChatWidget (12)ChatWindow (7) and GroupTabBar (4) increased so much that it stopped to make sense to keep them with other widgets code. It all happens for our believe in Single Responsibility Principle.

For me it is very important to keep everything as clean as possible. When your code grows in lines be sure it also grows in number of files and number of directories. Keeping too much code in one class, too much classes in one file or too much files in one directory will eventually lead to Big ball of mud architecture. It is very easy to smuggle 1000 lines of code to a place that already has 10000 than to add small method to small class. Keep good structure on every level - it will force everybody to think deep about new code placement.

Wednesday, 6 November 2013

RAII + move semantics = ?

After last post about RAII I was looking for parts of Kadu code that could use this pattern.

One example was particularly interesting: before each write to libgadu socket notifiers must be disabled. And after each write they have to be enabled again. All writes are done by calling some libgadu functions with pointer to gg_session structure and, of course, some other parameters.

But there was nothing in Kadu implementation that prevented calling these functions without blocking socket notifiers. So I thought of extending our RAII class to be the only place where gg_session pointer could be acquired. I did it by making it movable (but not copyable). Result was named "token":

class GaduWritableSessionToken
{
  GaduConnection *Connection;

public:
  GaduWritableSessionToken(GaduConnection *connection) :
      Connection(connection)
  {
    Connection->beginWrite();
  }

  GaduWritableSessionToken(const GaduWritableSessionToken &copyMe)
    = delete;

  GaduWritableSessionToken(GaduWritableSessionToken &&moveMe)
  {
    Connection = moveMe.Connection;
    moveMe.Connection = nullptr;
  }

  ~GaduWritableSessionToken()
  {
    if (Connection)
      Connection->endWrite();
  }

  GaduWritableSessionToken & operator =
    (const GaduWritableSessionToken &copyMe) = delete;

  GaduWritableSessionToken & operator =
    (GaduWritableSessionToken &&moveMe)
  {
    Connection = moveMe.Connection;
    moveMe.Connection = nullptr;

    return *this;
  }

  gg_session * rawSession() const
  {
    return Connection ? Connection->rawSession() : nullptr;
  }

};

This looks pretty much like FileHandle class from last entry, but with two exceptions: it does not control resource lifetime but access pattern; and it has move semantics.

Next we need to replace gg_session getter from GaduConnection object with a method to acquire token:

GaduWritableSessionToken GaduConnection::writableSessionToken()
{
  return {this};
}

(This new C++ uniform initialization syntax looks pretty weird in this case).

Now it is the only way to obtain our protected resource. Thanks to move semantics we are sure that no instance of GaduWritableSessionToken will be ever duplicated in dangerous way (degenerated moved-from instances do not count) and only one pair of acquire/release methods (beginWrite()/endWrite()) will be called for each token. It can be moved (but not copied!) between scopes of objects and functions and can be used to safely access resource for its whole lifetime. When token goes out of its last scope, it is destroyed and its resource is released. Also, at the same moment, gg_session is no longer available.

This code is of course not thread-safe, but Kadu only uses separate threads to handle storing and retrieving history entries, so for our application it is not needed. For multithreaded code method for creating token should also disallow creating another one when first still exists. This can be easily implemented with some condition variables.

I hope that someone will find this useful.

Thanks for beevvy for pointing out that making this class movable is way better idea than to wrap it in std::unique_ptr.

Wednesday, 23 October 2013

Design for errors #2: RAII

RAII stands for Resource Acquisition Is Initialization. This is another simple technique that prevents a lot of errors in code.

Imagine that you have to read some content from a file. You can create code like this:

void readFile(char *fileName)
{
  FILE *f = fopen(fileName, "r");
  // some code
  fclose(f);
}

Note: error handling was removed from all code samples for simplicity.

This looks fairly good on first sight: opened file is closed at the end of function, so no file descriptor is leaked. But what if some code part contains branches and returns early in some of them? In that case explicit fclose() call must be added before each return. Other option is to enforce single exit point policy for whole code. But that is a bad coding practice and can lead to code with unusually large indentations.

RAII idiom helps in such situations. Minimal implementation of RAII for FILE object could look like this:

class FileHandle
{
public:
  FileHandle(char *fileName) : m_f(fopen(fileName, "r")) {}
  ~FileHandle() { fclose(m_f); }
  FILE * file() const { return m_f; }
private:
  FILE *m_f;
};

void readFile(char *fileName)
{
  FileHandle handle(fileName);
  // some code
}

This is simple class that handles all aspects of FILE initialization and destruction - it opens file in constructor and closes it in destructor. That is everything required to implement RAII idiom. What are the benefits?

Object handle exists only in scope of function readFile  When the scope is leaved (no matter how and where) handle is destroyed. This means that destructor is called and fclose(f) is executed. There is no need to add any code before each return and at the very end of function. Compiler takes care of it. In addition this version is exception safe - object's destruction will take place if an exception is thrown inside some code block.

Another plus of this is that we have to write at least 2x less code for file management - one line instead of two in case of just one exit point. Less code means less possible errors.

Such RAII classes should be used for everything that has state that needs to be restored, like:
Qt lack at least one of such classes. QObject defines blockSignals(bool) method that make a perfect pair (with true and false parameter) for a simple RAII class:

class QObjectSignalBlocker
{
public:
  QObjectSignalBlocker(QObject *o)
    : m_o(o),
      m_block(o->blockSignals(true)) {}
  ~QObjectSignalBlocker() { m_o->blockSignals(m_block); }
private:
  QObject *m_o;
  bool m_block;
};

This one block all signals from given QObject and stores previous value of signalsBlocked() status. At the end of scope this value is restored.

Within Kadu we use similar method to block change notifications of our business objects. Each of it has a dedicated ChangeNotifier (.h, .cpp) instance. On each change of property value a notify() method is called on this object and all interested parties are informed by changed() signal. However, if a bulk change is made (several properties are modified at once), there is no need to emit changed() each time. So a pair of block() and unblock() method is provided to enable buffering notifications and merging them into one. Lately I've realized that this is perfect candidate for RAII class, so ChangeNotifierLock (.h, .cpp) was born. It can be set to one of two modes: fire notification or forget all changes that was made during it existence.

Changing code to use it was really simple. We do not have many uses of ChangeNotifier. Also there was no single misuse of this (all block() calls were properly paired with unblock()), but in two cases distance between the calls was bigger than one screen of code. This could lead in future in unfortunate omissions and GUI breakdown (as nothing would get updated anymore). Using RAII class saved us a few lines of code and removed possibility of introducing one kind of error into our code.

From now I'll look for such cases more carefully and implement this idiom where applicable.

Wednesday, 16 October 2013

Kadu translations

In dark ages of Kadu our translations were handled exclusively in source repository (SVN, then git). Some time ago WhE has discovered Transifex site. It makes translating process very user-friendly. Everyone can join and help to make software accessible to more people. Interface is simple yet powerful - it even has suggestions! So now instead of forcing users to send files over e-mail or registering github/gitorious accounts we let them work on their own. All the hard work of updating and merging files is now handled by us.

Switching to Transifex helped Kadu a bit - we now have more translations. Here is list of active ones (percentages are for: whole package, core without plugins, only jabber):
  • English - 100% - 100% - 100% (this is source language)
  • Polish - 100% - 100% - 100% (native language of all current developers)
  • Czech - 95% - 100% - 90% (that is so nice!)
  • Vietnamese - 3% - 10% - 0% (someone just started working on this, we will see how far it will go)
Czech translation will be very probably ready in 100% for 1.0 and next releases.

Unfortunately, most translations are in abandoned state. Some are 2 years old, some were just started and forgotten a while later.
  • French - 64% - 55% - 66%
  • German - 41% - 59% - 60%
  • Russian - 28% - 78% - 0%
  • Italian - 25% - 18% - 39%
  • Spanish - 17% - 33% - 0%
  • Chinese - 11% - 1% - 78%
  • Slovak - 2% - 6% - 0%
  • Turkish - 1% - 0% - 0%
That means we will be able to deliver Kadu will only 3 languages (possibly 4, if Vietnamese gets over 80%). French, German and Russians will have to to be removed (its a shame, because we have shipped these with 0.12.x series).

We would be very happy to release Kadu 1.0 with more translations. So, if you have some spare time, know a language or two, register on Transifex and join Kadu project!

List of strings will not be changed much from now until release (I expect less that 1% of changed and new strings). So you can join and start right now :)

Wednesday, 9 October 2013

Design for errors #1: Immutable classes

There is only one thing in being a programmer that you can be sure of: you will make errors. There is no need for denying it, no matter how good you are, or how good you think you are. Not all mistakes can be detected during code review, unit or integration testing.

My proposition for dealing with this is: design for errors!

What does this mean? Make errors easy to spot and easy to fix. This can be done if you use SOLID principles in development. I would like to add some ideas of my own. Today I will start with first one (loved by functional programmers):

Create immutable classes

Immutable means: does not change after creation. Immutable class in C++ can only set its fields in constructor. No other method is allowed to change state of it (so all methods are declared const). Immutable class should not have any virtual method.

This has one big advantage over normal classes: you are 100% sure that no code is changing created value and you can use it everywhere without worrying about its state.

Look at OAuthParameters class in Kadu source code. It has only one of required properties: it does no have any virtual methods. But it has a lot of setters that change its state. We see in the code that these setters are all used just after object construction, so a Builder pattern should be used for creating instances of this class. There is also a mysterious sign() method. It has very bad property of changing state of this object from unsigned to signed - this state is hidden and not exposed by API in any place. Solution for this would be to make a new class, OAuthSignedParameters that can be created from OAuthParameters by new const method signed() or by some external class.

When all of this is done there will be two separate immutable classes (that are easy to reason about and really error-proof) that have strictly defined relationship. All methods that now require signed version of OAuthParameters will make this clear by requesting OAuthSignedParameters directly. A few types of errors go away with this rather simple change.

That's all for today.

Friday, 4 October 2013

Status update

After changing job (again...) I've found that I have more time and will to code at home ;)
Pace of Kadu development increased significantly and I must announce that OTR is now 100% usable with verification and policy support! (Well, we do not support instance tags as well as in Pidgin, but I don't think it is necessary for first release).

I hope to stabilize it in next week, do some refactoring and renaming in core to adjust for new API requirements and of course add Doxygen comments. After that 1.0-alpha1 will be released and I'll focus on bugfixing others are of application. Kadu has 148 open bugs on redmine, some of them are regressions from 0.12.3. I want to get rid of all regressions and crashes. It would be good to get under 100 again ;)

Sunday, 8 September 2013

OTR Update

Finally OTR in Kadu is usable. After fixing one bug that was a result of my misunderstanding of LibOTR API we can finally start and stop OTR conversations from Kadu and accept incoming OTR sessions.

This does not mean that Kadu supports 100% of OTR protocol. Multiple sessions for one account are not supported (and won't be for 1.0 release), generating keys is still synchronous (this will be changed soon) and there is no way to verify peer identity (this will be fixed for 1.0 release).

All of the code is now in master branch on gitorious and github - please download it and test ;)

Thursday, 11 July 2013

OTR is taking shape

OTR is almost usable now. First GUI elements were just added - status bar at top of chat widget (similar to ones used in browser, but much uglier for now). For now it only shows red or yellow background (for non-private and unverified conversations). Green color will be available after verification is implemented ;) I hope we will have full UI created in a month - then first alpha of 1.0 will be released.

Thursday, 13 June 2013

Be brave! Delete code!

Most developers have fear of changing code that they don't know. Adding new code is OK for everyone - even in large functions or large classes. That of course leads to mess in code and big ball of mud syndrome.

The main problem is that only a small fraction of developers actively try to understand what others have written before. Is this lazines? Or lack of confidence? Or maybe deadlines are tight and writing new features is more important that code refactoring or fixing bugs?

My advice is: don't be one of these guys. Don't do just what is assigned to you. Spend a little bit of time looking around for possible improvements. If you see a big method - split it. If some flow is hard to understand - improve it. If naming is bad - change it. And one of most important things - be brave and delete dead code. If it will be needed again (it probably won't) - you have source control systems (if not - change company).

Now imagine if everybody in your team have spend some time every week on improving exiting code, from beginning of project. I know it is a bit of investment of time for the project/company, but remember: changing clean code is much easier that changing big ball of mud. It will pay off.

Tuesday, 4 June 2013

Source access

Access to source is the key for developer's happiness.

Imagine that you must use some closed third party library and one thing just don't work. You do everything as documentation says (if there is documentation) or as internet says. And you don't have access to source code. What to do in that case? If you are lucky, and this is Java or .NET, you can use decompiler and hope that code was not obfuscated. If you are unlucky and library is pure binary - good luck with that!

With Open Source this is just easy - you can look at source code, get a version with symbols (or build your own) and use debugger to find what is wrong. Here is example from last days: Kadu was crashing when OTR conversation was initiated from it. Debugger only showed that there was bug somewhere inside otrl_message_sending function. I grabbed source code, got a quick look and found a bug that only occurred when specific parameter value was passed to it. Fix was ready in no more than 10 minutes - no need for stress!

PS. This bug was already fixed at the begining of this year in libotr 4.0.1 that will be released soon.

Wednesday, 29 May 2013

On stupidity

Everyone is stupid from time to time. Especially developers. When I first decided to implement otr in Kadu, I've looked at http://www.cypherpunks.ca/otr/ and for some reason, I've decided that 3.2.0 was the current version.

Unfortunately, this was not true. Current version is 4.0.0. And it is not compatible with 3.2.0. I had to spend additional day on porting our plugin. And this does not work yet (I don't know why). Sigh... Next time I will be more careful.

Tuesday, 28 May 2013

Living on my own

It is almost month now after I've left my job at Cognifide and decided to work with technology of my choice - Qt (I'm not really a Java guy). I was lucky enough to find Bluecherry - an US company that creates video surveillance software. They were looking for a Qt developer and I was looking for new challenges.

Bluecherry is based in US and have employees all around the world. So there is only one way to cooperate - remote working. After one month of doing only that I must say that this is the best way of working for me. The most important change is of course lack of interruptions. In can be very loud in open-space with about 50 people on one floor, without doors and walls. Also, there are always a lot of people that have questions for you and it is rude to not answer :( It is said that one interruption can cost developer 15 minutes to return to task. With big open-spaces interruptions can be so frequently that you will never go back to your task. In home I feel 10 times more productive (but this is probably only 2 or 3 times).

Next big thing is that I have more freedom with managing my time. I like to work late at night and I hate to wake up early. Now I don't have to :) And when there is a nice weather I can grab a bike and go for a ride in the middle of the day. And just work more at night ;)

Working at home also means less junk food. It was very easy to order a pizza with colleagues from work. Now I cannot remember when I last eaten pizza...

In summary: less stress, less distractions, more health!

PS: I work on Bluecherry Client that was recently open-sourced and is available on Github. So I'm working on open-source, C++/Qt project and I'm being paid for it! That's a developer dream...

PS2: Recently I got a second contract with Arise - so I was 100% open-source developer for only about a week, but this still counts for my developer karma

Tuesday, 16 April 2013

Google 2-step verification vs GTalk in Kadu

You want to have really secure mail account? You need to use 2-step verification process with cell phone as token. Google's GMail makes this very easy - just go to Security Settings of your Google account and enable it. On next login you will have to provide password along with special code sent to you by SMS.

Next thing that needs to be done is to allow Kadu to access your GTalk. 2-step verification can not be used, as Kadu has only one field for entering password. You need to create special application password just for it. Go back to Security Settings, click on Setting next to 2-step verification and select Manage application-specific password. Enter "Kadu" as application name, click "Generate Password". Enter this created password into Kadu's Account settings and you are done.

Saturday, 6 April 2013

Delivering

One of the most important thing in developing software is making releases. Almost no one will use your application, no matter how useful and stable it is, until it is released and has a nice version number (the bigger the better).

Releases

Making releases for Kadu is long and painful process. There is list of steps that must be done. Most of them are not automated, like:
  • create packages, sanity test them
  • upload packages into three different places
  • update git repositories (version branches, tags)
  • write release notes in two languages
  • publish release notes in four different places on two websites and one mailing list
Personally, I hate doing that. And for some reason I've never found time to automate most of the task. When I looked at the list last time, I've realized that most of these tasks could (and should) be automated. Of course release notes and announcements must be written manually, but what about all the other tasks? Lets check:
  • bugs checking on redmine - scriptable
  • translation checking - scriptable
  • git updates - scriptable
  • creating packages - scriptable
  • uploading packages - scriptable
  • sending mails to lists - scriptable
  • forum  - scriptable
  • blip - scriptable
  • kadu.im server stuff - scriptable
So we are left with only three things: writing some release notes (including taking screenshots and making screencasts) and updating our wiki and Wikipedia. If we replace our wiki with something simpler, like Wordpress or Drupal - we could get to just two manual steps. Also, we don't make release notes for bugfix releases. So we will need to do only one manual step in such case. That seems a lot better.

I'm sure that this is the way we need to follow. If we can do painless releases then we will be able to deliver software with less bugs faster. Maybe even weekly bugfix releases will be possible.

Delivering

After creating release we need to deliver application to end users. Fortunately most Linux distributions take care of applying updates automatically. So we are left with Windows and MacOS. For first iteration we will provide link to installer file in New Version dialog or notification. In second, we will automatically download it and run installer from Kadu (after user accepts it, of course).

Chrome has ideal solution for that. Updates are silent and very small (Google is using very clever executable diff files) and you are never ever informed about update - it just happens when browser is restarted. I don't think that we will be able to get that far with our manpower. But who knows :)

Time?

There are lot of things in Kadu that must be done. But this is base for them all. If we cant make releases then nobody will be able to use our new features and new bugfixes. So this task will be done as part of 1.0 release. Be ready.

Thursday, 4 April 2013

Maintainer needed for libgadu

Unfortunately Wojtek Kaniewski no longer has any time to take care of libgadu - the library that was the very base of Kadu since it earliest days. We are very grateful for all the hard work that he was doing over last 12 years and we wish him more spare time and good luck!

Now libgadu needs another maintainer who will be able to update it to the newest GG protocol with remote history support and other nice features. I don't know yet if we will be able to handle that, so if you want to help open source community - do not hesitate and take this project under your cover.

Sunday, 24 March 2013

Keeping up promises

During last years we've made a lot of promises to our users. We have promised features, less bugs and regular frequent releases. As it turns out it was impossible to keep up these promises. Only a small percent of features were added and our release schedule is already broken.

From the other side we've added what was most important to our users: new, searchable history window with updated backend, more stability and more usability. And some other, smaller things: proxy configuration, roster with conferences, MUC for XMPP. We've released 4 big version almost on time. That is not bad for a small group of developer working part time on a best Polish Open Source project.

Too much optimism

The source of problem seems obvious: we overestimate our capabilities and underestimate problems. Every task always takes more time that we think it will. We set our targets too high: our road map had a few or even a dozen of big features per release.

The other issues are more of a personal kind. We have much less time to work on Kadu (jobs, studies and so on). We hope to resolve this one by finishing studies and leaving our jobs ;)

Solution

Solution is very simple: start underpromising and overdelivering. No more than one big feature per release.

At first I've removed almost every feature that was marked 1.0 but OTR support. Then I've removed all bugs that are not regressions or crashes. This means 1.0 will not be as awesome as we would like it to, but it will be released soon.

Next thing I've changed was removing RoadMap from wiki. We don't need two separate lists of things to do. Looking at them was very demotivating (oh, how many things to do, and we still haven't even done 15% of this...). Go and read Joel's entry on software inventory - this is pure truth.

This does not mean that only one feature will be done and only crashes and regressions will be fixed. Probably we will do more. But we won't promise this.

Saturday, 16 March 2013

Status update

We are not dead

I know that for lat few months (almost half a year) Kadu repository looked abandoned. That was partially true. Our team was really busy with real life in this period. Some of as are working half-time while continuing education. Others are working over-full time. But fortunately this will change soon (at least for me) and from May I'll be spending much more time working on Open Source.

1.0 release

1.0 release is planned this year. Last feature that will be added to it is OTR support. It is going quite well already.

It does not mean that Kadu is complete, done or bug-free. It only means that we are confident that it is mature enough to be used everyday by everyone. We will continue to work on improving it after the release and making our users happier.

Please be patient :)