The Martian Chronicles

Fighting entropy (Dealing with CogVM branches)

Aligning This days I continue working with the PharoVM. My goal is to produce a SpurVM made with Pharo and to produce a Pharo image who works with SpurVM. It is complicated, because a lot of other tasks need to be done before even start to make the build.

My first task in that direction was to move forward VM development, to Pharo3 (it was based in the older version, Pharo2), as you can see in this post.

And this is not enough, of course. Now I need to use latest vm sources, and that's an interesting problem (to use an euphemism).

Why? Because during last years, our sources have been diverging increasingly with their counterpart on CogVM. While this source changes does not means a fork (or a "complete fork", since we careful try to keep always same code base where possible, specially in VM), the fact that we use a different branch than Eliot causes a significant and always increasing pain.

When you work on a branch durning a time, weird things starts to happen:

1. You apply a seamesly innocent refactor, or a source code format.

Then, small changes start to accumulate. The result is a lot of difference that actually does not introduces a real improvement.

That's noise and should happen.

Or should not be allowed.

The reason is that after a while doing this, almost all your source is formatted different than the parent branch. Honestly, without any real improvement in VM, the "better looking" source does not pay the work of merge it everytime.

2. You change your sources for using Pharo APIs

But Eliot branch is made with Squeak. That's a complicated problem.

The core problem is that we do not have FileDirectory package anymore. Instead, we use FileSystem.

So we changed sources to reflect that, while what we should do is to propose an abstraction layer and ask to adopt it in all development branches. But well, that takes time and we were in a hurry.

The result is a number of methods who behaves the same, but uses different file API.

Other small differences between Pharo and Squeak also causes some minor changes:

  • Pharo does not concatenates strings and number objects directly, so thinks like Transcript show: 'something', 43 do not work, you have to make the conversion manually.
  • Pharo and Squeak processor API are slightly different: Project uiProcess does not exists (we do not have Project). So we use: UIManager default uiProcess
  • Transcript uses different definitions and then API is different.
  • Class protocol API is different (and is used to filter some methods in code generation)

That introduces some (minor) method changes.

3. You fix a particular problem for a platform you use but other branches not.

And since those changes are not syncronized, then you have methods with different implementation. That happens just couple of times, AFAIK. But in a some very important place: image R/W and input event fetchers.

  • We use a different read macro, that allow us to read by page of memory in the case of iOS platform. In all other cases it behaves exactly the same.
  • We introduced complex event types (to be able to get things like touch events, etc.).

4. You introduce changes to the VM, to be used exclusively in Pharo.

Because that's because we did the branch after all, isn't? But looking closer, our changes are not so much (they are few, but some important ones).

  • Native boost support is the most important change we introduce in our sources. And even if it is vital for us, in practice is an override of three methods, and the adding of a couple more.
  • Also, our WeakRegistry was not working properly (at least for us, I don't know how it works in Squeak and Newspeak), so we changed the finalization process to enhance the weak finalizer. Again, is just two overrides and one extra method.
  • We changed the way external semaphores are handled. We made it to allow more than a maximum fixed number (default is 256) of them opened. Now it allows an indefinite number of them which is very useful in certain scenarios.
  • Finally, we introduced ephemerons. This is a lot more complex in source than I explained before... but turns out that we never used it. And SpurVM will introduce a different implementation, so we can drop it for this time.

5. You make changes to VM plugins

We introduced a lot of changes in the plugins.

Some changes are important, some others are not. For example we introduced posix permissions in FilePlugin and we replaced platform dependent code with posix code (available by default in linux and mac, and through mingw in windows).

Like FilePlugin, we made changes during the years in many other plugins. I can think now in SerialPlugin, SSLPlugin, etc., etc. But, looking at the real amount of differences between our branch and the other, I realise now there are not so many.

What actually happened is that many of our changes are in the platform specific files (in mac we use a complete different implementation based on Cocoa, while Eliot's uses Carbon). And some of them are also integrated in both branches.

Aligning paths

Clearly, there are some important changes that we want to keep, but after all... they are not so many. I worked a lot to restore our sources to Eliot conventions and refactors.

I would like to remove more incompatibilities soon (like the ones made by different File APIs... I can get rid of that by using a compatibility layer).

Some others I would like to see them included in all branches (FilePlugin for example is an improvement and not Pharo specific, I do not see any reason why it shouldn't be in the common sources).

And some others... well, we will still need to live with them. But they are just a few changes, certainly a lot easier to track.

If we are careful on the way we threat sources in the future

Posted by Esteban at 11 June 2014, 8:11 pm link