In the past few years Axiom development has been a process where all code changes happened on my local machine. I interacted with people on a 1-on-1 basis, applied new code, managed local "broken" development branches, and built new features. Once these features were working they were tested on all the platforms I had available to me in my local compile farm in my basement. Clearly this process has flaws. The main problem is that it does not scale. A second problem is that all of the parallel changes are wrapped into one big development swamp on my local machine (long time readers will recognize axiomgnu/new which still exists here). So, in order to develop a process which scales I've been looking at the way Linus Torvolds does kernel development to try to understand how to coordinate many people doing development in parallel. The paramount problem is how to release quality software that is stable and well tested. It's not obvious to anyone but there is a long path of "process" steps I have been taking to try to make sure that Axion "just works". Now that the process is more open we will all end up following that process. So I feel the need to explain the "development model" so everyone is on the same page. Feel free to give me feedback as this is a cooperative effort. ===================================================================== Remember that the goal is to release a quality piece of software that is well tested, well documented, and "just works". ===================================================================== At the present time there are about 1/2 dozen tasks that I've "exposed" to the world (see http://arch.axiom-developer.org). I have a dozen more in the local code swamp that need to be exposed. They will show up as time and interest permits. Suppose a developer shows up with a problem, say porting axiom to the Itanium. Step 1 of the process is that a new "arch branch" will get created. (Likely the branch will be named axiom--itanium--1). This branch will allow any developer interested in the problem to be able to read and modify the code at will. We can experiment with any changes we want in the "itanium" branch". Step 2 is the documentation phase. All changes that survive the struggle to get itanium to work need to be properly documented. This is my own mania, I realize, but it's part of the 30 year horizon nonsense. We're writing code that our children will have to change. Thus all changes need explanations that will make sense in the future. Step 3 is the testing phase. We have to get the system to build and run on a couple Itanium platforms to ensure that it works somewhere other than on our desk. At this point we have a completed "axiom--itanium--1" branch and have solved the problem of getting axion to run on the Itanium.... ------------------------------------------------------------------- Step 4 is the integration phase. The problem to be solved here is to merge the axiom--itanium--1 branch into the axiom--main--1 branch. There are two issues; integration and compile-farm. First, the axiom--main--1 branch has moved. Other developers have been integrating their code, possibly changing the same file we changed. So we need to merge our code into the axiom--main--1 branch. Clearly this could be done automatically but I do it by hand so I can check each change. (Machines don't care about quality.) Step 5 is the compile farm (issue two) Second, once we've integrated our changes into the axiom--main--1 the build process for axiom--main--1 has to pass the "compile farm". That is, we need to build axiom on all of the different platforms and make sure that our new itanium hacks don't break the main build on other platforms. ------------------------------------------------------------------- Step 6 is the savannah CVS change At this point we've changed, documented, and tested a big new feature for axiom. Now we need to check in the changes to the savannah CVS. The changes are now going out to the world. For a lot of reasons this step is likely to happen months after starting step 1. The hard part is that developers are now going to see new features appear in the branches (e.g. graphics appears to work) but it seems to take forever to get it into the savannah CVS. I know this feeling as I've had various pieces of axiom running long before the world knew. People who knew things like graphics were working wanted it NOW but quality is much more important than speed when releasing features. Step 7 is the binary release Several of the platforms have binary-only releases. These need to be packaged from the CVS version so they match the published reality. Thus all of these binaries need to be recompiled and tested. Step 8 is the spreading-tarball problem. Several sites keep old tarballs of source around. This is convenient but problematic as I occasionally get bug reports from old tarballs which have been fixed in the CVS. ==================================================================== Arch? ARCH?! We don't need arch!!! ==================================================================== Why arch? Why not CVS? I've been using CVS for the past few years and now I've run into the same problem that the Linux kernel people solved. The problem is that CVS is unable to handle "changesets" and "branching" easily. EASILY. Arch can. Changesets are a group of changes to several files that are all handled as one big change. Thus if we make a fix that involves 10 files we can apply and retract the changes all at one time. This is technically possible with CVS if you carefully tag every file before applying the change. However there are thousands of files in axiom and mistakes are very hard to correct. Arch allows this to occur as one command. So CVS treats each file change as different and arch combines many semantically-related changes into one "changeset". Branching can also be done in CVS but Arch also handles this problem in a much more refined way. Since there will be many parallel development efforts the ability to branch easily is important. Arch just does it better. I realize that switching to Arch is a learning curve and none of us need yet another learning curve. But consider the fact that the kernel has switched fron CVS to BitKeeper, a proprietary piece of code for exactly the same reason; to manage the complexity better you need better tools. Worldwide group development is hard. So individually it will be painful but process-wise it will be easier. I'll provide cookbooks schemes to anyone who asks and also provide copies on the http://arch.axiom-developer.org page to minimize the learning and the pain. I ask for your patience as we open up the development process. There is no such thing as a simple job. Tim