I work online. If I can’t get on the network I can’t work, cannot synchronize my repositories, cannot read email, check the issues/tasks/defects left for me to clear up etc.
But I also work away from the office, behind firewalls that do not let me do VPN, on the train without any network connection etc. As far as code is concerned, there is no problem. Most source control systems will allow you to work offline and synchronise changes after you come back online. What about all the other collaboration tools though?
We use wikis a lot, it’s a very handy tool to develop documentation for a project and keeping it up-to-date without much effort. There’s a lot of knowledge in the project wiki, knowledge that is not available when you are offline.
We also use Trac as an issue tracking system. Same problem there: The tickets are not available offline.
This has been a major hurdle in the adoption of the wiki and the issue tracking system by my team. There’s a lot of travelling involved, a lot of coordination meetings with the client, time where the people are essentially “offline” in the sense that they do not have access to the corporate network.
Which means that issues raised during a meeting will not immediately be recorded in Trac.
Which means that solutions for problems that are described by the developer team in wiki are not available to the project manager when in a meeting.
Both problems either generate more work (wiki documentation has to become Word documents and changes must be then manually recorded), or lead to the disuse of the tool (meeting protocols become todo lists with issues that are never entered in Trac etc.)
I want an offline-capable wiki and issue tracking tool. And by saying offline-capable I mean a tool that will allow someone to make local changes that can be then synchronised with the “master” server.
So I have a complete copy of the project wiki in my drive and can, i.e. in the meeting, read but also edit pages. When I get back to the corporate intranet, I use my nifty tool and enter the changes in the master wiki also updating my local copy with any changes done in my absence.
Same thing with the issue tracking system. I can enter new tickets locally and then update the server with my new tickets.
Essentially I want a perfect replica of the wiki and issue tracking system running on my laptop.
This more or less requires the backend for the system to have some kind of source control capabilities, to detect conflicts, produce diffs and do merges.
I imagine a combination of subversion backend with a mini webserver plus some modified scripts for the local replica.
The master installation commits directly after every change done online, while the local replica only makes changes in the local working copy.
In case of conflict the master always wins, but we make sure that the master is always up-to-date (a post-commit script will take care of this).
Updating the replica means just updating the working copy.
Getting the local changes online means commiting the changes made in the working copy.
There are problems, especially using this scheme for issues, where one wants to have numbers for the tickets and conflicts can arise very easily. A workaround would be to assign temporary ids locally, which then get resolved on commiting (i.e. a pre-commit hook script can do this). Also keeping the master always updated can create situations where a user sees the “somebody changed the page” more often.
A challenge is to set this up so that it requires the minimum intervention by the user.
It needs to be easy to setup. Ideally just a check out of the working copy from the server should give you a ready working replica.
Commiting should be a one step process in the no conflict case and also callable from the replica’s application instance. It would be something like update/resolve possible conflicts/commit.
Also commit logs should be generated automatically from the content of the edited page.
Another challenge is the design of the repository. For a wiki going back to the format used by the original Qwiki (each page a text file) would be the easiest way.
Continue Reading…
Posted by Vassilis Rizopoulos on Sep 30, 2006
I have in the past made a lot of fuss about automation of builds and tests in the projects I was in, making life hell for the various environment/tool managers. Making the life of the people that support your development hell is not particularly wise, so after a while I tend to present them with my own solutions. This means that initially I did a lot of extra work, but after a few projects the expertise begins to show and techniques and tools are readily available.
So I slipped in the role of tool and test manager for the current project mostly because of my insistence in past projects for infrastructure, regular testing reports, regressive testing and because of a lot of (pardon my saying) bitching.
More the “since you want it so much, you do it and shut up” reaction than anything else.
The current project is ending and after a long time I can say that it’s a project that has more or less finished within the bounds set for it.
From a business management point of view this makes the project a success and unarguably it is a success with the client as well, since we go right back in for v2.
It’s usually right at the end though where the big surprises spring at you and derail all budget and timing considerations.
I am a firm believer in the concepts of test-first coding, continuous integration, automatic and regressive testing and generally letting the computer do the work for you. Implementing such practices in floundering projects has saved my work (and the project) in more than one occasion.
This time I got to do it more or less from the start and it showed. Let me tell you, it’s an immense satisfaction to be proven right.
But the problems never end (good, otherwise I would be out of a job). Getting into the (development) environment management role means setting up a whole lot of nice toys: source control software, team collaboration software (mailing lists, wiki, project portal) etc.
These toys are just that: toys. You can play with them, you can dream about the possibilities but if your friends (see project team) don’t want to play along, then you’ll quickly find something else to do.
Most of the day-to-day communication in my project team is based on emails and phone calls (when not accross the desk). This means that when a build breaks, a bug is found or a test does not run, you get at best an email with a list of things to do and some explanations.
It’s fast, very efficient in the short run and it does not require more than your email program.
The problem with email is that it gets lost. It gets lost under the tons of emails that accumulate during the day/week. It gets lost because you will always erase some of the emails just to unclutter the folders from the tons of emails. It gets lost because you will just not read it.
It also has a very nasty side effect: Many people (myself included) will just fire a question or a todo list in the projectsphere if they are not sure who exactly is the expert/responsible. This gets you the quicker response and it’s how developer mailing lists for all open source projects work. Only, in the project everyone has commit rights. Without an issue tracking system to support this process though you easily get in situations where more than one person will take it upon himself to fix the issue.
I just hate spending half an hour fixing something only to get a source control conflict because someone just did the same job and commited before me.
Another nasty effect is that emails are transient. Nobody archives emails (well, not entirely true, but we are not talking about me or developer mailing lists here).
This means no effective project issue history. And it comes back to bite you at various points of the project.
The advantages of an issue tracking system are well known to those with some years in this business. Supporting the development and testing with such a system is for me as essential as automating your tests (and then you automate the process of evaluation of the automatic tests so that you can generate issues automatically. Like I said, let the computer do the work). But there is more.
There is something Philippe Kruchten said in a workshop I was lucky not to miss due to workload: Track the decisions, not just the solution. It is very important to track the alternative solutions and the decision and reasons for not using them. You cannot do this with email.
And you cannot do it if the team will not embrace the idea of generating history for the project.
There must be a medium where the information is accessible by all (access rights permiting) so that it stays with the project and not the person.
And it must be dead easy to use, otherwise everybody will revert back to sending emails.
This last point is where the whole breaks. You can find issue tracking systems that are really simple to use (Trac for example), but the whole concept is based on tickets/bugs/issues, things that once closed dissapear. Tracking decisions should be like the sticky threads in forums: always visible once set, quickly reachable by everyone so that you can also check if a decision has been made for something (“find the holes”).
You should be able to link them to bug reports in order to tie in the history behind the decision and other cool tricks. I love Trac’s wiki interpretation of ticket comments and commit logs along with the easy linking between revisions and tickets. But this is just added sugar, the basic idea is keeping some items visible even after the issue has been cleared.
And the positives come quickly: New members do not just get the “it’s the way it is and that’s how you should do it” treatment (otherwise known as the “Because!” treatment for the answer you get when you ask “Why?”). There is a clear record of the logic behind the current methodology in the project.
This also means that obsolete decisions that have established tiring and complex procedures are not lost in the mists of the project past. Everybody should know why something is done. There are a lot of clever minds out there and somebody might always have a better solution than the original and help you get rid of cruft.
Now, how do we go about adding such a feature to say Collaboa?
Posted by Vassilis Rizopoulos on Sep 15, 2006