Hobby-hacking Eric



It's that time of year again, the one where all submission deadlines converge. I've been putting in many hours at work lately trying to get something semi-publishable on time. Strangely, I've gotten into this nice habit of going to bed before 2h and getting to work around 10h. I'm loving it! More please! I've got this fantasy of me getting up at 7h, having a good breakfast, reading some articles for work, and then making my way calm and happy because I had made the effort to nourish my scientific brain. I dunno how long this will last, or why I seem to think that getting up earlier is better.

Anyway, been working on the PatchApply stuff for bug 154. Got the flag in, now I just have to see to all the stuff that calls an apply function. Not very clear what's going on with apply_to_slurpy though...


Repository diagrams

I just uploaded 9 darcs repository diagrams for the darcs model chapter of the wikibook. I'm not sure these are really useful, that they actually help communicate the idea behind darcs, but it was worth a shot. Just taking a break from all this patch theory stuff, doing something easy for a change. Got a friend at work that seems to get confused by all the darcs commands, so maybe a little visual support could be useful. Or it could just add to the confusion.


patch theory intro

Spent this afternoon finishing off the rewrite of the patch theory chapter. Like it a lot better. There's still a bit to polish off regarding the definition of a merge, and the commutation with inverse property, but it's definitely progress. Don't know if I have the energy to work on the other two chapters. Probably charge the mental batteries for a while, go work on something else. We'll have to see.


Constructive patch theory

I have not only been procrastinating on the Understanding darcs rewrite but have been a total flake this week on darcs and life in general.

Think I was having a somewhat bad couple of weeks and have been self-medicating by picking up a short freeciv addiction. I hate freeciv. I don't know what I can't shake it. It is something I do not find enjoyable, but every once in a while, I get this inexplicable craving, and well, you know how it goes. Lost sleep. Bad dietary habits. And all for a stupid game you've played over and over again (each time badly) with no surprises and no change in strategy. Anyway, I'm glad that's done with. Geez, even watching Firefly was better than playing freeciv.

Right... darcs. I've finally started the rewrite that I had been promising Arjan. Rewriting is really good for you. Forces you to look at crufty old bits of text and say to yourself, "waitasecond, that doesn't make one bit of sense!''. There's still a couple of holes in the introductory stuff. Need to write the complex undo bit, and reexplain what a context is, because I think the way I have it now is probably flat out wrong.

Holy shit. Did it just become 3h? But... is today leap forward day? It is indeed! Gah! And all this time I was being impressed by myself for being tired before 2h and wanting to call it an early night. Foiled!

Oh, and uh, Juliusz accepted the ssh patches today, along with the diff command stuff. Glad that's off my plate and I can move on to other stuff, like the wikibook.


At last, ssh multiplexing.

Patches sent to the list.

Well, glad I got that done. Hope it actually works as advertised. Unfortunately, this is not using Ganesh's idea, but I guess I'll let somebody more competent take care of that. This easier solution only works for people who have a fancy new ControlMaster-enabled ssh. It is nice not to have to type my password a whole bunch of times.


Whilst waiting for my tests to complete (I use record --no-test, followed by darcs trackdown when I'm almost ready to send), I took the opportunity to freshen my --diff-command patch and the refactoring stuff. Yeah, I know Daan's already sent his --external-diff one, but mine handles something which is perculiar to Macs, that being opendiff. The thing about opendiff aka FileMerge is that it quits immediately, leaving a subprocess on chew on the diff. But if darcs automatically cleans things up after opendiff exits, then FileMerge gets confused. The solution, at least, the one I borrowed from --external-merge, is to make the user press a key to continue. This way, we don't do the temp file cleanup until we really have to. The other thing is that my version of the patch is pickier about keeping things refactored. Oh well. I guess I don't really care either way, since I don't use --external-merge or --external-diff. I just wanted to get rid of my patches. Had been sitting around since January. What had happened was that Daan asked if the feature was there, I volunteered to implement it, sent it off, Juliusz got busy, and I forgot to tell Daan that there was a patch he could play with. So he decided to go implement it himself, and now we have two patches that do the same thing :-)


ssh control master

Made one useful bit of progress in the ssh control master stuff.

Before, I was having trouble with getting the control master running and out of the way. I didn't really know how to launch stuff in the background, so I basically forked the thing and polled the control master every 3 seconds to see if it was connected or not. But that was a really fragile setup, and sometimes, if I didn't type my password fast enough, I wouldn't be able to get a connection.

Tonight I have found a much simpler way to get the CM running: I just use the -f flag to make it go to background right before running the command, that is, only after it has completed the connection. This way, I don't need wait around polling, to see if the CM is running. This is terribly embarrasing how much mysterious all this process stuff is to -- seems shameful to call myself a computer scientist and not at least have some basic notions of how operating systems work. I built a toy OS for school once, but that was a really long time ago. Anyway, it does seem to work. All I know is that the exec function is a piece of darcs-internal code that makes some FFI call to a thing called smart_wait. And I guess smart_wait is happy to return if the thing goes to background or something like that.

Anyway, I was scratching my head for a while as to why I could get darcs put to work, but not darcs get. Turns out that the -S flag to scp means something else than it does for ssh. But no fear, we can use -o ControlMaster instead. Ooh tired. Will have to save this for tomorrow. Something to look forward to. Killing this project at last.

cvs keywords

A self-described newbie posted a message on darcs-users asking if there was any CVS keyword type functionality. Not something I care very much about, but I did find it kind of useful back in the dark days of CVS. Maybe it could also be useful to have in darcs. Of course, there are reasons why this kind of thing doesn't make any sense for us -- we don't do versions -- but maybe something approximative might be good enough for everyday needs.


back to darcs life

Have finished watching Firefly. Enjoyed it very much, but I am quite happy to be free of the shackles of mindless albeit high quality entertainment. So, what's on the plate?

Understanding darcs

Arjan had some bug fixes for the diagrams I made, basically a label being on the wrong transition, which makes things just wrong (I was rather uneasy about that, glad he picked up on that too). He'd also prefer a more constructive approach to explaining things. At first, I was rather hesitant, because I thought this would be mean backing away from the focus-on-a-concrete-story and explain-things-as-you-need-them approach, but then he showed me that the story could easily be extended with some concrete, uncontrived examples for the simple stuff. That is, you can at times want to commute patches outside of a merge situation. So I was happier with that.


Met a happy customer of the TracOnDarcs howto on #darcs. That was gratifying. Not the thanks, of course, besides, Lele deserves the lion's share of that, but it's always a big boost to your ego to see that something you made was genuinely useful to somebody, even if that consists only of an instruction manual. That's the biggest compliment you can pay anybody in the free-content/free-software universe: use their stuff.


no darcs for a while

Have not been working on any darcs stuff lately, and probably not till the end of this week. I, uh... started watching Serenity and season 1 of Firefly.


early to bed

Alright. I left work early today because I was feeling really tired... gee wonder why.

Have a friend who is currently learning Haskell. He asked me a question which ultimately had to do with Haskell and its use of layout, so after answering his question, I went ahead of wrote up what I knew in a new wikibooks module on Haskell indentation. What's funny is that I've been haskelling for two years now, and I still don't really know what the layout rules are. So I don't have all the elements I need to complete this page, but it's good way to learn. Would be useful to know for sure what I can and cannot do, instead of just using intuition.

Other than that, I also wrote up the results of today's discussion with Arjan. Think tomorrow I'll work on coding stuff instead. Need to let the conflictors stuff digest for a while.


posting from work

Crap! What happened to my discpline of keeping work and play neatly separated? Not that I completely believe it in, though. I have learned a great of Haskell by working on darcs, much more than if I had just continued plodding on with GenI (work).

Anyway, I just felt the need to rejoice. After last night's (this morning's) wikibooks debacle, I now understand ordinary conflictors! Not the complicated ones, mind you, no middle anything, just the simple stuff.

14:24 [kowey] could i check something with you on conflictors?
14:24 [arjanb] sure
14:24 [kowey] i've worked this out backwards, and want to make sure i'm not wrong, here
14:25 [kowey] say A and B conflict (this is the beer and pasta example in the wikibook)
14:25 [kowey] we want to pull A into B's side...
14:25 [kowey] so we want to calculate some A1 that we can apply like BA1
14:25 [kowey] so far so good?
14:25 [arjanb] yes
14:26 [kowey] in order to do this, we have to calculate A^-1
14:26 [kowey] and commute that with B to get (B1 A1^-1)
14:26 [kowey] so, in the beer and pasta example
14:26 [kowey] A is "add beer to the shopping list"
14:27 [kowey] A^-1 is "remove beer from the shopping list"
14:27 [kowey] since A^-1 and B conflict; we have to use forced commutation
14:28 [kowey] and when you do a forced commutation, the effect of A1^-1 is exactly the same as the effect of B
14:28 [kowey] so A1^-1 says "add pasta to the shopping list"
14:28 [kowey] and when you take the inverse of that, you get "remove pasta from the shopping list"
14:28 [kowey] which you can then apply to B to cancel the pasta out
14:29 [kowey] is that right?
14:29 [arjanb] yes though i prefer to do it the other way around
14:29 [kowey] you mean, pull B into A or something else?
14:31 [arjanb] make first a sequence B B^-1 A then commute to B A1 (B1^-1)
14:31 [kowey] that sounds clearer

Arjan made a very good point that permutivity is a property of all patches, not just conflictors. After some discussion, we came to the conclusion that it would be useful to have an intermediary chapter for formalising patch theory. We're going to put stuff in it like: symmetry of merging, what happens when you have N non-conflicting patches, and the stuff that Ganesh is working on.

Argh! How did I get here again? 4h30... exhausted, muddled thinking at home, muddle thinking at work. Must put a stop to this nonsense. Spent some time working on the ssh stuff, now that Juliusz has applied my latest salvo of patches. It compiles, but doesn't do what I want. While waiting for it to compile, I decided to continue working on the Understanding darcs wikibook... was making great progress, things were starting to make perfect sense. But then, I think it was somewhere around 3h30, I realised I had gotten everything wrong. MRRRRHH. After I had even bugged someone on IRC to go guinea pig the new text.


this is getting ridiculous

4h30 on a frickin workday! And twice in a row too, but it's so hard to stop when things finally start working. After all my refactoring of SelectChanges, I think I've fixed the graphical patch selection for obliterate. The basic idea is that the gui and text selection modules should share as much code as conveniently possible. Anyway, now that this scraiest looking bug is fixed, I have resubmitted my wxDarcs stuff. It's still broken in a couple of known ways and probably a million other unknown ones, but it's important for me to get it out the door so that people can start hacking on it, and so that I can move on to other stuff like ssh and Understanding darcs. We all know it's broken. Nobody's going to claim that the GUI is working, but the code should at least be there for people to play with.


refactored SelectChanges.lhs

I'm probably being a little quick to judge here, but the function with_any_selected_change in SelectChanges.lhs is some serious spaghetti. I'll bet it started out all nice and simple, and we just kept adding functionality to it one tiny feature at time, and us the darcs developers being so used to the code, never noticing that things were getting out of hand.

It took my all Sunday staring at it, making notes on the wiki, and finally tearing it to pieces and putting it back together (just rearranging code) to get a clearer picture what it was doing. I hope my refactor is correct, that it doesn't introduce any weird bugs into the code. One thing I'm slighly worried about is that I took out seemingly pointless seq, something that looks like other_ps `seq` ps' where, if I understand correctly, other_ps is defined to be []. Anyway, so I seperated the notion of counting patches from that of simply deciding which patches to consider. This way I can reuse the latter function in the gui selector. The code compiles. I only hope it does what it's supposed to.


absolute file stuff / mutt / SelectChanges

Sent off my absolute file patches this morning. I hope they do the right thing. I don't know, but I don't think it want to play with this anymore. Hopefully a third person can look at this because Juliusz is nervous about it and so am I... would be useful if somebody else came in and said, "yes, this is the way to go" or "no! are you crazy?"

Spent most of my hack day (i.e. the part where I wasn't at the gym or grocery shopping) switching to mutt on my laptop. I've been using mutt for a long time, but only at one account at a time, most recently at work. Spent some time splitting my configuration up and automating the relationship between my aliases and save hooks: makefiles and sed are useful for that. Whowouldathunkit. Using make to automatically generate parts of your muttrc. Wish I had thought of it earlier. Anyway, pleased with the switch. Apple Mail infuriates me in many small ways. I also switched to zsh today. Don't know why. Random impulse.

Other than that, I'm slowly starting to understand how SelectChanges works. It's kind of interesting, the frontier between the outer and middle layers of darcs (*). Also very gratifying. I like understanding stuff. You just have to stare at it long enough, and poke at it at the right angles, and eventually it opens up. The trick for SelectChanges was to grep with_selected in the code and write it up as a table on the wiki. I still don't actually know how it all works, but I've got a clearer picture what questions I need to be asking next.

I tend to see darcs in three layers... this is my naive view so far: you got an outer layer which consists of the commands and all their helper code, including stuff to invoke sendmail, etc; next you have the middle layer which deals with the various patch types and how they commute; and finally you have the inner layer, which is the core patch theory stuff. It's probably very hard to distinguish beteen the middle and inner layers for example.


absolute directories

Two things: got the absolute directories tests running, although I had to hack them up a bit to fix what seems to be darcs bitrot and weird perl. Hope my modifications aren't actually breaking the tests.

Also, I've been going over my patch a little more thoroughly, and have found a few bugs. End result is that patch deux is smaller and cleaner. I'm still not entirely sure if it's correct though, and am hoping for some review from fellow developers. Here's the issues with my patch that I've fixed:
  1. Changes.lhs - was not accounting for the fact that repodir could be relative path; fixed by generalising fix_filepaths_here to fix_filepaths_wrt somerepo, where fix_filepaths_wrt has some extra smarts for detecting what kind of path the current repository is
  2. DarcsCommands.lhs - forgot that ghc 6.2.2 does not have canonicalizePath; tough luck, got rid of code which canonicalizes all the absolute paths. Disadvantage is that now, absolute paths detection doesn't always work. Advantage is that the code is a bit tidier and the patch is smaller
  3. Apply.lhs - was needlessly performing the absolute path conversion
  4. FilePathUtils.lhs - was not producing the right error message, and was not strict enough on what kind of repository paths it expects

Sigh... these aren't fixes to darcs, mind you. They're fixes to my patch, which were supposed to fix darcs.

Ok, wow, it's almost 4 in the morning. I keeping telling myself I'll go to bed earlier, yet I keep getting sucked in to stuff like this...


wxDarcs, absolute paths and conflictors

Wow, I've been really tired lately. Ought to be going to bed earlier or something.


Right, so one little bit of good news to lift my spirits: I managed to fix a couple of layout bugs today. For the OS X Tiger layout issue, all I had to do was pack my splitter window inside its own panel, and everything was ok. As a more general issue, I finally figured out how to force one of my buttons to the very bottom of screen (nitpicky that way) -- I just pack all the other buttons, the ones that go on top into their own column. Gui programming can be a pain that.

I also know why the obliterate gui was behaving funny - it seems to be more or less obliterate the reverse of the patches you tell it to. So if you've go 3 patches, foo bar and baz, and you tell it to obliterate foo, it will end up obliterating baz. It seems like gui_select was never implemeneted to deal with is_reverse, which is probably why David (i assume) didn't allow Unrecord/Unpull/Obliterate to accept --gui in the first place. So my new job if I want to get obliterating right is to figure out how to deal with selecting patches in reverse. It's kinda funny -- all this first middle last stuff is vaguely reminiscent of how Vijay-Shanker and Weir 1993 represent chart items in their parsing algorithm (more related to eric job stuff). Of course, there is probably zero relationship between the two except for this notion of head, middle tail... but who knows?

absolute paths

Anyway, after looking at that stuff, I got discouraged and decided to tackle some of my absolute path actions instead. Applied my patch -- very few conflicts -- and cleaned up some of my own code. Now I need to test it out, and sit down and figure out where it could go wrong. I hope the code is right though, because I would really like to get rid of it as soon as possible, and move on to other stuff like Understanding darcs.


Starting to read the mailing list in chronological order. I had forgotten that I did not in fact download the entire archives. Wonder if there's any way I can convert this thing into something a lot friendlier for mass-reading. Hopefully the combination of this and my FOSDEM notes will allow me to produce a useful introduction.


Juliusz seems at last to have had a pocket of free time to apply a few patches. Unfortunately, this breaks the wxDarcs (easy to fix). I'm kinda hesitant about sending these patches in. The darcs gui is really horribly horribly broken, the way I have it set up. Consider, for instance, that it obliterates the wrong patches! Sigh.

There's also my patch for fixing the absolute directory stuff which I need to think about and confirm.

Man... Been a long day. Time for bed.


darcs developers at fosdem 2006!

Unfortunately, I didn't think to do this on Saturday when there were more of us, so sadly, Juliusz and Andres are missing from this little video:

In the video, you get David, Arjan, Ganesh and me (Eric). You also get members of the future darcs users group, Dmitry, Alexander, Michael, and Joseph. Although, I might add Dmitry has expressed a certain interest in formalising patch theory as a sequent calculus, or something like that, so maybe the darcs developers will have another logician on their side?

Oh, yes and I did mention a videoblog somewhere. This is being cross-posted to a collective video blog shared by a small group of friends from high school, gfdjax.

understanding darcs

I was supposed to spend my day off doing laundry, getting my hair cut, and running a million little errands, but instead I spent it writing my patch theory introduction in Understanding darcs. Oh well, at least I now know how merging works in the very simplest non-conflicting cases. Typical pattern for me. If I want to understand how something works, the best way for me to go about it is to write something in wikibooks about it.