koweycode

untangling a cabal install problem

2011-05-03T00:44:00.000+02:00

I sometimes have trouble translating abstract general explanations to my particular concrete cases. I hope that by sharing a very concrete situation I experienced, other users may recognise themselves and get unstuck on their own problems.

I finally untangled out a cabal install problem that's been bugging me for some time, almost driving me to use cabal-dev on all my packages (which seems like it might be a bit inconvenient)

So I have a fairly standard setup (at least, it was standard when I wrote this post), GHC 6.12.3 with the latest released Haskell Platform. I'm working on two packages, GenI and nltg-hillwalking simultaneously. Switching from one to the other is painful. When I try to install GenI typing "cabal install" results in this horribly disheartening sequence, where it installs random, haskell98, cpphs, haskell-src-exts, derive and finally GenI. If I then switch back to working on hillwalking, I then get this another discouraging sequence involving random (again!), QuickCheck, test-framework, ntlg-hillwalking. And going back to working on GenI, I go through the same pain again.

It took me a while to work out that the problem was just the interaction between these two packages. Having had a chance to chat about this with Duncan and Ian, I got a bit of a clue about what the problem might be. Indeed, when I ran "cabal install --dry-run -v2", this little bit of output caught my eye:

In order, the following would be installed:

random-1.0.0.3 (reinstall) changes: time-1.1.4 -> 1.1.2.4
haskell98-1.0.1.1 (reinstall)
cpphs-1.11 (new package)
haskell-src-exts-1.10.2 (new package)
derive-2.4.2 (new package)
GenI-0.21 (new package)

See that little arrow? It says that the reason random, the cause of all my heartache, is being reinstalled because of it wants to depend on an older version of time. Why on earth would it want to do that? ... Oh, because I told it to. Apparently, some past version of myself decided to put this dependency in GenI.cabal: time == 1.1.2.4

Oops!

I think the problem looks like this. GenI uses the derive package, which triggers a chain of dependencies all the way down to random and time. Unfortunately, GenI also directly depends on time but now we have an issue. I'm not entirely clear on why this causes a recompile as opposed to the more usual "this will likely cause an error" output (maybe the latter is only appropriate for direct dependencies, ie. if derive depended on time itself?).

By forcing GenI to use this old version of time, I was indirectly forcing it to install a version of the random package that depends on this old version. In doing so, I would clobber the version of the random package that QuickCheck uses.

Fixing the issue in GenI was relatively straightforward. Did I really need to be using such a constrained version of time? It turns out that time == 1.1.* works perfectly fine (taking advantage of the PVP promise of backwards compatibility in all A.B.* versions of a package). Just one little dependency and everything works a lot more smoothly.

So what did I learn from this?

take a deep breath - I think when I'm faced with these issues, I'm feeling really impatient to get on with my work. But solving the issue involves recognising just some silly little problem, which can be hard to do when I'm being impatient. So part of the trick is to defocus somehow and shift to poking mode.
use cabal install --dry-run -v2 and study the end part : what packages are we trying to install and why? The -v2 is important because it tells you why packages are being installed.
??? hunt for the offending dependency - for me this was a simple case of staring at GenI.cabal. What if GenI depended on some library which in turn depended on time-1.1.2.4? I guess the answer would lie in the list of packages that cabal-install says it would install. The dependency must lie *somewhere* in the chain.

If I understand correctly, this may actually an improvement over the pre GHC 6.12 days before the ABI hash was introduced. I don't actually know, but I could imagine there's something that'd make one random not-quite-compatible with the other, even if they're both version 1.0.0.3 and silently swapping one out for the other would cause subtle breakage. At least now, we know if something is wrong and we can fix it relatively easily by just reinstalling the missing package.

This dependency stuff must be really tricky! It looks like there may be some work that could make life better, for example, a Nix-like approach where both versions of random 1.0.0.3 could co-exist. But we should be glad in the meantime that Duncan et al have not torn their hair out yet. (Just think of the pre-Cabal-install days if it helps, life's much better now, isn't it?)

why darcs users care about consistency

2011-04-19T18:42:00.008+02:00

In the Darcs community, we've been discussing the recent blog posts saying that Git is inconsistent, that it cannot be made to be consistent.

With Darcs being the foil to Git for the purposes of this discussion, I thought it would be useful if I cleared up a few points, particularly this first one:

consistency is a usability issue

When people say they like Darcs, they don't generally talk about it having a beautiful or elegant theory. Instead, they talk about how easy and simple it is to use, about how they never really had to grapple with a learning curve or feel stupid for doing something wrong.

What makes Darcs so simple to use? Did it hit the right notes by accident or through David Roundy's good taste? Or is usability merely in the eye of the beholder? Some of these explanations may be true, but I think what lies at the heart of Darcs' usability is that it supports a very simple way of understanding a repository:

a darcs repository is a set of patches

This mental model may not be suitable for everybody, and in the long run Darcs may need to improve its support for history tracking. But if you want to understand why, for all its current shortcomings, people continue to use and develop Darcs, you must appreciate how refreshingly simple the set-of-patches mental model can be. As a Darcs user you are freed from a lot of the artefacts of worrying about commit order. Collaborating with people is just question of shuffling patches around, with no merge states, no rebases, way fewer spurious dependencies to worry about.

But simplicity is hard. In order to make this simple world view possible, Darcs has to guarantee a property that any ordering of patches allowed by Darcs commutation rules is equivalent. If Darcs gives you the option of skipping a patch, it has to work hard to make sure that if you include the patch later on, that the repository you get is equivalent. That's what the patch theory fuss is about. While it's useful that Darcs tends to attract purists and math geeks, we're really not engaged in the pursuit of some sort of ivory tower theoretical elegance for its own sake. Ultimately what we're after is usability.

A good user interface minimises work for the user, be it cognitive, memory or physical work. The joy of Darcs is being able to focus cognitive work on our real jobs, and not on babysitting version control systems. So when Russell O'Connor says that merges ought to be associative, he's not saying this to tick some sort of mathematical box, what I think he's really saying is as a Darcs user, he doesn't want to worry about the difference between pushing patches one at a time vs all in one go. Consistency is a usability issue.

darcs is imperfect

Darcs is very much a work in progress. Some users have felt let down by Darcs: whenever performance grew to be unacceptable for their repositories, when they hit one exponential merge too many, or when Darcs just plain did something wrong. Even our much vaunted usability has cracks at the edges, a confirmation prompt too many, an inconsistent flag set, a non-reversible operation or two.

I particularly want to make sure I'm very clear about this point:

darcs patch theory is incomplete

We still don't know how to cope with complicated conflicts. Moreover the implementation of our first two theories is somewhat buggy. Darcs copes well enough with most every day conflicts, but if a conflict gets hairy enough, Darcs will crash and emit a nasty message. This is one of the reasons why we don't recommend Darcs for large repositories.

Our version of "don't do that" is not to maintain long term feature branches without merging back to the trunk on a regular basis. This is not acceptable for bigger projects, but for smaller projects like Darcs itself, the trade-off between a simple user interface in the general case, and the occasional hairy conflict can be worth it. In the long run, we have to fix this. We are revising our patch theory again, this time taking a much more rigorous and systematic approach to the problem.

In the interim, we will be gaining some powerful new tools to help work around the problem, namely a new "darcs rebase" feature that will allow users to smooth away conflicts rather than letting them get out of hand. This will be a crucial bridging tool while we continue to attack the patch theory problem.

patch theory is simple at heart

I am in the awkward position of being a non-expert maintainer, having to defer a lot of thinking about software engineering and patch theory to the rest of the Darcs team. In a way, this is healthy for Darcs, because we have long suffered from an excess concentration of expertise. Inverting the pie so that you basically have the number one Darcs Fan as the maintainer is useful because it forces everybody else to break things down into words an Eric can understand.

The good news is that basic patch theory is one of these things an Eric can understand: patches have inverses and may sometimes be commuted. Just learning the core theory teaches you how merging and cherry picking works, why you can trust the set-of-patches abstraction and most importantly, how simple Darcs is. So we're not after some kind of magical AI here, nor are we trying to guess user intention. The things we do with patches are much more mechanical, systematically adjusting patches to context, one at a time, click-clack on the abacus until the merge is complete.

patch vs snapshot is not so important

We think it's important to continue working on Darcs because we are exploring territory that no other version control system is looking at - patch-based version control. That said, patches and snapshots are duals of each other. We think that things that Darcs can do are possible in snapshot based version control and we would be very interested to see work in that direction.

The secret to Darcs merging is that it replaces guesswork (fuzz factor) with history. A darcs patch only exists in the context of its predecessors, and if we want to apply a patch to a different context, we mechanically transform the patch to fit. We think this sort of history-aware merging could be implemented in Git. In fact, we would be excited to see somebody taking up the challenge. Git fans! How about stealing history-aware merging from us?

exponential merges still exist but there are fewer of them

We have developed two versions of patch theory. The second version avoids a lot of the common causes of exponential merge blowups, but it is still possible to trigger them. Recent Darcs repositories are created using version 2 of the theory. For compatibility's sake, repositories created before Darcs 2 came along tend to still be using version 1 of the theory (we only recommend converting if conflicts become a problem).

The most well-known remaining cause of blowups in theory 2 is the problem of "conflict fights" where one side of the conflict resolves the conflict and gets on with their life without propagating the resolution back to the other side. What tends to happen there is that we not only encounter the conflict again in the future, but we also conflict with the resolution!

So life is definitely better with Darcs 2. We've given the exponential merge problem a good knock on the head, but it's still staggering around and we're working our way to the finishing blow.

performance is improving

I think that when people complain about Darcs being slow, they're not talking about the exponential merge problem. They're mostly referring to day-to-day issues like the time it takes to check out a repository. Our recent focus has been to solve a lot of these pedestrian performance issues. For example, the upcoming Darcs 2.8 is like to use a new "packs" feature which makes it possible to fetch a repository in the form of two larger tarballs rather than thousands of little patch files. This makes a big difference!

Another improvement we hope to bring to Darcs 2.8 is the performance of the darcs annotate command (cf. git blame). Annotate has neglected for a while, and to make things better, we've basically reimplemented the command from scratch with more readable output to boot. As an example of something fixed along the way, one misfeature of the old annotate is that would work by applying all the patches relevant to a given file, building it up from the very beginning. But if you think about it, annotating a file is really about annotating its current state; we don't care about ancient history! So one of the Darcs hackers had the sort of idea that’s obvious in hindsight: rather than applying patches forwards from the beginning of history, we simply unapply them from the end. Much faster.

We're not yet trying to compete with Git when working on these performance issues. We admire the performance that Git can deliver and we agree that getting speed right is a usability issue (too slow and your user loses their train of thought). But we've been picking a lot of low hanging fruit lately, solving problems that make Darcs faster with very little cost. We hope you'll like the results!

practical QuickCheck revisited - separate testing hierarchy

2011-02-18T12:27:00.003+01:00

I'll begin this post with a quote from 2009-Eric:

This may go down as the kind of bad advice that "seemed like a good idea at the the time".

The advice in question was to "bake unit tests in". The basic idea was that whatever module you write should have its own testSuite function exposing unit tests for that particular module. The advantages were simplicity (no parallel test hierarchy), the ability to ship a binary with self-tests, and the ability to non-exported functions, helper code with a granularity that lends itself more to testing (easier to think of tests for them).

I was unconvinced by the counterargument that it was not a good idea to mix testing and business logic. To be clear, I did agree with the spirit of the advice -- I'm not about go around questioning the kind of wisdom a community gains by watching rockets blow up -- but I felt that I was not advocating any such mixing. All I wanted was to put my testing code in the same file as the business code, cordoned off in a testing section at the end of the file if you want without any sort of if-testing-mode-do-X logic. So I thought that the counterargument was right, but that it didn't apply to this particular context. (I'd be interested to see when/if I change my mind on this, maybe it leads to temptation to mix logic, which is bad.)

In any case, I don't need to change my mind on that particular point. Being the kind of person that only learns the hard way, I've found myself forced to divorce my test code from the business code after all. It's mainly a practical problem of dependencies (this was pointed out by Echo Nolan and Ivan Miljenovic). Forcing users to install QuickCheck and test-framework, when they probably don't care about testing, when they just see your module as yet another dependency on the the road to some other more pressing goal, is really a bit anti-social.

The problem isn't installing the package per se (it all happens automatically with cabal install), but dealing with package version dependencies. So GenI depends on test-framework 2.x and QuickCheck 1.2. What if I go away for a few years, stop hacking on GenI and in the meantime the rest of the world moves on to using QuickCheck 2.x and test-framework 3? What happens when they try to install GenI and cabal install needs to rebuild the random package, which then breaks QuickCheck-2.4 because it depends on random too. Headaches all around.

I think I can live with a separate hierarchy. Arguing with past-Eric a bit:

All the extra modules and what not are not that big a deal (and I could probably let myself go wrt imports, etc).
Who cares if there's an extra geni-test binary, which only gets enable with -ftest anyway?
Self tests, shmelf tests. Seriously, who is going to run that geni --test function anyway?
If I forget to cabal configure -ftest, I can always cabal configure again and build
If I'm really desperate to test some internal function, I could always export an alias like testingFoo for every foo I want to test, applying a sort of Pythonesque we're-all-grownups-here principle.
Also maybe forcing yourself to test only the exported functions, enforces a kind of general black-box thinking which is healthy if you're writing a library.

So, with apologies to Ivan for not understanding his rants 2 years ago; and also anyone that may have listened to 2009-Eric for any messes I got you and your users in, I'm retracting that particular bit of advice and separating my test hierarchies like a good boy. Let's see if 2013-Eric decides to post some kind of retraction retraction.

personal gitit wiki on MacOS X

2010-12-02T19:54:00.005+01:00

Here's a quick little recipe for using gitit as a personal wiki on MacOS X. I assume here you already have the wiki itself set up, and now you just want it to run automatically in the background whenever you log in. You can do this by using launchd.

Download net.johnmacfarlane.gitit.plist from this Gist
Replace the WorkingDirectory with the path to your personal wiki
Replace the last part of the PATH to include your cabal directory, and possibly something like /usr/local/bin or /opt/local/bin if you're using Git instead of Darcs
Save the file in ~/Library/LaunchAgents
Test it with launchctl load ~/Library/LaunchAgents, maybe using the Console application to search for logs should something go wrong.
Log out and log back in (or maybe even restart your computer if you want to be sure)

Helpful bits and pieces:

This MacGeekery article
launchd.plist man page
Property List Editor in Developer Tools (beats looking at XML)

Edit 2010-12-04: Fixed broken link

Early Career Researcher: the computer game

2010-09-22T10:34:00.006+02:00

Here's an idea for a computer game called Early Career Researcher. The simple version being a fairly mindless turn-based RPG-esque deal. Nothing earth shattering in terms of game mechanics, but perhaps an amusing toy.

You have

personal attributes (eg. writing, social skills, initiative)
inputs (eg. ideas, papers to review)
daily resources (eg. time, energy)
actions (eg. check email, write paper, write grant proposal, lab work [or some generic term for "actual" research leg work], take nap, go to pub)
outcomes (eg. paper accepted, grant awarded, contract extension)
light bulbs (XP)

The goal of the game is just to maximise light bulbs. The basic model is that every turn consists of a "day" (a day should take about 5-10 minutes to play). In each day, you can do any number of actions, but the kinds of actions are limited by the inputs and daily resources you have. For example, you could do write a paper, but in order to do so, you'd need a paper-topic resource to consume, not to mention time. Likewise, you could check your email and it may only take a few minutes, but it could also use up a lot of your energy. Actions may result in outcomes, but whether or not they do so depends on a combination of personal attributes and luck. For example, writing a paper may result in paper accepted, depending on writing skills, research-fu and the dice roll. Going to the pub (presumably chatting with colleagues) may result in Ideas depending on social skills and creativity and the dice roll. Outcomes generate inputs (eg. ideas) and Lightbulbs (XP). If you get enough XP to level up, you can use your lightbulbs to purchase personal attributes.

As the game develops it should become clearer that it's important to choose your actions wisely, and also to pay attention to the notion of balance. Spending all your time doing lab work or writing grant proposals may seem like a good idea, but if you fail to spend enough time in the pub or take sufficient naps, you may not generate sufficient idea resources to make very much progress. Or maybe if you're too lazy and spending all your time just trying to be inspired, you just don't make sufficient practical progress to get anywhere.

So if anybody wants to code this up as a little exercise...

hsgtd and friends 1: mutt inbox and actions

2010-03-28T17:50:00.015+02:00

I've been practising the methodology of Getting Things Done for over 4 years now, but I'm still not very good at it.

I hope to write a small serious of postings showing my current GTD state of the art. I hope it will be useful to somebody out there and that I will get some ideas on fine-tuning my approach.

Another hope I have is to reach out to technical people who are resisting "becoming more organised" because of the apparent overhead involved. I hope to demonstrate that you can actually get a lot of mileage out of a handful of shell scripts and simple practices (keeping all your mail in a single folder).

Ingredients

mutt - The appeal here is to have a mail client that is malleable and which can talk to 3rd-party software. So it doesn't necessarily have to be as old school as mutt, just scriptable and capable of playing with others.
hsgtd - a command line GTD tracker written in 351 lines of Haskell. Everything is stored in a simple text file

I also use mairix, xmonad and Unison, but these will likely only be relevant in future postings.

Background

In this first instalment, I would like to talk about how I deal with inbox triage. It's useful to know a little bit of GTD terminology for this.

Inbox - things which are not yet triaged. Practicing GTD is like using an issue tracker; you decouple triage from actions. One priority in GTD is to empty out the inbox by performing triage on all items. Working this way is efficient because you avoid looking at the same item or having the same thought about it (gee, I oughta...) twice. Things go in stages.
Next actions - One of the results from the triage process is a set of "next actions", concrete physical actions like, eg. call Bob 398-0811 to see if he wants that spare external disk drive

I use two different programs: mutt to view my inbox, and hsgtd to view my list of next actions. In this series of posts, I'll be exploring how mutt and hsgtd might talk to each other.

Inbox triage : from email to next actions

The most common source of next actions for me is my email, so it is very important for me to good integration between my hsgtd list and my email. In particular, one thing I like to be able to do is to read an email, figure out what "next action" to do with it, record that next action, and pin that email to the next action for reference.

To this end, I have a simple shell script and muttrc macro that you can copy from the hsgtd contrib directory. The shell script greps an email from stdin for its message id and reads the command line parameters for the next action text. It combines the two by adding an hsgtd action using the message ID as a project name. Here's the script to show you how simple and stupid it is:

#!/bin/bash
MSGID=$(grep -i '^message-id' | head -n 1 | sed 's/Message-I[Dd]: /:/')
hsgtd add "$@" "$MSGID"

To make this work with mutt, I also have a small macro that lets me call the shell script whenever I'm viewing a message:

macro pager \Ca "|email-add-action"
macro index \Ca "|email-add-action"

Triage example

So how does this get used in practice? Let's say my inbox has a patch to Darcs from Guillaume.

If you saw Merlin Mann's Inbox Zero talk, there are 5 "verbs" you can apply to an inbox item. Let's run through these. Clearly this is not a mail I want to [i] delete, and for a variety of reasons, it's not something I want to [ii] delegate, or to [iii] defer. Let's look at the email in mutt:
I can't [iv] respond yet because I need take some time out to review the patch so I need [v] track an action for this to do later. I hit Control-a in mutt, and type in "@darcs review this". This creates an action in hsgtd. If I later visit hsgtd and type "list" to see the actions available, I will see the email from Guillaume:

By the way, if you're wondering about the "@darcs", the use of an at-sign before a word is an hsgtd convention for contexts. Contexts are a useful way of dividing up actions because they signify certain constraints on where you can perform the actions (typical contexts might be @home, @work). I use @darcs because working on darcs is sometimes something I'll do in one block at a time. If I type "list @darcs" in hsgtd, it will show me only the actions for that context:

Back to main story. We've now added Guillaume's message to hsgtd. Let's take a closer look at the entry that was created. You see the original action text that we typed in "@darcs review this". Notice how the context @darcs was helpfully highlighted in yellow. In green you will also see a strange suffix like ":<4ba5fc74.0e0db80a.261d.ffff8b51@mx.google.com>". This is useful for three reasons:

It creates a GTD "project" for that email. Sometimes dealing with an email requires more than one action. In the GTD world, any set of >1 action is considered a project.
[most important] It gives you a means for retrieving the email that goes with this action when you are actually predisposed to do that action.
It allows you to be fairly oblique in your next action texts, you can type in any short string which seems to be meaningful without having to be super-precise about it.

Next up: waiting and review

In this posting, we saw a way of extracting "next actions" from your mutt inbox and storing them in an hsgtd list. In a future posting, I hope to expand on this by exploring delegation (asking somebody else to act) and review (going over your actions and delegated items). Actually, the review was what initially motivated this blog posting. I'd finally worked out how to create a virtual mailbox of my hsgtd-tracked items and wanted to show it off. But that will have to wait as this post is long enough as it is.

darcs team at ZuriHac

2010-03-20T16:15:00.004+01:00

Just a quick photo showing what happens when you give a bunch of Darcs hackers a flipchart and a marker pen...

(With thanks to David Anderson for gamely taking this photo for our collective memory)

This was the result of a lively discussion on the future darcs rebase feature, which will make maintaining long-term branches in Darcs a lot easier. Perhaps it'll be ready in early 2011. We'll be sure to take our time to get this right...

heapgraph tool

2010-01-21T18:15:00.013+01:00

Here is a small program to help draw diagrams of heap graphs.

You feed it (via stdin) a text file written a silly little language, for example:

graph g0
node n0 (closure "double" (closure "(*)" "5" "4"))
 
graph g1
node n0 (closure "(+)" n1 n1)
node n1 (closure "(*)" "5" "4")
 
graph g2
node n0 (closure "(+)" n1 n1)
node n1 "20"
 
graph g3
node n0 "40"

...pipe the results through Graphviz

./heapgraph < example | dot -T pdf -o example.pdf

...and what you get back is a little series of graphs like the following:

I never worked out how to tell graphviz to draw the subgraphs from top to bottom instead of left to right. Help would be appreciated :-)

The context is that I'm in the process of reading Rabhi and Lapalme's Algorithms: A Functional Programming Approach. One of its introductory chapters has an explanation of graph reduction. It occured to me that I ought to write lots of little graphs and just walk through them. The general idea is that maybe one of the impediments to my understanding Haskell laziness/strictness was sheer impatience, that I was being far too motivated to make my programs go faster. I'm hoping that a slower and more methodical approach will work, for example, starting by making sure I understand basic ideas like a heap first.

Perhaps such a tool will be useful for you if you are in a similar position, or if you happen to be teaching this sort of stuff.

darcs hashed-storage work merged (woo!)

2009-10-08T18:44:00.002+02:00

The following is a copy of my recent post to the darcs-users mailing list.

Hi everybody,

So you may have noticed me saying this in a couple of recent threads. Petr Ročkai's hashed-storage work from his 2009 Google Summer of Code project has been merged!

I thought I would take a few moments to give everybody an overview of how this work benefits us, and where we'll be going in the future.

In a nutshell

What does this mean for you? Faster repository-local operations.

Hashed format repositories (with darcs-1 and darcs-2 patches alike) should now be faster to use on a daily basis. We saw the very beginnings of this work in Darcs 2.3.0 with a faster darcs whatsnew. Now these speed improvements cover all repository-local operations.

The next Darcs beta is a couple of months away, but before that, I would like to encourage you to try this out for yourself:

darcs get --lazy http://darcs.net
cd darcs.net
cabal install

For best results, please run darcs optimize --upgrade followed by darcs optimize --pristine. Pay attention over the next couple of weeks when you try a record, amend, revert, unrecord. If we've done our work right, there should be nothing to see. Darcs should be less noticeable, with fewer "Synchronizing pristine" messages and a faster return to the command prompt. We think you'll like it. But please get back to us. Is Darcs faster for you?

If you're particularly interested, I will step through these changes in greater detail at the end of this message. Meanwhile, I would like to step back a little and take stock of how these improvements fit in to the bigger picture.

The road ahead

The hashed storage work is a big step forward and definitely a cause for celebration. I think it is useful to reflect on this progress and consider how it fits in with our progress since darcs 1.0.9:

ssh connection sharing (darcs transfer mode)
HTTP pipelining
lazy repositories
the global cache

and now

index-based diffing
hashed-storage efficiency

We cannot promise that Darcs will magically become fast overnight. But what we can and will do is continue chipping away at it, solving problems one at a time; release by release, a little bit better, a little bit faster every time until one day we can look back and marvel at all the progress we've made.

So Petr's work makes Darcs easier to live with on a day-to-day basis. But that's not enough. Now we need to turn our attention to that crucial first impression; what happens when people try Darcs out for the first time is that they darcs get a repository they want and... then... they... wait...

This is embarrassing, but we can fix it. In fact, we already have started working on the problem. The next version of hashed-storage will likely introduce a notion of "packs" in which the many often very small files that Darcs keeps track of will be concatenated into more substantial "packs" that compress better and reduce the ill effects of latency. My hope is that we will be able to complete the packs work by Darcs 2.5.

There's a lot more progress to be made: smarter patch representations, tuning for large patches, file-to-patch caching for long histories. And that's just performance! For more details about our performance work, please have a look at

http://tinyurl.com/darcs-performance2

If you could do anything to help, benchmark, profile, anything at all, please let us know :-)

The fight continues.

Thank-you!

Petr and Ganesh deserve a huge round of applause. Petr, thanks for thinking up this work, getting it done and pushing it through. Ganesh, thanks for an extremely thorough and thoughtful review. The two of you, thanks for holding on, for tenacious cooperation in the face of adversity.

Thanks also to all the wider Darcs community for all your support, comments, patch reviews.

I'm looking forward to seeing you at the upcoming Darcs hacking sprint. The sprint will take place in Vienna, Austria on the weekend of 14-15 November. Everybody, especially Darcs and Haskell newbies, is welcome to join in. Details on http://wiki.darcs.net/Sprints/2009-11

And if I may take a paragraph to mention this, Darcs needs your support. Every little counts, if you can send patches, review patches, tweak documentation, profile, benchmark, submit bug reports. Barring that, you could also make a contribution to our travel fund via the Software Freedom Conservancy. See http://darcs.net/donations.html for details.

Thanks everybody and enjoy!

Eric

Changes in detail

Darcs uses an "index" file to compute working directory and pristine cache diffs. This avoids timestamps going out of synch when you have multiple local branches, which saves a huge and needless slowdown.
Hashed storage is more efficient in general. Even if you already have perfect timestamps, the new optimisations should make Darcs faster in general.
The new 'darcs optimize --pristine' reduces spurious mismatches on directories.
Darcs no longer requires a one second sleep after applying patches.

cabal installing graphical apps on MacOS X

2009-09-11T17:10:00.006+02:00

I have a graphical command line tool written in wxHaskell. For the longest time, my tool was relatively easy to install on Linux but a pain on MacOS X because my users had to jump through extra post-installation hoops like creating application bundles.

Thanks to some very patient help from Beelsebob, quicksilver, dcoutts on #haskell I was finally able to cobble together a Setup.hs file that lets me do just this. Now when I write install instructions for my program, I no longer need to add extra bullet points telling people to turn knobs and twiggle blops just to run the GUI. It just works.

Note that this was written with wxHaskell in mind. I hope that folks using gtk2hs and qtHaskell either do not have this problem or can make use of a similar solution.

desiderata

What I wanted was for the 'cabal install' command to work as well on MacOS X as it did under Linux. My core desiderata were:

Ability to call my application from the command line the same way you would under Linux with command line arguments correctly recognised
No need for the user to add extra junk to the path (besides $HOME/.cabal/bin which they'll already have added)
No manual intervention after cabal install (eg calling scripts to create application bundles)
No need to be super-user.

basic ideas

The basic ideas behind this solution are

Replace "foo" with a shell script that calls "foo.app/MacOS/Contents/foo"
MacOS X Leopard seems to want graphical applications to live in application bundles. At least for wxHaskell if you invoke "foo" you get a GUI that does not respond to input. On the other hand, if you invoke "foo.app/MacOS/Contents/foo" you get something that works.
Use a Cabal postInst to create the application bundle in the bin dir.

basic solution

Here is the solution. (I'll send it as a mail to the wxhaskell-users mailing list too)

-- --------------- BEGIN Setup.hs EXAMPLE ------------------------------
import Control.Monad (foldM_, forM_)
import Data.Maybe ( fromMaybe )
import System.Cmd
import System.Exit
import System.Info (os)
import System.FilePath
import System.Directory ( doesFileExist, copyFile, removeFile, createDirectoryIfMissing )

import Distribution.PackageDescription
import Distribution.Simple.Setup
import Distribution.Simple
import Distribution.Simple.LocalBuildInfo

main :: IO ()
main = defaultMainWithHooks $ addMacHook simpleUserHooks
 where
  addMacHook h =
   case os of
    "darwin" -> h { postInst = appBundleHook } -- is it OK to treat darwin as synonymous with MacOS X?
    _        -> h

appBundleHook :: Args -> InstallFlags -> PackageDescription -> LocalBuildInfo -> IO ()
appBundleHook _ _ pkg localb =
 forM_ exes $ \app ->
   do createAppBundle theBindir (buildDir localb </> app </> app)
      customiseAppBundle (appBundlePath theBindir app) app
        `catch` \err -> putStrLn $ "Warning: could not customise bundle for " ++ app ++ ": " ++ show err
      removeFile (theBindir </> app)
      createAppBundleWrapper theBindir app
 where
  theBindir = bindir $ absoluteInstallDirs pkg localb NoCopyDest
  exes = fromMaybe (map exeName $ executables pkg) mRestrictTo

-- ----------------------------------------------------------------------
-- helper code for application bundles
-- ----------------------------------------------------------------------

-- | 'createAppBundle' @d p@ - creates an application bundle in @d@
--   for program @p@, assuming that @d@ already exists and is a directory.
--   Note that only the filename part of @p@ is used.
createAppBundle :: FilePath -> FilePath -> IO ()
createAppBundle dir p =
 do createDirectoryIfMissing False $ bundle
    createDirectoryIfMissing True  $ bundleBin
    createDirectoryIfMissing True  $ bundleRsrc
    copyFile p (bundleBin </> takeFileName p)
 where
  bundle     = appBundlePath dir p
  bundleBin  = bundle </> "Contents/MacOS"
  bundleRsrc = bundle </> "Contents/Resources"

-- | 'createAppBundleWrapper' @d p@ - creates a script in @d@ that calls
--   @p@ from the application bundle @d </> takeFileName p <.> "app"@
createAppBundleWrapper :: FilePath -> FilePath -> IO ()
createAppBundleWrapper bindir p =
  writeFile (bindir </> takeFileName p) scriptTxt
 where
  scriptTxt = "`dirname $0`" </> appBundlePath "." p </> "Contents/MacOS" </> takeFileName p ++ " \"$@\""

appBundlePath :: FilePath -> FilePath -> FilePath
appBundlePath dir p = dir </> takeFileName p <.> "app"

-- optional stupff: to be discussed later
mRestrictTo = Nothing
customiseAppBundle _ _ = return ()
-- --------------- END Setup.hs EXAMPLE ---------------------------------

fancier solution

I also have some extra wishlist items.

Possibility of installing in --global
Fancy custom app bundles with custom icons and what not

Global installation might already be working with this basic script, but I haven't tested it yet. Fancy app bundles sort of work (if I double-click it in Finder, I get a customised icon, but running it from the command line does not give me one).

Here are extra hooks I created for this:

-- ------------- BEGIN FANCY Setup.hs ADDENDUM ------------------------
-- | Put here IO actions needed to add any fancy things (eg icons)
--   you want to your application bundle.
customiseAppBundle :: FilePath -- ^ app bundle path
                   -> FilePath -- ^ full path to original binary
                   -> IO ()
customiseAppBundle bundleDir p =
 case takeFileName p of
  "geni" ->
    do hasRez <- doesFileExist "/Developer/Tools/Rez"
       if hasRez
          then do -- set the icon
                  copyFile "etc/macstuff/Info.plist" (bundleDir </> "Contents/Info.plist")
                  copyFile "etc/macstuff/wxmac.icns" (bundleDir </> "Contents/Resources/wxmac.icns")
                  -- no idea what this does
                  system ("/Developer/Tools/Rez -t APPL Carbon.r -o " ++ bundleDir </> "Contents/MacOS/geni")
                  writeFile (bundleDir </> "PkgInfo") "APPL????"
                  -- tell Finder about the icon
                  system ("/Developer/Tools/SetFile -a C " ++ bundleDir </> "Contents")
                  return ()
          else putStrLn "Developer Tools not found.  Too bad; no fancy icons for you."
  ""     -> return ()

-- | Put here the list of executables which contain a GUI.  If they all
--   contain a GUI (or you don't really care that much), just put Nothing
mRestrictTo :: Maybe [String]
mRestrictTo = Just ["geni"]
-- ------------- END FANCY Setup.hs ADDENDUM ---------------------------

vim and building with cabal

2009-07-29T13:56:00.006+02:00

I don't know about you, but I've got map ,m :make<Enter> in my .vimrc to bind comma-m to my build program. This could be "ant" for Java files (for example) and "make" otherwise.

Now here is a snippet to set it to "cabal build" as needed

"-----------------------8<--------------------------
function! SetToCabalBuild()
  if glob("*.cabal") != ''
    set makeprg=cabal\ build
  endif
endfunction

autocmd BufEnter *.hs,*.lhs :call SetToCabalBuild()
"-----------------------8<--------------------------

Apologies for making noise in case this is already redundant with a piece of Claus Reinke's very interesting and modular-looking Haskell mode for Vim (which I've been promising myself to install some day). Perhaps the above will be useful anyway for those of us still limping along with configuration files cobbled together from bits and bobs on the web.

some ideas for practical QuickCheck

2009-07-28T09:18:00.012+02:00

I think I've found some answers to my practical QuickCheck questions. This post may be fairly long as I'm trying to make it concrete and explicit enough to overcome the kind of inertia I had when I was still resisting testing.

How do I make my tests easy to run?

1. Use test-framework

The key thing to know about test-framework is that it is very easy to get started. Just visit the friendly web page and copy the example.

Note: An earlier post suggested the testrunner package developed for Darcs, but at the time we didn't realise that test-framework already had all the features needed.

2. Support cabal test

Here's a Setup.hs recipe I copied. It has the handy property of the code is that it runs your tests straight from your dist/build directory.

-- EXAMPLE Setup.hs FILE 1 -----------------------------------------------
import System.FilePath

main = defaultMainWithHooks hooks
  where hooks = simpleUserHooks { runTests = runTests' }

runTests' :: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO ()
runTests' _ _ _ lbi = system testprog >> return ()
  where testprog = (buildDir lbi) </> "test" </> "test"
-- -----------------------------------------------------------------------

The code snippet for your Setup.hs file comes from Greg Bacon's Setting up a Simple Test with Cabal (I tacked on an import). As you can see, the recipe assumes you're building an executable called "test" (see Greg's post on how to do this)

3. Bake your unit tests in

This may go down as the kind of bad advice that "seemed like a good idea at the the time". For now, I can justify this by saying that it may be reassuring to users to be able to just run the same tests that I'm running and see for themselves that their program thinks it's working.

I've been working on a program called GenI. To help people test this program, I've added a simple "--tests" switch. Now people can run geni --tests for a self check. If they want, they can also "cabal test", using this slight modification to Greg's setup file (to call geni itself and to pass the --tests flag in).

-- EXAMPLE Setup.hs FILE 2 -----------------------------------------------

import System.FilePath

main = defaultMainWithHooks hooks
  where hooks = simpleUserHooks { runTests = runTests' }

runTests' :: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO ()
runTests' _ _ _ lbi = system testprog >> return ()
  where testprog = (buildDir lbi) </> "geni" </> "geni --tests"

-- -----------------------------------------------------------------------

As for GenI, whenever I see --tests in my arguments (for example "--tests" `elem` args), I just pass control to another module, which in turn strips the switch out and passes the rest of the arguments to test-framework.

-- EXAMPLE TEST-FRAMEWORK WRAPPER ------------------------------------------
module NLP.GenI.Test where

import System.Environment ( getArgs )
import Test.Framework

import NLP.GenI.GeniVal ( testSuite )
import NLP.GenI.Tags ( testSuite )
import NLP.GenI.Simple.SimpleBuilder ( testSuite )

runTests :: IO ()
runTests =
 do args <- filter (/= "--tests") `fmap` getArgs
    flip defaultMainWithArgs args
     [ NLP.GenI.GeniVal.testSuite
     , NLP.GenI.Tags.testSuite
     , NLP.GenI.Simple.SimpleBuilder.testSuite
     ]
-- -----------------------------------------------------------------------

There's some other things going on in this file, notably the organisation of test suites. More on that later.

Where should I put my properties?

4. Put tests in the same module (where relevant)

If a test is specific to one module, I tend to put them in that same source file. I do this because

It lets me test functions that I don't want to export
The tests serve as documentation
It forces me to update my tests along with my code

This approach is in contrast to (a) having one big tests module and (b) having a separate test hierarchy. It may turn out to be useful to have a single big tests module as well, for example, for tests that cross the boundary from one module to the next. That need has not arisen for me yet. Likewise, I don't particularly believe in a separation between tests and code, although on the other hand some very experienced hackers seem to do so, so I'll just have to let experience teach me why.

How do I avoid repeating myself?

5. Provide a testSuite function for each module

Commenting on my last post, Josef kindly pointed out that the book-keeping I feared isn't so bad in practice. He's right. Nevertheless, I want to avoid it. To do this, I make each of my modules export a testSuite function. Here is what one of my modules looks like, just focusing on the test suite

-- EXAMPLE MODULE --------------------------------------------------------
module NLP.GenI.GeniVal where

-- SKIPPED MAIN IMPORTS ...

import Test.Framework
import Test.Framework.Providers.HUnit
import Test.Framework.Providers.QuickCheck
import Test.QuickCheck
import Test.HUnit

-- SKIPPED MAIN CODE

testSuite = testGroup "unification"
 [ testProperty "self" prop_unify_sym
 , testProperty "anonymous variables" prop_unify_anon
 , testProperty "symmetry" prop_unify_sym
 , testCase "evil unification" test_evil
 ]

-- SKIPPED THE TESTS THEMSELVES
-- -----------------------------------------------------------------------

If you'll scroll up to the example that's marked TEST-FRAMEWORK WRAPPER, you'll see how these test suites are used in practice. Note the small trick of using the qualified module name to identify the test suite.

Anyway, the general principle of having a per-module test suite comes from Aidan Delaney's Organising Unit Tests in Haskell. The main difference between his approach and my approach are that I mix tests and code rather liberally.

Conclusion

I hope that some of these hints will make testing easier for you, or perhaps even get you started. If you still find yourself putting testing off, let me know. I'll be curious to see what else makes us resist. One thing that would probably be helpful is an extra guide to writing Arbitrary instances for QuickCheck, and also writing good properties that control the space well. Maybe even getting started with SmallCheck.

Note that I am still somewhat new to testing and have only recently started these practices. So take these ideas with the usual salt. Thanks to Greg, Reinier, Aidan, and also folks who commented on my previous posts.

Haskell syntax highlighting on Wikipedia and Wikibooks

2009-06-24T09:32:00.003+02:00

If you edit the Haskell Wikibook and Wikipedia entries with Haskell in them, you may be interested to note that Haskell syntax highlighting is now available on all Wikimedia projects.

Example:

<source lang="haskell">
-- foo
let x = foo
</source>

testrunner for practical quickcheck

2009-06-08T12:38:00.004+02:00

I had mentioned in a previous post three practical problems I had getting started with QuickCheck. My third question in this post was:

How do I make my tests easy to run? Do I have to write my own RunTests module? Should I just use something like quickcheck-script?

And one of the replies I got:

I'm sure people are writing tests, but we all hack up harnesses in our own idiosyncratic ways.... -- blackdog

Maybe we can do better. Instead of everybody hacking up their own harness, how about having one test harness that everybody wants to use? We may even have a candidate for such a harness. Reinier Lamers has recently released a "testrunner" package which supports some rather nice features:

It can run unit tests in parallel.
It can run QuickCheck and HUnit tests as well as simple boolean expressions.
It comes with a ready-made main function for your unit test executable.
This main function recognizes command-line arguments to select tests by name and replay QuickCheck tests.

That's all really good stuff, but I think the number one best feature for me would be the little tutorial on its homepage.

Testrunner is work that Reinier started in the context of the darcs project. We were trying to make our own custom test suite faster and more useful. Seeing ahead, Reinier did it not just by tweaking and tuning the harness we have, but by writing a more general purpose harness that did the things we wanted it to do and hopefully which other projects would want to do as well. So do you have a Haskell project that needs testing? Or maybe you already are doing some tests, but you just wish you could squeeze a little more out of your tests? Give testrunner a try!

Edit 2009-06-08 17:15
It turns out there is a second candidate, or rather a first candidate since test-framework has been around for months. Embarrassingly enough, I had started to use test-framework for my own stuff, but I never realised how feature complete it was. Maybe it'll be time to merge projects? I'll see what Reinier thinks. Apologies to Max...

inkscape layers

2009-02-26T11:41:00.011+01:00

Here's a small program that I wrote to extract a subset of layers from an Inkscape file. It may be handy if you have to give a talk and you want to include some "animated" overlays in your slides.

I'm writing this post because I'm pleased to be able to automate this process at last. Also, I want to demonstrate that you don't have to be particularly clever or ambitious to get some good practical use out of Haskell.

usage

So I've got my Inkscape file with a "base" layer and several steps of my animation "zero", "one", "two", "three".

If I do inkscape-layers myfile.svg base > /tmp/foo.svg && inkscape --export-pdf=/tmp/foo.pdf", I get just the base layer which isn't very interesting:

Now if I do inkscape-layers myfile.svg base zero (and convert the resulting SVG into a PDF as above), I get the zeroth layer:

Likewise, to build the rest of my animation, inkscape-layers myfile.svg base one

inkscape-layers myfile.svg base two

Now instead of going clickity-click all over the place, I just dump this in my Makefile. If I every have to change something about my animation (for example, in the base layer), I just run "make" and rebuild it automatically.

Yay, Haskell! Well, I'm sure you could just as easily have written this in your favourite programming language; I just like to randomly credit Haskell for making my life easier :-D

the code

I may upload this to Hackage if I could maybe get some other useful inkscape tools with it:

import Data.Maybe (fromMaybe)
import System.Environment (getArgs, getProgName)
import System.IO (hPutStrLn, stdout, stderr)
import Text.XML.Light

main =
 do args  <- getArgs
    pname <- getProgName
    case args of
      (f:ls) -> go f ls
      _      -> hPutStrLn stderr $ unwords [ "Usage:", pname, "filename", "layer1", "[layer2 [.. layer N]]" ]

go f ls =
 do d <- goodXML =<< parseXMLDoc `fmap` readFile f
    let o = stdout -- we may want to make this more flexible later
    hPutStrLn o . showTopElement . wrapTop walk $ d
 where
  goodXML = maybe (fail "bad XML") return
  --
  walk x@(Elem el) =
   let lbl = fromMaybe "" (findAttr qLABEL el)
       x2  = Elem $ el { elContent = map walk (elContent el) }
   in case () of _ | not (isLayer el) -> x2
                   | lbl `elem` ls    -> x2
                   | otherwise        -> Text blank_cdata
  walk x = x

isLayer el = elName el == qSVG "g" && findAttr qGROUP_MODE el == Just "layer"

qLABEL      = qInkscape "label"
qGROUP_MODE = qInkscape "groupmode"

qSVG l = QName l (Just nsSVG) Nothing
nsSVG = "http://www.w3.org/2000/svg"

qInkscape l = QName l (Just nsINKSCAPE) Nothing
nsINKSCAPE="http://www.inkscape.org/namespaces/inkscape"

wrapTop f e =
 case f (Elem e) of
 (Elem e) -> e
 _ -> error "programmer error: top content is not an element"

Note: as an exercise: modify the attributes of all exported layers so that they are visible. In Inkscape, I tend to make layers invisible so I don't get confused by them. But then Inkscape does not export them, which is annoying. This seems to be a simple matter of replacing "display:none" with "display:inline" in the style attribute (watch out, there could be more than one!). The 'split' library on Hackage could be handy for that.

implementing join in terms of (>>=)

2009-02-21T19:30:00.007+01:00

One of the things I got out of the Typeclassopedia is a somewhat more mature understand of monads (at last!). As a bonus side-effect it has also given me a slightly better understanding of myself. Specifically, I learned I often have trouble learning things because I suffer from a sort of "failure to unify". I thought I might make a note of it for the benefit of anybody else who is interested in how they learn... or not, as the case may be.

So,

we have (>>=) :: m a -> (a -> m b) -> m b
we want join :: m (m x) -> m x

My mind drew a complete blank. So I went with something "direct" via do notation:

join mmx =
 do mx <- mmx
    x  <- mx
    return x

Those last two lines are redundant:

join mmx =
 do mx <- mmx
    mx

Hang on, Eric, surely you don't need the crutch of do notation...

join mmx = mmx >>= (\mx -> mx)

That's just id:

join mmx = mmx >>= id

But wait! Surely that can't be right! Doesn't (>>=) require something of type a -> m b? And isn't id giving me m x -> m x? I stared at that for a while, almost panicking. What did I do wrong? And then it clicked. Of course, the a in a -> m b could stand in for any type, including m x. Just because it doesn't have a little m in it, doesn't mean that it's constrained not to have one.

A simpler version of this kind of error, although one that didn't get me this time: just because we have a and b doesn't mean we actually have to have two different types. They can, but don't need to. And that, is my "failure to unify", inventing completely illusory constraints and not seeing through them.

And so join is just (>>= id). It took a little struggle, but it was well worth it!

(PS, in my original attempt, I used the more conventional m (m a) when thinking of the types instead of what I reported here, m (m x). The reason I reported the later is because I didn't want to confuse the discussion with another stumbling block I have, which is a "failure to rename", i.e. forgetting that two things called a in different contexts are actually two separate things. It's like speaking a foreign language. Just because you are aware that you have to do something, doesn't mean you will always do it automatically. Anyway, the "failure to rename" may very likely have conspired with the "failure to unify" in making me confused for a while)

announcing: burrito tutorial support group

2009-02-16T16:45:00.005+01:00

It's really for the best if you leave these sorts of things out in the open.

The first step is to ask for forgiveness, right?

practical quickcheck (wanted)

2009-02-04T11:33:00.004+01:00

Despite all the glowing reports on how useful QuickCheck is, I find that I still have a lot of resistance to using it. A lot of resistance comes from uncertainty, so in this post, I'm going to write down some of my half-formulated questions about using QuickCheck.

Now, there may not be any right answer to these questions, but I'm writing them down anyway so that other people in my shoes know that they are not alone. Later on, as I find the answers that work for me, I'll hopefully put together some notes on 'Practical QuickCheck'.

Where should I put my properties? Xmonad and darcs seem to put them in a single properties module, but it would seem more natural to me to stick them in the same module as the functions I'm quickchecking. That said, I imagine that some properties can be thought of as being cross-module, so maybe a properties module would make sense.
How do I avoid redundancy, and generally repeating myself? Ideally, I would just write a property and be done with it. It would annoy me to have to keep updating some list of properties somewhere else (duplication). That said, maybe it's not really duplication if the list serves a secondary purpose of grouping the properties into some sensible hierarchy. Maybe the real question is "how do I make sure I don't forget to run all my properties?"
How do I make my tests easy to run? Do I have to write my own RunTests module? Should I just use something like quickcheck-script?

I might update this list later as I think of more "best practices" questions. Hopefully I can follow this up with a short article teaching myself and others that really getting started with QuickCheck is easy easy easy (or maybe a link to a pre-existing article of the sort). The Real World Haskell chapter on it seems helpful.

haskell-ji

2009-01-30T11:16:00.011+01:00

As a programmer, I find myself struggling with a lot of really mundane and stupid-looking issues like "how should I name my variables", or "should acronyms be kept upper case (XML), or smooshed down for easier CamelCasing (Xml)?" and finally "what order should my code go in?"

These questions do not so much keep me up and night, but cause me an inordinate amount of flip-flopping in my code. Not remembering my preference du jour, I'll sometimes do things four different ways in code and later on suffer because I forgot that in one bit of code, I had named something parseXML and in the other bit, I had named it xmlParse.

The good news is that things are settling down on at least one front. It seems that all the versions of Eric past and present are settling on a consensus on How To Lay Code Out. The result is a set of directional tips, akin to the kind of thing you learn when you are writing Chinese Hanzi (Japanese Kanji):

Types before code
High-level before low-level -- For example, generally using where instead of let...in, but also "higher-level" functions first, "detail" functions later
Input before output -- It's not that this was ever up for debate, it's just that sometimes, I'll write it the other way without realising that I'm doing it.
Odds and ends last -- At the very end of my code: an odds-and-ends section for all those little snippets of code you copy around but are that too small to justify making a library, e.g.
```
buckets :: Ord b => (a -> b) -> [a] -> [ (b,[a]) ]
buckets f = map (\xs -> (f (head xs), xs))
        . groupBy ((==) `on` f)
        . sortBy (compare `on` f)
```
Do you have an odds-and-ends.hs file on your computer?

Notice that the tips are not always compatible with each other, but they do sort of point in the same general direction.

Phew, I'm glad I'm starting to get at least this bit sorted. I really hope it reduces the amount of pointless erician flip-flopping. It's no big deal -- civilisation does not collapse because of inconsistent case conventions -- but it is a nuisance. This kind of thing is on the order of silly American-style dates vs. European-style dates causing confusion, where we could all just be using International yyyy-mm-dd dates, and while we're at it, 24 hour time, the metric system and A4 paper...

fold diagram revisited?

2009-01-08T10:11:00.002+01:00

z
|
f----1----f
|    :    |
f----2----f
|    :    |
f----3----f
|    :    |
f----4----f
|    :    |
f----5----f
     :    |
     []   z

riot is almost a Haskell mail client

2008-12-30T11:17:00.004+01:00

In case anybody wanted to write a mail client in Haskell, I should point out that Tuomov's riot (Riot is an Information Organisation Tool) outliner

provides a sort of mutt-like user interface and
stores its outlines as mailboxes (using in-reply-to to treat outline ancestry as thread ancestry).

So that's some of the work done for you :-)

iterative committing

2008-11-19T21:16:00.010+01:00

A few weeks ago, I saw this interesting complaint about distributed revision control advocacy:

But really, to read some of these articles, you'd think 99.9% of OSS contributions come from people who live on planes, only get 10% uptime on their broadband at home, and are incapable of spending the five minutes required to install something like Subversion locally for use with side projects.

This particular complaint resonated with me because I've always had a slight feeling that all this talk of airplanes and intermittent online access is missing the point.

I think what would help these discussions is to introduce the idea that there are really two ways to be disconnected: the involuntary way that most people talk about, and the voluntary way which is the really interesting one.

To be involuntarily disconnected is to be literally or technically offline. The universe prevents you from phoning home because it broke your wifi card or plunked you deep in an Amazonian rain forest. True, a distributed revision control system lets you continue hacking in the face of such adversity; but this fact isn't very convincing to some folks who are used to centralised revision control. How often in today's world are you really involuntarily offline? The trick is that sometimes your disconnectedness is entirely voluntarily. I don't really mean that in the sense of unplugging your cable modem and calling a moratorium on network access for the day. The minute you want to commit to a server and can't because of missing network access, you are offline involuntarily, even if this came as the result of a voluntary decision.

What I mean is that distributed revision control allows you to have pockets of deliberate disconnectedness from your peers. You want to work on something in little bits and pieces, you want to version control your work in progress, but you don't want to inflict your uncompleted work on your friends. A distributed VCS gives you a chance to step back for a moment and continue working with the benefit of version control. There are two alternatives to stepping back, neither of which are really acceptable. The first is to go ahead and commit your stuff with wild abandon, the consequences of which being that you pollute the change history with unfinished work and make life potentially difficult for your friends. The second alternative is NOT to commit your stuff at all, the consequences of which being that you lose the ability to track and log your your work as you go along.

A distributed revision control system gives you the choice of iterative committing. It doesn't really matter if you are online or offline actually. Sometimes you just want to commit something for your own sake and only later decide if the commits should be shared with the main repository or not. In the meantime you can choose to go back, undo a commit, redo a commit, undo all your intermediary commits and lump them all into one big commit, update from the main repository and then rework your commit in the new context. These are the choices that a distributed revision control system offers.

It's heartening to see that the idea of using a distributed VCS is catching on, that people are starting to adopt the likes of darcs, git, mercurial and bzr for their work. It means that the joy of iterative committing is spreading. Of course, I am partial to one of these systems in particular. Perhaps in a future article, I can describe what I think is the essential difference between darcs and our estimable competitors. I think I will call it iterative merging.

Happy committing in the meantime!

timesheet helper

2008-11-07T19:01:00.005+01:00

I wish there was a simple, no-fuss command line timesheet helper in the spirit of cabal-install and twidge. The kind of interactions I imagine are:


09:00 # timesheet start work draft 3 of the paper
10:00 # timesheet start darcs dwn
10:45 # timesheet start work regression test for ppack
12:00 # timesheet stop
12:30 # timesheet start darcs roadmap
13:15 # timesheet start work regression test for ppack
16:30 # timesheet start darcs patch review
17:00 # timesheet start work meeting
18:30 # timesheet stop

18:30 # timesheet summary
Today 2008-11-07
-------------------
darcs: 2h
work: 6h 30m

18:30 # timesheet details
Today 2008-11-07
-------------------
darcs: 2h
* dwn: 30 m
* roadmap: 45m
* patch review: 45m
work: 6h 30m
* draft 3 of the paper: 1h
* regression test for ppack: 4h
* meeting: 1h 30m

(Note the assumption here that you are never working on two tasks at the same time; clocking in to a new task automatically clocks you out of an old one). The key to this application is simplicity. In its present state, gpe-timesheet (0.32) uses too many confirmation dialogues to be really useful. Loggr is nice and simple, but if I close my browser window, I lose track of things. Another property I would like to have is for the application to be forgiving to mistakes. If it stored timesheets in a simple text format, for example, I could just edit out my mistakes in a text-editor.

For Haskellers, I also wish that we had a common library for writing command line applications with subcommands and switches. This would be useful for darcs, cabal-install, twidge, this timesheet application, and more.

official darcs blog!

2008-10-30T15:35:00.003+01:00

Darcs weekly news has moved! It will now be hosted on the official darcs blog at http://blog.darcs.net.

The latest entry, darcs weekly news #10 has been posted on the new blog.

darcs hacking sprint - Team Brighton Day 2

2008-10-26T18:29:00.005+01:00

Ganesh and Ian, slurpies and curl

More Important Looking Things for the whiteboard (faster slurpies, courtesy of Ganesh)

Team Brighton. (having worked out the auto-timer on Eric's camera)

Sprint on!

Sprint wrap-up later...

darcs hacking sprints - some pictures from Team Brighton

2008-10-25T18:21:00.009+02:00

Just a little update from day 1. Who's doing what?

Ganesh (Heffalump) profiling away and drinking coffee from a University of Brighton mug!

Ian (Igloo) looking serious and Campy

Eric (kowey) unscattering his brain

Healthy hacking (malteasers conveniently obscured by kettle)

Hope to have a report up after the sprint!

darcs weekly news #9

2008-10-23T12:34:00.004+02:00

News and discussions

Enfranchising darcs! An update on the build systems question
- http://lists.osuosl.org/pipermail/darcs-users/2008-October/014849.html
Darcs hacking sprint in 2 days!
- http://wiki.darcs.net/index.html/Sprints
What does it mean to commute? Darcs hackers like to talk about 'commuting' patches all the time. But what does that mean? Jason explains and provides a tiny bit of code for us to play with
- http://lists.osuosl.org/pipermail/darcs-users/2008-October/014779.html
darcsweb 1.1-rc1 Alberto Bertogli reports a release candidate for darcsweb 1.1, with support for darcs 2 repositories, and syntax highlight support if the pygments module is available
- http://lists.osuosl.org/pipermail/darcs-users/2008-October/014784.html
First impressions of darcs. A Pythonista named Benjamin tries darcs out for the first time. Here are his likes and dislikes.
- http://pybites.blogspot.com/2008/10/first-impressions-of-darcs.html
Choosing a revision control system. Daniel Carrera compares darcs with Monotone, Mercurial and Bazaar. Daniel finds our "brilliant patch management" to be unique, but what can we learn from the others?
- http://daniel.carrera.name/2008/10/14/choosing-a-revision-control-system/

Reviewers

Jason Dagit

New contributors

Christian Kellermann
Salvatore Insalaco
J. Garrett Morris

Issues resolved in the last week (1)

issue784 Salvatore Insalaco

fix file handle leak and check for exceptions on process running.
http://bugs.darcs.net/issue784

Patches applied in the last week (66)

See text entry for details.

darcs weekly news #8

2008-10-17T00:13:00.001+02:00

News and discussions

Improving the darcs build system? David Roundy is doing some interesting work on building darcs with his franchise build system. There are also attempts by other folks to Cabalise darcs. Discussions are underway about the future of building darcs.
Type Correct Changes: A Safe Approach to Version Control Implementation. Jason Dagit gave a Galois tech talk on the use of Haskell GADTs to make darcs code more transparent, robust and approachable.
- http://www.galois.com/blog/2008/10/09/type-correct-changes-a-safe-approach-to-version-control-implementation/
- http://www.galois.com/~dons/slides/08-10-dagit.pdf
Haskell, static typing, type witnesses and darcs. David Roundy gave a darcs talk at the ACM (5 October), presenting darcs and also explaining how the type witnesses are helping us to avoid errors in the code.
- http://files.codersbase.com/droundy-08.pdf
Darcs hacking sprint only 9 days away!
- http://lists.osuosl.org/pipermail/darcs-users/2008-October/014541.html

Reviewers

Thanks to our patch reviewers for this week for giving David a hand!

Trent W. Buck
Jason Dagit
Nathan Gray
Simon Michael

Issues resolved in the last week (3)

issue1062 Eric Kow

ignore setCooked and setEcho errors.
http://bugs.darcs.net/issue1062

issue1105 Dmitry Kurochkin

check if default options are valid.
http://bugs.darcs.net/issue1105

issue1139 David Roundy

make special cases handle no arguments gracefully.
http://bugs.darcs.net/issue1139

Patches applied in the last week (96)

See text entry for details.

darcs 2.1.0 released!

2008-10-10T03:15:00.002+02:00

I am delighted to announce the release of darcs 2.1.0, available at

http://darcs.net/darcs-2.1.0.tar.gz

What has changed?

This version provides over 20 bug fixes and 7 new features since darcs 2.0.2. The most notable changes are:

Defaulting to darcs-2. The darcs initialize command now creates darcs-2 format repositories by default. This change will make the the improved conflict handling and merging semantics from darcs 2 available to more users. Note that no action is required on your part. Darcs will continue working with all pre-existing repositories. You can explicitly request an old-fashioned repository if needed.
Better HTTP support. Dmitry Kurochkin has refined our HTTP support and fixed several http-related bugs from darcs 2.0.2. There is also an experimental --http-pipelining feature you can enable on the command line (or in your defaults file) for faster downloading. Note: --http-pipelining is enabled by default for libwww, and also for libcurl 7.19.1 (not yet released at the time of this writing)
Repository correctness. David Roundy has resolved a longstanding 'pending patch' regression (originally reported on 2008-02). Needless to say the offending case has been moved to our regression testing suite

See the attached ChangeLog for more details.

What should I do?

Upgrade! Binary versions should be available shortly, either from your favourite package distributor or by third party contributors.

Other than installing the new darcs, no action is required on your part to perform this upgrade. Darcs 2, including this particular version, is 100% compatible with your pre-existing repositories.

If you have not done so already, you should consider using the hashed repository format in place of your current old-fashioned repositories. This format offers greater protection against accidental corruption, better support for case insensitive file systems. It also provides some very nice performance features, including lazy fetching of patches and a global cache (both optional).

If darcs 1 compatibility is not a concern, you could also upgrade your repositories all the way to the darcs 2 format. In addition to the robustness and performance features above, this gives you the improved merging semantics and conflicts handling that give darcs 2 its name.

More details about upgrading to darcs 2 here:: http://wiki.darcs.net/index.html/DarcsTwo

What comes next?

We will now be shifting to a time based release model, with the next darcs release planned for January 2009.

For the next release of darcs, we will be focusing on optimising darcs's day to day performance issues. We want darcs to fetch repositories as fast as it possibly can over a network, and we especially want to rehabilitate known slow commands like darcs annotate. We believe that a few simple and practical changes can really improve the darcs experience for most users.

Think you can help? We would love to hear from you. In fact, the first darcs hacking sprint (25-26 October) is fast approaching! We have three venues available: Brighton, Paris and Portland and everybody is invited to come hack. See http://wiki.darcs.net/index.html/Sprints for details.

Thanks everybody, and enjoy!

darcs weekly news #7

2008-10-10T00:51:00.001+02:00

News and discussions

Darcs 2.1.0 released! With 20 bug fixes and 7 new features. Notable changes: darcs-2 repositories by default, HTTP robustness and better pending patch handling.
- http://www.darcs.net/darcs-2.1.0.tar.gz
- http://lists.osuosl.org/pipermail/darcs-users/2008-October/014504.html
Optimising darcs annotate. Darcs annotate is too slow. Proposed solution: create a cache mapping filenames to patches. Stay tuned for fast annotate in the future...
- http://lists.osuosl.org/pipermail/darcs-users/2008-October/014371.html
Eleven new contributors since darcs 2.0.2. Thanks, Alex, Florent, Gaetan, Judah, Matthias, Max, Nathaniel, Steve, Taylor, Thorkil, and Vlad!

Reviewers

Thanks to our patch reviewers for this week for giving David a hand!

Trent Buck
Tommy Pettersson

Issues resolved in the last week (6)

issue1104 Dmitry Kurochkin

stop progress reports in exec.
http://bugs.darcs.net/issue1104

issue1109 Dmitry Kurochkin

stop progress reports in exec_interactive.
http://bugs.darcs.net/issue1109

issue1111 Tommy Pettersson

patchset_intersection used wrong selection for partitionRL
http://bugs.darcs.net/issue1111

issue1124 Thorkil Naur

Test pull.sh failed when run as root
http://bugs.darcs.net/issue1124

issue1128 Benjamin Franksen

must call execSendmail inside body of withOpenTemp
http://bugs.darcs.net/issue1128

issue1131 Dmitry Kurochkin

accept download requests for different files.
http://bugs.darcs.net/issue1131

Patches applied in the last week (35)

See text entry for details.

darcs weekly news #6

2008-10-02T18:32:00.000+02:00

News and discussions

Third pre-release of darcs 2.1.0. Release pushed back to 17 October latest for more testing. We're getting very close to the finish line!
- http://www.darcs.net/darcs-2.0.3pre3.tar.gz
- http://lists.osuosl.org/pipermail/darcs-users/2008-October/014218.html
Darcs ideas in other VCS. Kirill Smelkov has kind words for us on behalf of the NAVY project, which is moving away from darcs. Best of luck to Kirill with whatever revision control system NAVY choose! While we are delighted that "Good ideas behind [darcs] were adopted by youth", we still have a thing or two to show these whippersnappers.
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/014095.html
Haddock + Hoogle == Javadoc on steroids. Simon Michael has combined haddock and hoogle to give us a lovely darcs code browser. In the meantime, Florent Becker has been adding value to this browser by sending in lots of haddock patches. Many thanks to Simon and Florent!
- http://joyful.com/repos/darcs-sm/api-doc
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/014084.html
Patch theory update. Ian gives us his latest progress on documenting, prototyping and improving darcs patch theory. "[S]ome proofs are finally starting to appear, albeit rather handwavey for now". Go Ian!
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/014168.html

Reviewers

Thanks to our patch reviewers for this week for giving David a hand!

Simon Michael

Issues resolved in the last week (5)

issue1003 David Roundy

don't sever transfer-mode connection on IO error.
http://bugs.darcs.net/issue1003

issue1043 David Roundy

fix bug in mergeAfterConflicting.
NB. half-resolved, more to follow!
http://bugs.darcs.net/issue1043

issue1078 Dmitry Kurochkin

make ioAbsolute work with symbolic links in file paths.
http://bugs.darcs.net/issue1078

issue1102 Eric Kow

recognise but do not generate patch log 'junk'.
http://bugs.darcs.net/issue1102

issue1110 David Roundy

fix get --hashed.
http://bugs.darcs.net/issue1110

Patches applied in the last week (47)

See text entry for details.

darcs weekly news #5

2008-09-25T12:28:00.001+02:00

News and discussions

Second pre-release of darcs 2.1.0 (formerly known as 2.0.3) This version of darcs will produce darcs-2 format repositories by default
- http://www.darcs.net/darcs-2.1.0pre2.tar.gz
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013977.html
New issue manager - Thorkil Naur. The darcs team now has an official Issue Manager role. Thorkil will be ensuring that incoming reports are responded to in a timely manner, and that all bugs are eventually moved to a resolved state.
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013887.html
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013920.html
Hoogling the darcs source?
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013976.html

Issues resolved in the last week (5)

issue27 David Roundy

add junk to patch identifiers.
http://bugs.darcs.net/issue27

issue53 Eric Kow

check for windows filename validity in darcs add/mv.
http://bugs.darcs.net/issue53

issue805 David Roundy

make darcs-2 format the default for new repositories.
http://bugs.darcs.net/issue805

issue1039 Dmitry Kurochkin

detect seemingly unrelated repositories when doing push, pull and send.
http://bugs.darcs.net/issue1039

issue1041 Vlad Dogaru

Get: if URL is invalid, direcotry is not created (#1041)
http://bugs.darcs.net/issue1041

Patches applied in the last week (54)

See text entry for details.

darcs weekly news #4

2008-09-18T13:18:00.003+02:00

News and discussions

First pre-release of darcs 2.0.3. This version of darcs has some very nice bug fixes on offer. A few more user-friendliness tweaks are planned for the actual release.
- http://www.darcs.net/darcs-2.0.3pre1.tar.gz
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013839.html
Third venue confirmed for darcs hacking sprint, 25-26 October. Brighton, Portland and now Paris are all CONFIRMED. Come hack with us!
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013710.html
code.haskell.org upgrades to darcs 2! /usr/bin/darcs is now darcs 2.0.2 on this server. No action is needed on the user's part.
- http://article.gmane.org/gmane.comp.lang.haskell.general/16419
Retiring GHC 6.4. Nobody seems to be using GHC 6.4 to compile darcs after all, so we shall be dropping support for it.
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013731.html

Reviewers

Thanks to our patch reviewers for this week for giving David a hand!

Jason Dagit
Nathan Gray
Trent W. Buck

Issues resolved in the last week (6)

issue691 Dmitry Kurochkin

distinguish between NoArg and ReqArg in defaults parser.
http://bugs.darcs.net/issue691

issue709 David Roundy

avoid adding changes to pending in rmpend when possible.
http://bugs.darcs.net/issue709

issue885 David Roundy

fix patchSetToRepository to work with hashed.
http://bugs.darcs.net/issue885

issue1012 David Roundy

it seems to be fixed by the fix for issue709.
http://bugs.darcs.net/issue1012

issue1054 Dmitry Kurochkin

--no-cache option to ignore patch caches.
http://bugs.darcs.net/issue1054

issue1057 David Roundy

this was fixed in the previous patch.
http://bugs.darcs.net/issue1057

Patches applied in the last week (86)

See 2008-09-17 text entry for details

darcs weekly news #3

2008-09-10T11:50:00.001+02:00

News and discussions

Venues confirmed for the darcs hacking sprint, 25-26 October. Brighton and Portland are CONFIRMED; Paris is likely. Come hack with us!
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013575.html
Planning darcs 2.0.3. We have started making steps towards a release for the end of September. Eric thinks we are only a buildbot and couple of bugfixes away from a prerelease.
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013516.html
Darcs patch theory. Ian Lynagh continues his patch theory research. He has written up a nice explanation and a working prototype of a darcs-like patch theory.
Retiring GHC 6.4. The darcs team would like to know if anybody is still using GHC 6.4 to compile darcs, so that we can focus on later versions (6.6 and 6.8).
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013659.html

Reviewers

Thanks to our patch reviewers for this week for giving David a hand!

Jason Dagit
Trent Buck

Issues resolved in the last week (7)

issue844 David Roundy

darcs doesn't guarantee always to compress patches.
http://bugs.darcs.net/issue844

issue924 Eric Kow

test for QuickCheck2 in disttest
http://bugs.darcs.net/issue924

issue1015 Ganesh Sittampalam

workaround internal API change on Windows in GHC 6.8.3
http://bugs.darcs.net/issue1015

issue1037 Dmitry Kurochkin

compile with -D_REENTRANT.
http://bugs.darcs.net/issue1037

issue1049 David Roundy

undo lazy patch bundle scanning.
http://bugs.darcs.net/issue1049

issue1050 Eric Kow

Disable haskeline by default.
http://bugs.darcs.net/issue1050

issue1063 Eric Kow

Wrap pwd in cygpath in shell tests.
http://bugs.darcs.net/issue1063

Patches applied in the last week (72)

See the darcs weekly news #3 email for the full list.

darcs hacking sprint (25-26 October 2008)

2008-09-06T10:52:00.002+02:00

Some news on the darcs hacking sprint. We have at least two venues confirmed, hopefully three shortly.

Venues

We plan to host the sprint across three sites:

CONFIRMED: Brighton, UK (University of Brighton)
CONFIRMED: Portland, USA (Galois)
likely: Paris, France (Université Paris Diderot)

So if you were waiting to book tickets, this is the time!

For more details, please see http://wiki.darcs.net/index.html/Sprints

Agenda

During this first sprint, we shall be focusing our attention on the day to day performance issues that darcs users commonly face.

This is what we are reaching for:

Fast network operations. We want to make it very pleasant for users to darcs get a repository and pull some patches to it over http and ssh. Git does this very well, and we plan to learn from them.
Cutting memory consumption. We want to profile the heck out of operations like darcs record, darcs convert and darcs whatsnew. What's eating up all the memory? And how can we can cut it down to size?
Responsiveness. Sometimes basic darcs commands can take long enough for programmers to lose their train of thought. We want to track down these lost seconds and kill that dreaded context switch.

Of course, if you are interested in other areas, then you can work on those instead.

Note that if you are new to the darcs code or to Haskell, there will also be a lot interesting jobs for you to get started with. Everyone will have something to hack on, so come join us!

Thanks very much to the University of Brighton, Galois and University of Paris VII for their generous offers.

Hope to see you there, everyone! :-)

darcs weekly news #2

2008-09-03T12:42:00.001+02:00

News and discussions

Growing the darcs team: The darcs unstable repository is coming back, with David Roundy as its maintainer. Eric will be taking care of stable and keeping in closely in synch.
- http://lists.osuosl.org/pipermail/darcs-users/2008-September/013388.html
Shiny new IRC logs: Thanks to Moritz Lenz, the #darcs and #darcs-theory IRC channels are now being logged with fancy formatting and search capability
- http://irclog.perlgeek.de/darcs
- http://irclog.perlgeek.de/darcs-theory
Hacking darcs: Petr Ročkai shares his recent hopes and experiences as a darcs user turned developer. Come share the excitement!
- http://mornfall.net/blog/hacking_darcs.html

Reviewers

Thanks to our patch reviewers for this week for giving David a hand!

Jason Dagit
Nathan Gray
Eric Kow
Petr Ročkai

Issues resolved in the last week (1)

issue966 Dmitry Kurochkin: fix apply_inv_to_matcher_inclusive. http://bugs.darcs.net/issue966

Patches applied in the last week (37)

2008-08-31 David Roundy

don't show ssh stderr output unless we're passed --debug.
fix bug in --list-options (tab completion).
fix bug in makeRelative.

2008-08-30 Ganesh Sittampalam

add warning to configure about Haskell zlib speed
make use of Haskell zlib dependent on bytestring
add option to use Haskell zlib package

2008-08-22 Eric Kow

Remove unused FileSystem module.
Add a link to a repository browser for darcs's code.

2008-08-29

Replace grep invocation by perl code

2008-08-24 David Roundy

clean up network/get.sh test.
fix type of withRepository and friends.
fix recent bug in --list-options.

2008-08-28 Dmitry Kurochkin

Check for package random on windows, used in Ssh module.
Debug messages in curl module.

2008-08-28 David Roundy

TAG working version.

2008-08-27 Dmitry Kurochkin

Use InclusiveOrExclusive instead of Bool in apply_inv_to_matcher.

2008-08-27 David Roundy

add more modules to make witnesses.

2008-08-27 Jason Dagit

updates to Darcs.Patch.Unit for type witnesses

2008-08-27 Dmitry Kurochkin

Refactor get_matcher and apply_inv_to_matcher functions from Darcs.Match module.
Resolve issue966: fix apply_inv_to_matcher_inclusive.
Simplify withCurrentDirectory.

2008-08-27 Jason Dagit

updates to Sealed.lhs to support type witness refactor in commands
updates to Ordered.lhs to support type witness refactor in commands
make Annotate.lhs compile with type witnesses

2008-08-27 David Roundy

fix type witnesses in Internal.

2008-08-27 Jason Dagit

updates to Repository.Internal to fix conflicts and support type witness refactor in commands
fix error in Properties due to new commuteFL
fix minor type witness compile error with new commuteFL
fix conflicts with get_extra changes
improve reporting for bug in get_extra
Finish refactor of Unrevert as well as making it pass double-unrevert.sh
add double-unrevert.sh test
partial type witnesses in Unrevert

2008-08-26 Eric Kow

More ChangeLog entries since 2.0.2

2008-08-27 David Roundy

fix bug in defaultrepo.

2008-08-26 Jason Dagit

fix accidental reversal in tentativelyAddToPending
minor refator to get_extra improve comments

simple random numbers in Haskell

2008-07-30T14:03:00.018+02:00

Random numbers are the kind of thing I use rarely enough that by the time I want to use them, I have forgotten the relevant details, but frequently enough that I get annoyed whenever it happens.

Hopefully these notes will be useful to somebody in a similar situation.

two things to know

(1) import System.Random

(2) randomIO :: Random a => IO a

The one function you really need to know about is randomIO (The type of this function is Random a => IO a. Don't worry if you do not understand the type; it suffices to know that it involves IO). In this example, we use and generate a random Int:

import System.Random

main =
 do r <- randomIO
    print (r + 1 :: Int)
    -- Note re the ':: Int' above: Haskell can't figure out from
    -- the context exactly what type of number you want, so we
    -- constrain it to Int

One neat feature is that you can randomly generate anything that implements the Random typeclass. In the example below, we generate a random Bool. Notice how we do not do anything differently, except to treat the result as a bool (i.e. by applying not to it)

import System.Random

main =
 do r <- randomIO
    print (not r)

A useful exercise, if you know about typeclasses, is to implement Random for one of your own types. The toEnum function may be useful.

more advanced stuff

you can use randomRIO :: Random a => (a,a) -> IO a to generate random numbers constrained within a range
Instead of using the functions randomIO and randomRIO, you can separate obtaining a random number generator, from using the generator. Doing so allows you to minimise your reliance on the IO monad. It also makes your code easier to debug, because you can opt to always pass the same generator to it and make life much more predictable. See the functions random and randomR for details.
a potentially handy trick is to generate an infinite list of random numbers, which you can then pass to a function. See the randoms function for details.

Edit: fixed s/randomR/randomIO/

pandoc gets mediawiki support

2008-07-27T07:02:00.002+02:00

Pandoc is a universal document converter. You feed it documents in one format (say, HTML) and it spits them out in another one (say, ODF). Assuming it works correctly, Pandoc has the potential to replace all those little one-to-one convertors (e.g. latex2html) in my toolbox. Just the one simple Pandoc.

And now, thanks to fiddlosopher (John MacFarlane?), it knows how to write Mediawiki files! Mediawiki? That's the syntax/software that powers Wikipedia, Wikibooks, and a whole slew of organisational or community wikis (like HaskellWiki).

Hey, Haskellers probably have a lot of LaTeX documents lying around. Maybe this is their chance to get them on Haskell wiki?

We're halfway to being able to do a roundtrip between LaTeX and Mediawiki! All we need is for somebody [maybe John :-D] to implement a Mediawiki reader for Pandoc and things could get mighty interesting... Oh and yes, and if anybody is working on a wiki with direct LaTeX support, hats off to you! Sometimes Mediawiki is a fact of life, though.

rose zipper on hackage

2008-07-23T18:01:00.004+02:00

I guess this isn't big enough to go on the haskell@ mailing list: I have uploaded Krasimir Angelov and Iavor S. Diatchki's Data.Tree implementation of zippers onto hackage. The package is called rosezipper and it is available under the BSD3 license.

For the interested, "The Zipper is an idiom that uses the idea of “context” to the means of manipulating locations in a data structure." (Haskell wiki).

For me, zippers are just a very nice way to navigate and edit trees. By "nice", I mean elegant, efficient and purely functional. Before learning about zippers, I only knew how to navigate trees from top to bottom, but if I wanted to go back up a node, or visit a sibling node, I basically had to start over from the root. Zippers allow me to walk the tree in any direction, visiting a node's parent, children and siblings without starting over from the top. This kind of thing is especially handy for Natural Language Processing people, basically, anybody who eats trees for a living.

If you would like to learn more, I would recommend Apfelmus's very friendly tutorial (part of the Haskell wikibook).

Thanks to Krasimir and Iavor for implementing this and for allowing me to package it up.

encodings-aware hex editor

2008-07-22T12:10:00.007+02:00

Here's another coding-project idea: I would like to see a hex editor that knows how to display characters in other encodings than ASCII (specifically: I want to debug messed up UTF-8 text files).

Google and apt-cache search reveal no such editor, at least not in the free/open-source worlds, nowhere in Linux or MacOS X freeware land. On Debian based systems, there are a couple that handle some Japanese encodings, but nothing that deals with UTF-8.

Likely features:

toggle between an ASCII-only mode and a show-as-UTF-8 mode
good UI for the fact that UTF-8 characters have a variable length in bytes
graceful handling of encoding errors

Haskellers could possibly do this as a part (plugin?) of Yi, or maybe just a completely standalone product.

And if you want a slightly simpler project, a UTF-8 hex dumper would be good. Hmmph... come to think of it, maybe it would have been more productive to just go write that instead of this blog post.

Edit: Well, I went ahead and made a stupid little dumper for my needs. Here is the output on some sample corrupted UTF-8

20 28 5b 47 65 6f 72 67 69 61 6e                     ([Georgian
3a 20 e183a1 e183 3f e183a5 e183                    : ს«e1 83»?ქ«e1 83»
3f e183 20 e18397 e18395 e18394 e1839a e183         ?«e1 83» თველ«e1 83»
3f 5d 0a                                            ?]

20 28 5b 47 65 72 6d 61 6e 3a 20                     ([German:
44 65 75 74 73 63 68 6c 61 6e 64                    Deutschland
5d 20 5b 49 50 41 3a 20 cb88 64 c994                ] [IPA: ˈdɔ
c9aa 74 ca83 6c 61 6e 74 5d 29 2c 20                ɪtʃlant]),
6f 66 66 69 63 69 61 6c 6c 79 20                    officially
74 68 65 20 46 65 64 65 72 61 6c                    the Federal
20 52 65 70 75 62 6c 69 63 20 6f                     Republic o
66 20 47 65 72 6d 61 6e 79 20 28                    f Germany (
42 75 6e 64 65 73 72 65 70 75 62                    Bundesrepub
6c 69 6b 20 44 65 75 74 73 63 68                    lik Deutsch
6c 61 6e 64 2c 20 5b 49 50 41 3a                    land, [IPA:
20 cb88 62 ca8a 6e 64 c999 73 72 65 70               ˈbʊndəsrep
75 62 6c 69 cb 3f 6b 20 cb88 64                     ubli«cb»?k ˈd
c994 c9aa 74 ca83 6c 61 6e 74 5d 29 2c              ɔɪtʃlant]),
20 69 73 20 61 20 63 6f 75 6e 74                     is a count
72 79 20 69 6e 20 43 65 6e 74 72                    ry in Centr
61 6c 20 45 75 72 6f 70 65 2e 20                    al Europe.
0a

Highlighting by hand. I should probably go figure out how to colourise the corrupted characters. Or maybe I should just go ahead and package this, put it up on hackage? Make it available via darcs? I would need a decent name. So far, I have hexy-xxy and hexdump-utf8 neither of which are that great :-/

simply reading and writing UTF-8 in Haskell

2008-07-21T11:43:00.009+02:00

A year and a half ago, I posted what seemed to be the simplest recipe for reading and writing UTF-8 in Haskell. In this post, I will provide an even simpler recipe, made possible by Eric Mertens' utf8-string package.

For those who are not familiar with Haskell, its internal representation for characters is Unicode, but for IO it effectively assumes that that it is reading and writing in the ISO8859-1 format. This used to be annoying for those of us who wanted to work with the UTF-8 encoding, but now there is a very simple solution, perfect for those of us who don't want to think too much and just get the job done.

the example

The sample problem from my last post was to take a UTF-8 encoded file as input, reverse all its lines, writing the results in the same file, with a ".rev" extension appended to its name. The solution might be self-explanatory if you are used to Haskell, but I will make some minor comments below, just in case.

import System.IO.UTF8
import Prelude hiding (readFile, writeFile)
import System.Environment (getArgs)

main =
 do args <- getArgs
    mapM_ reverseUTF8File args 

reverseUTF8File f =  
 do c <- readFile f
    writeFile (f ++ ".rev") $ reverseLines c

reverseLines = unlines . map reverse . lines

In the above code, we use some drop-in replacements for some System.IO functions. Some of these functions are also provided in the Prelude, so we must hide them so that they do not overlap with what we import. (Alternatively, we could import the UTF-8 ones qualified, which could be handy in contexts where we want the option of reading and writing in UTF-8 without committing to it). The rest is straightforward. Notice that we do not jump through any hoops whatsoever. In fact, you can pretty much take any pre-existing Haskell program that you have written and turn it into a UTF-8 version by changing the import statements.

Here are the results of running this script on a UTF-8 sampler:

)udrU( یتوہ ںیہن فیلکت ےھجم روا ںوہ اتکس اھک چناک ںیم 
)othsaP( يوږوخ هن ام هغه ،مش ېلړوخ هشيش هز 
)naeroK(요아않 지프아 도래그 .요어있 수 을먹 를리유 는나
)keerG( .ατοπίτ ωθάπ αν ςίρωχ άιλαυγ ανέμσαπσ ωάφ αν ώροπΜ
)cidnalecI / aksnelsÍ( .gim aðiem ða sseþ ná relg ðite teg  gÉ
)hsiloP( .izdokzs ein im i ,ołkzs ćśej ęgoM
)nainamoR( .etșenăr ăm un ae iș ălcits cnânăm ăs toP
)nainiarkU( .ьтидокшоп ен інем онов й ,олкш итсї ужом Я
)nainemrA( ։րենըչ տսիգնահնա իծնի և լետւո իկապա մանրԿ
)naigroeG( .ავიკტმ არა ად მაჭვ სანიმ
)idniH( .तह हन डप ईक स सउ झम ,ह तकस ख चक म
)werbeH( .יל קיזמ אל הזו תיכוכז לוכאל לוכי ינא
)hsiddiY( .ײװ טשינ רימ טוט סע ןוא זאלג ןסע ןעק ךיא
)cibarA( .ينملؤي ل اذه و جاجزلا لكأ ىلع رداق انأ
)esenapaJ( 。んせまけつ傷を私はれそ。すまれらべ食をスラ
)iahT( บจเนฉหใำทมไนมตแ ดไกจะรกนกนฉ
)slobmys ycnerruc( ₯·₮·₭·₫·₪·₩·₨·₧·₦·₥·₤·₣·₢·₡·¢·$·€·£·¥

The utf8-string package is available on HackageDB. Thanks to Eric M. for providing this little wrapper! It's a perfect example of the kind of thing which seems obvious... after somebody else has thought to do it.

recurring problem (boring text file merging)

2008-05-13T15:49:00.007+02:00

I keep solving variations of this problem at work, whether I'm trying to merge some log files together, or identify token offsets with bits of parse tree. I had better jot it down so that I don't forget there may be something more general hidden behind all this.

mergeFoo :: [a] -> [(Int,Int,b)] -> [Either a ([a],b)]

I'm not necessarily looking for a solution -- I could just boil one out from my previous solutions -- but I am at least officially and publicly reminding myself that I shouldn't keep solving the same thing over and over again (unless I'm engaged in some kind of lateral thinking exercise, which is a different story)

lispparser on hackage

2008-05-09T18:34:00.006+02:00

Ever wanted a LISP S-expressions parser?

I have. I do some natural language processing work, where some people like to output parse trees as S-expressions. Very natural. But then I always balk because I have to go whip up a little parser for it, which I know to be easy in principle, but... well, you know how that goes.

Anyway, if you're at my level of programming mediocrity, the one where "write an S-expressions parser" makes you think "I know this is easy, but do I have to?", then perhaps the lispparser package is for you! I guess this is too minor a package to warrant a mailing list announcement, but I've taken a bit of Jonathan Tang's tutorial code and put it on hackage as lispparser. If you think it needs improvement, I might consider putting a darcs2 repository online somewhere.

lingscore

2008-05-05T14:52:00.004+02:00

A little bit work-related. In a mail that I'm about to send out to the Corpora mailing list:

We're looking for implementations of scoring algorithms for coference resolution. Specifically, the algorithms we are interested in are MUC-6 (Vilain et al., 1995), B-CUBED (Bagga and Baldwin, 1998), and CEAF (Luo, 2005).

Our hope is to compare a few pieces of coference resolution software. Does anybody have preferably standalone software that we could use to calculate these scores?

I am sorely tempted to just sit down for a few moments and create these (scoring) tools myself.

It'd be a small Haskell package called 'lingscore', probably a library and an executable. I'd stick the scorers under the 'NLP.Evaluation' package. The library would be dedicated to NLP evaluation algorithms. No actual NLP, just the scoring algorithms for evaluation campaigns. Should not be difficult, and would very slightly advance the agenda of making Haskell a viable platform for NLP-hacking.

I quite like the idea of using Haskell for the stupid reason that type signatures make it a bit clearer what kind of inputs we're expecting and what kind of outputs we can produce.

short day (reg 5h)

2008-04-28T18:49:00.004+02:00

Edit 23:16 - Argh! This post was meant for koweynlg, my daily work diary. Not so much for public consumption (like koweycode), but public to keep me honest. Sorry for the noise (and the meta noise).

Got here at 10h, leaving at 18h; various tea and cake breaks in between. Almost done with the REG stuff. Got my tri-text framework setup (more and more scripts, some of them throwaway). I am very grateful for the automation, am thanking past-eric (of 1 week ago) for making the results file generator. Trying to do this by hand [what was I thinking] would have suicide.

Back to reading Grosz and Sidner.

More REG stuff tomorrow. Not done yet.

native speakers wanted

2008-04-28T11:55:00.003+02:00

Sorry, this is a little off-topic for koweycode, but we would be interested to have some native speakers participating in a little experiment.

All you have to do is to read some encyclopaedic texts and plug some stuff into drop-down-boxes along the way. There are NO WRONG ANSWERS. We want to see your answers and to learn from them. It only takes a few minutes (but you can do as much of it as you want).

We are now running the second instalment of an experiment in which we ask people to select referential expressions (REs) that refer to the main subject in the context of a simple encyclopaedic text. The idea is to investigate to what extent people agree when choosing REs. The experiment is designed as a multiple choice task, where REs can be selected from a menu. The texts are short and can usually be done in under a minute.

We would like to ask any native speakers of English who have a few minutes to spare to help us complete the experiment. It would be great if participants could do at least three texts, but you can do as many as you like.

There is more information on the experiment website. To participate, simply read the instructions and then click on the 'start experiment' button at the bottom of the page:

http://www.nltg.brighton.ac.uk/home/Anja.Belz/CMSR

Any non-native speakers who would like to try out the experiment can do so at this alternative website:

http://www.nltg.brighton.ac.uk/home/Anja.Belz/TESTDRIVE

We would be very grateful for any feedback, comments and suggestions.
Many thanks for your time,

Anja Belz

Regretfully, this has not been coded up in Haskell. But surely that is forgivable :-)

darcs 2 at last!

2008-04-11T09:07:00.003+02:00

I'm sure you've all seen David's announcement: darcs 2.0.0 is out!

what's good

In short, darcs 2.0 is safer and faster.

Particularly, the dreaded exponential-time merge bug has now been largely resolved. Let me say it more carefully: while it may still be possible to run into exponential time merges, our improvements to conflict-handling should make it considerably less common. We hope that nobody ever runs into such a situation in practice.

Other key points are improved the hashed inventory and pristine cache which darcs more robust (you no longer have to worry about third party tools like Eclipse or Unison messing things up by mucking around with darcs internals), the ssh-connection mode which speeds up SSH-issues a lot and kills the typing-your-password-ten-million-times issue dead (at most you'll have to type it in twice).

what's bad

On the one hand, darcs 2.0.0 should be much smoother and faster for most users. On the other hand, people with large repositories (e.g. GHC-sized) might find certain operations to be somewhat slower. David does not (yet) have ideas on how to make things better for such users, and is even recommending them to switch to something else. If you've got a repository of darcs' size (over 5000 patches, 6 years, 131 contributors) or smaller, you should continue using darcs, because we still think it works better: we're still the only ones around to offer deep cherry picking... something which we think would be hard to do without radically changing the way other VCSes work. If you would like to prove us wrong, please do so and we would be most grateful!

Also, taking advantage of darcs 2 will require you to upgrade your repository to the darcs-2 format (see darcs convert), which unfortunately, is not compatible with older versions of darcs. People with new repositories should definitely start using this format. People with old repositories should probably do so at the earliest convenient moment, although this means your users will have to upgrade. Please switch to the new format. It will make everybody's lives easier.

The final piece of bad news: we're going to have to shift to a lighter weight development model, something which puts less strain on David and the rest of the contributors. The consequences are that patches might get less review [one maintainer and not two], and that you'll be seeing less of David on the mailing lists. The good news in the bad news is that our lighter weight development model is now being supported by increased automation of the administrative stuff. For example, our bug tracker is now integrated with the darcs repository so that it automatically knows when a ticket has been resolved by a patch. This increased automation gives us extra rigour and more time to think about making darcs better. The only thing we need is more of us. If you want a place to hone your Haskell, Perl or C... or if you think you know a thing or two about user interfaces, please spend some time with us.

to sum up...

Have you been hesitating to try darcs out? Well, now is a good time to do so, as our killer bugs have been fixed as well as the kind of minor nuisances that get most of us. Or... are you looking for something to work on? Uncle David needs you!

[note: Thanks to David Roundy for comments on a draft of this post]

wxHaskell 0.10.3 is out!

2008-04-02T21:04:00.005+02:00

As you may have noticed in Jeremy's announcement, wxHaskell 0.10.3 is now available for download. This version offers the following key improvements over 0.9.4:

Support for Unicode builds of wxWidgets
Support for wxWidgets 2.6.x (support for wxWidgets 2.4.2 retained if
you compile from source)
Support for building with GHC 6.6.x and 6.8.x
Parts of wxHaskell are now built with Cabal
Profiling support
Smaller generated binary sizes (using --split-objs)

At the moment, wxHaskell works with wxWidgets 2.6 (the previous stable release, still widely available). But we're working to get you wxWidgets 2.8 support as soon as we can.

In the meantime, here's a screenshot of the wxFruit paddle-ball example running on my Mac. (Thanks to shelarcy for putting up the package!)

This was painlessly installed with the help of cabal-install. You don't have to use cabal-install -- the wxHaskell sourceforge site includes binaries -- but it may make your life easier for dealing with dependencies. The only caveat you might want to know about when using Cabal to install wxHaskell and its companion libraries are that wxcore should be installed as root, i.e. sudo cabal install wxcore.

XTC on hackage

2008-03-15T19:58:00.013+01:00

Just a quick note to say that XTC (XTC: eXtended & Typed Controls for wxHaskell) is available on hackage.

the haddock

The XTC library provides a typed interface to several wxHaskell controls.

radio view (typed radio box)
single-selection list view (typed single-selection list box)
multiple-selection list view (typed multiple-selection list box)
choice view (typed choice box)
value entry (typed text entry)

XTC controls keep track of typed values and items, rather than being string based. Selections in XTC controls consist of actual values instead of indices.

my notes

XTC library was developed in Utrecht University, and has been used to develop Dazzle, a Bayesian Network toolbox, and very much a "real world" application. You can read more about XTC and Dazzle in their Haskell Workshop paper from 2005.

If you're using wxhaskell, XTC could make your code a bit cleaner, without imposing a steep learning curve. Here is a quick example of the library in action.

And here is the source code. Notice how we work directly with the Fruit type, eschewing any intermediary strings:

import Graphics.UI.WX
import Graphics.UI.XTC

data Fruit = Apple | Banana | Orange deriving Show

instance Labeled Fruit where
 toLabel = show

main :: IO ()
main = start $
  do f <- frame []
     txt <- staticText f [ text := "pick a fruit and I will give you a slogan" ]
     radioV <- mkRadioView f Vertical [Apple, Banana, Orange] []
     --
     set radioV [ on select :=
      do mf <- get radioV typedSelection
         set txt [ text := slogan mf ]]
     set f [ layout := margin 5 $ column 1
         [ hfill $ widget txt, widget radioV ] ]

slogan :: Fruit -> String
slogan Orange = "orange you glad I didn't say 'orange'?"
slogan Apple  = "an apple a day keeps, well you know"
slogan Banana = "buh-naaaaa-naaa"

If you like this kind of thing, be sure to also check out AutoForms and Phooey

wxhaskell 0.10.3rc1

2008-03-12T10:55:00.004+01:00

In case you missed Jeremy's announcement, the first release candidate for wxhaskell 0.10.3 is now ready for download.

Highlights of 0.10.3 rc1 include:

Support for Unicode builds of wxWidgets
Support for wxWidgets 2.6.x (support for wxWidgets 2.4.2 retained if
you compile from source)
Support for building with GHC 6.6.x and 6.8.x
Parts of wxHaskell are now built with Cabal
Profiling support
Smaller generated binary sizes (using --split-objs)

See Jeremy's message for a more complete list.

Note that we have postponed the goal of supporting wxWidgets 2.8. We are definitely making this a priority for wxhaskell 0.11. In the meantime, we will focus on making wxhaskell 0.10.3 easy for everyone to install. So please let us know if you run into any trouble. (Many thanks to Neil Mitchell and other users who have helped in testing)

Finally, users of MacOS X Leopard have reported difficulty building the older wxWidgets 2.6. According to the wxWidgets wiki page for more details), doing so is indeed possible, if you use the Tiger SDK. We would love to hear from you if you have succeeded in doing this.

operation Roundy Tears

2008-02-28T18:31:00.003+01:00

I have a very serious issue that I'd like to raise with the darcs and Haskell communities: you're not being evil enough.

Darcs2 is getting closer and closer to completion (I am not saying this in any official capacity), but you've all been pretty complacent about making it hurt. Sure, some of you have done performance testing, for which thanks, and yes, some of you have thrown in a couple of conflict related tests. But the closest we have ever come is a

darcs: src/Darcs/Patch/Real.lhs:422:21-50: Irrefutable pattern failed for
pattern Data.Maybe.Just a2'

And that was fixed within a week.

This isn't good enough. Be more evil! Submit tests to our bugs/ directory. Think of devious conflicting ways to make darcs fall down.

Make... David... cry...

Please.

[note: the best way to participate in Operation Roundy Tears is to use the --darcs-2 format; you can get a darcs2 repository from a darcs1 repository by using darcs convert]

wxhaskell components and news

2008-02-24T19:57:00.005+01:00

So, I made this little diagram showing the basic components of wxhaskell. It might be not entirely correct, but I hope it will be useful for anybody who wants to help out.

Also, as you might have noticed, wxhaskell is now on hackage as an experimental pre-release. Let me know if you have any trouble building it, or getting it to run sample applications. There's still a few painful bits, (1) you still have to use wxWidgets 2.6 and not the newer 2.8 [we're working on it] (2) that it assumes your wxWidgets is compiled with --enable-mediactrl (this should be relatively easy for us to fix and (3) for Linux, wxcore 0.10.2 requires that you configure wxhaskell --with-opengl (the darcs version fixes this) and (4) for Windows... well I don't know; shelarcy can build it just fine using Visual Studio, I think and the darcs version of cabal should now be happier with our Cabal files.

In other news, we're now much more disciplined about using the wiki to note problems installing wxhaskell and to propose solutions. We're also now paying closer attention to the bug tracker (triaging them), and have made it much easier for users to submit bug demonstrators (see our bugs/ directory). I hope these new habits will make us more responsive.

So we're not really ready for an official release, but we're getting closer. I'm hoping we get there sooner rather than later. I want to see more people playing with neat tools like Phooey and Autoforms, both of which are on hackage.

maybench underway

2008-02-22T11:48:00.006+01:00

The maybench project (formerly checkquick) is now underway. Maybench is a tool for comparing the performance between two versions of the same program, on a series of benchmarks that you design. Maybench aims to be easy to use, almost as easy as running time your-program arg1..arg2. Ideally, it should be very straightforward for outsiders to write timing tests for your programming project and contribute them as part of your performance testing suite.

We have a Google Code page, a mailing list and a darcs repository:


  darcs get http://code.haskell.org/maybench

The repository basically consists of the preliminary code written by the tehgeekmeister and also some new code by ertai (see darcs-benchmark). I'm also hoping that some of the code written for nobench can be used, for example, to generate fancy reports. Right now, running maybench looks like this:

% dist/build/maybench/maybench 'sleep 5' 'sleep 3'
"sleep 3" took 60.0% of the time "sleep 5" took.

As a first priority, we're going to get maybench useable for benchmarking darcs. After that we'll start thinking of how to generalise it, so that it can be used for the Haskell benchmark suite, for example, or for your software.

Interested? Come join us!

darcs circa 2003

2008-02-18T19:24:00.005+01:00

There's always room for improvement.

Archaeological findings courtesy of Zooko on issue687.

Darcs exercise: can you find the original patch in the darcs darcs repository where this logo was introduced? My hope is that the darcs UI is friendly enough that you just say "duh, of course!"

checkquick

2008-02-08T10:44:00.001+01:00

Darcs needs a benchmarking tool. After 'stopwatcH', this is the first name I came up with, horrible as it is. I was thinking that surely, this is a general problem, so we should throw something up on hackage. The basic wish is to be able to run a program N times (comparing it with a different version of the same program).

Can you help? If you've got code to share, put it up! If not, and you want to contribute to project, comment on this blog. If nobody gets moving [and gives me a better name] I guess I'll create a 'checkquick' project, probably using code.google.com and code.haskell.org. If nobody submits anything and I really do have to start this project (note: I do not want to), I am hoping for a liberal commit model, where we pretty much hand out push rights to anybody who wants them.

I've already a little description written up, just in case:

Checkquick is a tool for comparing the performance between two versions of the same program, on a series of benchmarks that you design.

Checkquick aims to be easy to use, almost as easy as running 'time your-program arg1..arg2'. Ideally, it should be easy for outsiders to write timing tests for your programming project and contribute them as part of your performance testing suite.

It is written in Haskell and named after the illustrious, though wholly unrelated, quickcheck.

darcs at a glance

2007-12-23T09:41:00.000+01:00

Check out Paul Bedaride's very handy darcs cheet sheat. It gives you basically everything you need to know about darcs on a single sheet of A4 paper. I'm sure Paul would appreciate any appropriate criticism you may have.

mail?

2007-12-18T18:32:00.001+01:00

So we have in Haskell quite a few consumer applications

When's the mail client coming? I'm asking this in the random speculative sense, not in the "gimme a Haskell mail client now" sense. What would a Haskell mail client look like? I'm guessing that plugins will play a nice role, extensibility and all. Being able to handle really large mailboxes (a big "archive" folder) would be nice. Perhaps Sup would be good inspiration. Come to think of it, maybe the client should be implemented in Cat, just for the name...

Setup.(l)hs

2007-12-06T15:51:00.000+01:00

One minor annoyance when using Cabal to install stuff is that not every project agrees on whether the Setup script should be an .hs or an .lhs file. The consequence is that when doing a bunch of installations, I'll hit the up arrow to re-run, say, runhaskell Setup.lhs configure and it will complain that the file is not found, because this project uses .hs and the project just before uses .lhs.

What I'm wondering, though is if it would be a good idea for projects to distribute both files? We could even use something like this to have the same file regardless of extension:


#!/usr/bin/env runhaskell
{-

> import Distribution.Simple
> main = defaultMain

-}
import Distribution.Simple
main = defaultMain

I guess that once cabal-install comes out, none of this would really matter since all that building-of-dependencies stuff would be taken care of automatically.

David Roundy is the new darcs unstable maintainer

2007-12-06T15:07:00.001+01:00

Just wanted to announce that I am ending my stint as unstable branch maintainer and transferring the responsibility back to David. This will help me to keep up with my new life and job, and help David to usher in the new darcs-2.0.

I hope I'll be able to continue participating, maybe submitting a patch or two when things settle down over here. In the meantime, it's been fun and I've learned a lot.

So thanks for everything and good luck with 2.0!

PhD viva (defense)

2007-11-06T15:39:00.000+01:00

I'll be presenting my thesis next Wednesday. You are cordially invited to attend the defence and to join us for drinks afterwards.

Surface realisation: ambiguity and determinism
Eric Kow

14 November 2007
Amphi C, LORIA
Nancy, France

Surface realisation is a subtask of natural language generation. It may be viewed as the inverse of parsing, that is, given a grammar and a representation of meaning, the surface realiser produces a natural language string that is associated by the grammar to the input meaning. This thesis presents three extensions to GenI, a realisation algorithm for Feature-Based Tree Adjoining Grammar (FB-LTAG).

The first extension improves the efficiency of the realiser with respect to lexical ambiguity. It is an adaptation from parsing of the "electrostatic tagging" optimisation, in which lexical items are associated with a set of polarities, and combinations of those items with non-neutral polarities are filtered out.

The second extension deals with the number of outputs returned by the realiser. Normally, the GenI algorithm returns all of the sentences associated with the input logical form. Whilst these inputs can be seen as having the same core meaning, they often convey subtle distinctions in emphasis or style. It is important for generation systems to be able to control these extra factors. Here, we show how the input specification can be augmented with annotations that provide for the fine-grained control that is required. The extension builds off the fact that the FB-LTAG grammar used by the generator was constructed from a "metagrammar", explicitly putting to use the linguistic generalisations that are encoded within.

The final extension provides a means for the realiser to act as a metagrammar-debugging environment. Mistakes in the metagrammar can have widespread consequences for the grammar. Since the realiser can output all strings associated with a semantic input, it can be used to find out what these mistakes are, and crucially, their precise location in the metagrammar.

darcs conflicts faq

2007-10-29T15:38:00.001+01:00

Pekka Pessi and I have put together an FAQ on the darcs conflict problem. It has three major sections:

Everyday conflicts: what conflicts are and how you should deal with them in the general case.
The big conflicts bug: what it is, how you can avoid it, and what to do if you have run into trouble.
Darcs 2.0: from a user perspective, how it will change from darcs 1.x (not very much), and what resolving conflicts might look like (rollback will play a bigger role). David has been making some great progress. Nevertheless, our best estimates are that we will be ready for alpha testing only in Feb 2008, and release by the second quarter of that year.

There may be some mistakes or gaps in the FAQ, but I hope that you will find it useful.

getting things done with mutt 3 (action counter)

2007-10-24T20:42:00.000+02:00

Here's a short tip for putting a reminder on your desktop wallpaper of how many next-action and waiting-for messages you have. The result looks like this:

ACTION:  4
WAITING: 1

The approach is to combine a small shell script with the wonderful GeekTool program for MacOS X (similar programs exist for Linux, I'm sure). GeekTool lets you display arbitrary text, say console text on your Desktop. One of my GeekTool windows is set to run the script and display its results with a big red font.

The shell script is pretty dumb; it just counts the number of files in my Maildir directory. I guess if you're using the mbox format, you'll have to find some other way to count messages. Count the number of instances of '^From: ', maybe?

#!/bin/sh

# $2 is just for padding
function count_messages (){
  echo -n "$1:$2"; ls -1 ${HOME}/Mail/current/$1/cur | wc -l
}

count_messages ACTION " "
count_messages WAITING

I'm not expecting this to have a huge impact on my productivity, but maybe this will be that extra little bit that counts.

Edit 2007-10-25Fixed counting to account for empty boxes. Zero. We like that number.

kowthese 1.0

2007-09-21T17:10:00.000+02:00

I thought I might announce version 1.0 of my PhD thesis: Surface realisation: ambiguity and determinism

This is the version that went out to the committee, so it is not definitive (you may prefer to wait for version 1.1 in December). It's also not very Haskell-related, aside from the fact that the software inside is written in Haskell. If you are interested, the overall topic area is computational linguistics (natural language processing), specifically, natural language generation.

Oh, and the darcs repository for the thesis (LaTeX sources):

 darcs get http://www.loria.fr/~kow/kowthese

The viva/defense to be held on 2007-11-14.

getting things done with mutt 2 (auto-review)

2007-08-25T19:15:00.000+02:00

Last year, I posted some ideas for applying GTD to mutt. As you may have heard, GTD is a simple methodology to help people stay on top of things. The basic idea in my post was to use an on-board X-Label editor to associate each message with a 'next action' or a 'waiting for' tag, and to store them in the respective mailboxes ACTION and WAITING. This was accompanied by a small colour configuration to highlight the X-Label field of each message, thus making it clear at all times what the next action was.

After one year of use, I can report that I am 95% happy with the system. It is simple and effective... however, not entirely eric-proof. In this post, I propose a small addition to tighten up the system and make it more resistant to my foolishness. The problem is that GTD is a heavily review-oriented system. Once you move tasks out of your head and onto some external device (e.g. a pad of paper), you must also consult that device from time to time or risk forgetting to do them. For example, one thing that can easily happen to me is that I will move messages into ACTION and WAITING and simply forget they are there.

This is where the idea of automated review comes in. What I propose a simple method for reminding yourself that you have next actions to perform, or things that you are waiting on. It consists of a shell script and a crontab entry. First the script (I call it gtd-review):


#!/bin/sh

# note: I am using the maildir format; if you are using mbox, 
# you should just replace the 'find $1 | xargs $1' with 'cat $1'
function summarise () {
  find $1 | xargs cat |\
    sed -n -e '/^X-Label/G' -e '/^X-Label/p'\
           -e '/^From/p'    -e '/^Subject/p' -e '/^Date/p' |\
    sed -e 's/From: //' -e 's/Date: //' -e 's/Subject: //'
}

echo '======================================================================'
echo 'ACTIONS'
echo '======================================================================'
echo
summarise ${HOME}/Mail/ACTION

echo '======================================================================'
echo 'WAITING'
echo '======================================================================'
echo
summarise ${HOME}/Mail/WAITING

And now the crontab:

@daily gtd-review 2>&1 /dev/null | mail -s "GTD review `date +%Y-%m-%d`" me@myaddress.com

I'm sure you could improve on this. For example, I would rather the dates were presented in yyyy-mm-dd format and accompanied with a friendlier description like "3 weeks ago"... but working on that would probably count as fidgeting.

Anyway, I hope others find this to be useful.

a history of monad tutorials

2007-08-16T14:20:00.000+02:00

Here's a historical overview of monad tutorials since Phil Wadler's original observation that monads can be implemented in Haskell and become extremely useful.

When I wrote this, I originally wanted to do a real history, with an analysis of how people have tried to teach monads over the years, but I guess this is about all I have time for. Dates, authors and blurbs. Corrections/additions always welcome! As you can tell, I have not read them all.

Edit 2007-08-17: I have updated and moved this timeline to Haskellwiki. This might be useful when some future Haskell archeologist tries to figure out the precise "ah-ha!" moment when every single programmer in the world 'got' monads.

Haskell 是一门函数式编程语言。

2007-08-14T16:35:00.000+02:00

Looks like some wikibookian(s) have embarked upon a Chinese translation of the Haskell wikibook. Good luck to them! Chinese speakers might be interested in jumping on board. As for non Chinese speakers, the sentence above is "Haskell is a functional programming language."

monomorphism redux

2007-07-19T13:45:00.000+02:00

There may be errors in this, of course!

the context from wikibooks

Following the previous example, you might be tempted to try storing a value for that radius. Let's see what happens:

Prelude> let r = 25
Prelude> 2 * pi * r

<interactive>:1:9:
    Couldn't match `Double' against `Integer'
      Expected type: Double
      Inferred type: Integer
    In the second argument of `(*)', namely `r'
    In the definition of `it': it = (2 * pi) * r

Whoops! You've just run into a programming concept known as types. Types are a feature of many programming languages which are designed to catch some of your programming errors early on so that you find out about them before it's too late. We'll discuss types in more detail later on in the Type basics chapter, but for now it's useful to think in terms of plugs and connectors. For example, many of the plugs on the back of your computer are designed to have different shapes and sizes for a purpose. This is partly so that you don't inadvertently plug the wrong bits of your computer in together and blow something up. Types serve a similar purpose, but in this particular example, well, types aren't so helpful.

the new monomorphism-based explanation

The quick solution to this problem is to specify a type for the number 25. For lack of other information, Haskell has "guessed" that 25 must be an Integer (which cannot be multiplied with a Double). To work around this, we simply insist that it is to be treated as a Double

Prelude> let r = 25 :: Double
Prelude> 2 * pi * r
157.07963267948966

Note

There is actually a little bit more subtlety behind this problem. It involves a language feature known as the monomorphism restriction. You don't actually need to know about this for now, so you can skip over this note if you just want to keep a brisk pace. Instead of specifying the type Double, you also have given it a polymorphic type, like Num a => a, which means "any type a which belongs in the class Num". The corresponding code looks like this and works just as seamlessly as before:

Prelude> let r = 25 :: Num a => a
Prelude> 2 * pi * r
157.07963267948966

Haskell could in theory assign such polymorphic types systematically, instead of defaulting to some potentially incorrect guess, like Integer. But in the real world, this could lead to values being needlessly duplicated or recomputed. To avoid this potential trap, the designers of the Haskell language opted for a more prudent "monomorphism restriction". It means
that values may only have a polymorphic type if it can be inferred from the context, or if you explicitly give it one. Otherwise, the compiler is forced to choose a default monomorphic (i.e. non-polymorphic) type. This feature is somewhat controversial. It can even be disabled with the GHC flag (-fno-monomorphism-restriction), but it comes with some risk for inefficiency. Besides, in most cases, it is just as easy to specify the type explicitly.

Write Yourself a Scheme in 48 Hours (PDF)

2007-07-17T14:43:00.000+02:00

One of the wikibookians has made a rather nice PDF out of Johnathan Tang's Write Yourself a Scheme in 48 Hours. The conversion was done semi-automatically from mediawiki syntax to LaTeX [in Java... tsk]. Many thanks to Hagindaz for the conversion!

monomorphism and the unintentional fib

2007-07-15T06:55:00.000+02:00

The monomorphism restriction is a simple problem with a scary name.

I think I've run into the first instance where I have actually been able to identify something as being the monomorphism restriction in action (*). The good news is that if you don't understand what the restriction is, this might be your chance. The bad news is that many Haskell newbies might have been hurt in the process, because it involves an unintentional fib that I told in near the beginning of the Haskell wikibook.

(*) I'd appreciate it if somebody could fact-check me on this.

circles and half-truths

High school Geometry. We've got a circle of radius 25 and we want to know its circumference (2 * pi * r). Now let's see what happens when we type it into GHCi


Prelude> let r = 25
Prelude> 2 * pi * r

<interactive>:1:4:
    No instance for (Floating Integer)
      arising from use of `pi' at :1:4-5
    Possible fix: add an instance declaration for (Floating Integer)
    In the second argument of `(*)', namely `pi'
    In the first argument of `(*)', namely `2 * pi'
    In the expression: (2 * pi) * r

Oops! At this point, the reader runs into types for the very first time. This morning, I realised to my horror that I might have given a very wrong explanation of the problem to the gentle Haskell newbie. I said

The main problem is that Haskell doesn't let you multiply Integers with real numbers. We'll explain why later, but for now, you can get around the issue by using a Double for r so that the pieces fit together:
Prelude> let r = 25.0
Prelude> 2 * pi * r
157.07963267948966

Which is true, but is only sort of a half-truth. The problem with my explanation is that it lets the reader walk away with the mistaken belief that numbers without a decimal point, such as 25, can only be integers, when really they can be any Num a. It's as if somebody had taught OCaml too recently and forgotten that haskell /= ocaml! (Coughs discretely)

A concrete example of why it's important to get this right can be found further down the chapter. Here, I want to show that we can use variables within other variables. So here's what I have the reader type in area = pi * 5^2:

Prelude> let area = pi * 5^2

And here everything works... except...

Except if you are a sharp-eyed reader, you remember from above that I had just said that types prevent you from multiplying integers with reals and I had unintentionally implied that numbers like 5 are integers. So if you are a sharp-eyed reader, you look at that thing and you say to yourself, "HUH? That shouldn't work! You can't multiply an integer with a real!"

monomorphism, like polymorphism without the poly

The real problem is not that we had set r to an integer. By right r = 25 should give us any Num a => a; however, since we do not give it a type signature, r gets restricted to being an Integer. In fact, the example of 2 * pi * r would have worked if (a) we had given r a polymorphic type signature or (b) we had run GHC with its 'no monomorphism restriction' extension.

polymorphic type signature

Here we add an explicit type signature to r. Everything works:

Prelude> let  r :: Num a => a; r = 25
Prelude> 2 * pi * r
157.07963267948966

(Thanks to shachaf on #haskell for reminding me that we can use {}/; syntax and showing me how to apply it to write type signatures)

-fno-monomorphism-restriction

And if you don't like type signatures, here's what it looks like running without the monomorphism restriction:

dewdrop /tmp % ghci -fno-monomorphism-restriction
   ___         ___ _
  / _ \ /\  /\/ __(_)
 / /_\// /_/ / /  | |      GHC Interactive, version 6.6, for Haskell 98.
/ /_\\/ __  / /___| |      http://www.haskell.org/ghc/
\____/\/ /_/\____/|_|      Type :? for help.

Loading package base ... linking ... done.
Prelude> let r = 25
Prelude> 2 * pi * r
157.07963267948966

No surprises.

helping the newbies

Clearly, my explanation needs to be fixed. I just need to figure out the right way to do it though. I really don't want to get too technical (the reader doesn't even know about types yet!), but I'll need to avoid lying to the reader. It's already bitten two users, and it took me one of my readers typing in this text for me to figure out what had happened:

You might be thinking that this won't work--isn't 5 an integer, and therefore 5^2 also an integer? And didn't we just get an error trying to multiply pi by an integer?. What's going on is that when you "let r = 25", the "r" you get is more restricted than using the literal string "25" or 5^2. Try :t r and :t 5^2 and see the difference

I'm not entirely satisfied with this attempt (sorry), and want to deal with the situation more gracefully. Just don't know how to go about it yet.

knowing, Knowing and poly/mono

Oh a word about the scary name 'monomorphism restriction'. I find it curious that I should have found that name scary for so long. I mean, for the longest time, I've told myself that I liked 'polymorphism' right? So how difficult would it have been to do a little etymology and notice that 'monomorphism' sounds exactly like 'polymorphism', except that you replace the 'poly-' (many) by 'mono-' (one). By the right, the two words have equivalent scariness! But for some reason, my brain somehow refused to recognise the fact. Instead, it associated one word with warm fuzzies and the other word with huh-does-not-compute.

Otherwise, one thing I found interesting about this is how you can simultaneously (a) hold a piece of knowledge and (b) not apply it consistently. For example, when I had written that module, I knew that numbers could be any Num a (and that you could do fun things making up crazy implementations of Num), yet at the same time as I was using this knowledge (I told myself 'true, but let's not talk about that now, no point confusing the newbies'), I can completely failed to apply that knowledge, because I went back and reflexively treated 25 as an Integer, probably due to all that freshman OCaml teaching. You know something, but you don't Know it.

what's in season?

2007-07-13T19:33:00.000+02:00

Another programming project for those of you who are just itching for something to code. Basically, I want a website which can tell me, given my current location, what fruits and vegetables are in season right now.

Some thoughts, questions and requirements

The site must be dead simple to and very pleasant to use. This is not something you should have to read documentation to figure out.
This kind of thing could be easily international, so I want pictures. Maybe you can grab them from the Wikimedia Commons. I guess it would be fair to give you bonus points if you localise the thing, or maybe let me play with the language settings so I learn how to say 'rutabaga' in Arabic.
How does the user tell you where s/he lives? As a default, it would be nice if you auto-detected it, but what might be nicer also is if I could play around and plug in different locations. How would you pick a location? By pointing at a map of the world? Also, maybe you don't just want a single point in the map, but a region of N kilometers around me. The question is basically, what grows N kilometers from where I live, where I get to specify
Likewise, what do you do about the current date? It would also be nice if I can play around with this, asking not just 'what's in season right now' but 'what's in season during wintertime?'
Where is all of your data coming from? How are you going to store it and look it up? What kind of data do you really need?
If you want to get really really fancy, you can make the site adapt to current events. Maybe parse newspaper texts to find out that it's really not a good year for mangoes.

This is not necessarily a Haskell project (although in my silly, silly eyes, everything is potentially a Haskell project), but this might be a fun way to learn how to do web stuff and maybe play around with databases. In fact, such a project might also be useful for building a Haskell web tutorial, a kind of site that we can build together. Simple objectives that everybody can understand, and actually useful for something.

This project might also be a good way to learn about building user interfaces. Or maybe if you're not interested in working on that stuff, it would be a good opportunity to partner up with somebody else. They worry about the UI stuff and you worry about the code. I don't know anything about UI, except that I think it's important to get it right. If it helps, I greatly enjoyed Donald Norman's The Design of Everyday Things, as well as Bret Victor's Magic Ink.

mediawiki code projects

2007-07-05T13:31:00.000+02:00

Three more projects to practice your Haskell with, and produce something useful while you're at it:

a mediawiki writer for Pandoc (should be a piece of cake)
a mediawiki reader for Pandoc (perhaps less easy, note that there is also some HTML to deal with)
a mediawiki template engine : given a set of mediawiki pages on the local filesystem, convert mediawiki input into mediawiki output, with all templates interpreted

My super-seekret reason for suggesting these projects is so that we can one day convert the Haskell wikibook to LaTeX, or something else, and produce a nice PDF. Naturally, it would also be useful for the other wikibooks, many of which have crappy PDFs, or none at all. But that's not for you to worry about :-)... just have some nice exercises to play with.

(#3 might be a bit of work... and also, it's not clear that this is exactly what we want for converting wikibooks. It seems that the point would not be to expand the templates, but to convert them to something else. If I have a template called HaskellExercises, I don't want to expand it to a bunch of div tags or whatever; I want to convert it to something else)

haskell for reprap?

2007-07-01T00:05:00.000+02:00

Another potentially fun and probably big project for Haskellers: software for the RepRap project. The goal of RepRap is to build an open-source 3D printer for under $500 and with the ability to print out its own parts. Their control software is written in Java... borriiing.

Maybe this is the kind of niche that Haskell could jump in to, something a little unusual (like darcs and xmonad), and very useful at the same time (like darcs and xmonad). My rough thought was that we could similar successes with xmonad, building solid and minimal code for something that is real world (you don't get any more real than physical objects).

#haskell-books channel

2007-06-16T18:15:00.000+02:00

Created on apfelmus's suggestion. Should provide a place for writers of Haskell books, tutorials, blogs, teaching materials of all sorts. We mostly had the wikibook in mind, but figured that others might like to join in on the fun. Real World?

Maybe we should host an annual monad tutorial contest or something, each year judging on different criteria, like "most creative use of tuna fish".

good, simple advice

2007-05-28T10:09:00.000+02:00

I was looking at Audrey Tang's presentation Visual Basic rocks, and came across a piece of good, simple advice:

Dogmatic
✗

Humble
✔

Well put. I was grateful to get a little reminder, a little booster shot. I'm just hoping that I have taken it sufficiently to heart.

(I'm assuming you see the red X under 'Dogmatic' and the green check mark under 'Humble').

burn your gas factories down

2007-05-21T08:32:00.000+02:00

I've been Haskelling for three years now and have been loving it. The only discomfort is that from time to time I become painfully aware of my habit of making things horrible and ugly. Consider this code for instance:

usine à gaz

List of random indices. It is a list [ rM, r(M+1), ..., rN ] where rX is a random number from X to N

> nexts g (l,h) = unfoldr nxt (g,l,h)
>  where
>    nxt (_,l,h) | l >= h = Nothing
>    nxt (g,l,h) = let (r,g2) = randomR (l,h) g
>                      glh2   = (g2, l + 1, h)
>                  in  Just (r, glh2)

This code has been called "clever". Anytime somebody calls your code "clever", you should begin to worry. I mean just look at that thing! You can positively hear the gears turning and the doodads clicking. This code is what the French call a "gas factory", something incomprehensibly and needlessly complex.

is there no simpler way?

Stop micromanaging! Just tell the computer what you want, not what you want it to do

> nexts2 g (l,h) = map (\x -> fst $ randomR (x,h) g) [l..h-1]

No gears or doodads. Geez... I don't what it is with me and making things more complicated than they need to be.

unsort bis

2007-05-20T20:54:00.001+02:00

Boy, do I feel like a big dummy!

Here is a super short and functional version I saw from a Pythonista on reddit.


import Data.List ( sort )
import System.Random ( Random, RandomGen, randoms, getStdGen )

main :: IO ()
main =
 do gen   <- getStdGen
    interact $ unlines . unsort gen . lines

unsort :: (Ord x, RandomGen g) => g -> [x] -> [x]
unsort g es = [ y | (_,y) <- sort $ zip rs es ]
  where rs = randoms g :: [Integer]

edit!

Heffalump points out that (i) if you have duplicates in rs, you don't get scrambling for the relevant bits (ii) this requires Ord x, which it really ought not to (although that is ani improvement from the array thing where I had to lock the type down for some reason)

edit deux

Hmm... what does it mean to get a list of random arbitrary precision integers?

unsort

2007-05-20T18:10:00.000+02:00

Here's a small script I wrote partly for wikibook stuff and partly to learn how to use arrays and random numbers in Haskell.

I'm not a particularly enlightened Haskeller, so you might want to be careful about learning from me. Go ask somebody more experienced.

Note: one thing I'm a bit annoyed about is that I can't figure out how to make the unsort function generic, how to make it work for any type of array, element, index. Suggestions and by all means simplifications welcome.

unsort

> import Data.Ix ( Ix )
> import Data.List ( unfoldr )
> import Data.Array.MArray ( MArray, getElems, newListArray, readArray, writeArray )
> import System.Random ( mkStdGen, getStdGen, Random, RandomGen, random, randomR )
> import Data.Array.IO

> main :: IO ()
> main =
>  do gen   <- getStdGen
>     ins   <- lines `fmap` getContents
>     outs  <- unsort gen ins
>     putStr . unlines $ outs

Our objective is to scramble a list. To do this we convert the list into a mutable array and scramble it in place. This consists of traversing the array from left to right, swapping each element N with a random element from N to the end of the array.


> unsort :: RandomGen g => g -> [String] -> IO [String]
> unsort g es =
>  do arr  <- newListArray (l,h) es :: IO (IOArray Int String)
>     unsortH arr l idxs >>= getElems
>  where
>   idxs   = nexts g (l,h)
>   (l, h) = (1, length es)

The swapping itself is pretty straightforward. We swap the element at the
given index at the next random index. Recursion to the next element until
we run out of indices.


> unsortH :: (MArray a e m, Num i, Ix i) => a i e -> i -> [i] -> m (a i e)
> unsortH arr c [] = return arr
> unsortH arr c (r:rs) =
>  do rElem <- readArray arr r
>     cElem <- readArray arr c
>     writeArray arr c rElem
>     writeArray arr r cElem
>     unsortH arr (c+1) rs

And here is how we generate that list of random indices. It is a list [ rM, r(M+1), ..., rN ] where rX is a random number from X to N... Hmm... I'm pretty sure this can be greatly cut down


> nexts :: (RandomGen g, Num n, Ord n, Random n, Ix n) => g -> (n,n) -> [n]
> nexts g (l,h) = unfoldr nxt (g,l,h)
>  where
>    nxt (_,l,h) | l >= h = Nothing
>    nxt (g,l,h) = let (r,g2) = randomR (l,h) g
>                      glh2   = (g2, l + 1, h)
>                  in  Just (r, glh2)

risk and recursion

2007-05-18T07:43:00.001+02:00

Lately, I've been playing a bit of correspondence Risk with some childhood friends. The good thing about correspondence Risk, when you're scattered across twelve timezones is that it is inherently self-limiting. You are forced to wait your turn every day. Well, that was my hope anyway, but it turns out that things are not quite that simple. The problem is that though you may well your turn, nothing is stopping you in the meantime from thinking about World Domination instead of doing something useful.

But maybe something useful just came out of these daydreams. I was thinking that Risk might be a good tool for teaching programming techniques. For example, it seems calculating dice roll outcomes would a rather nice example for teaching recursion and dynamic programming. This is something we typically do with factorial and fibonacci, but for some students, that might get a little boring. Perhaps dice rolls aren't all that exciting either, but maybe you could sell the Risk angle an awaken your audience's inner ten year old. Maybe you could sell this as part of a bigger package... in this class, we'll be putting together our own Risk implementation, with fancy graphics and everything.

The idea is that you want to ask your program "if the attacker has 37 armies and the defender has 10, what is the most probable outcome and its probability (assuming that each side uses as many dice as possible)?"

My belief is that there is no way to calculate this 'instantly' and without computing all possibilities. You would have to basically crunch through each one of the (a * d) eventual outcomes. (anybody with halfway decent math skills want to confirm that?)

One idea you might able to show with this is just plain simple recursion (although you probably want to start with a simpler example first). So I've got 37 and 10, and if I need the expected outcomes of 37/8 (attacker wins), 36/9 (both win one), and 35/10 (defender wins), I could just factor in the probabilities of getting those.

But I thought was interesting was that you could show that there is some redundant computation going on here. You start at 37/10, but then you go into

37/10
37/8	36/9	35/10

which in turn expands into

37/10
37/8			36/9			35/10
37/6	36/5	35/8	36/7	35/8	34/9	35/8	34/9	33/10

Here, we are recalculating the scenarios 36/7, 35/8 and 34/9. What happens if we turn the crank some more? If I'm not mistaken, the naive algorithm would have to do 3^n calculations (loosely speaking), when really we shouldn't be doing any more than n^2.

I suppose what you'd really want is to have some kind of record of all the outcomes you've already computed. I'm not sure how it would work out code-wise or how you'd shift all those weights around (and if you are interested, I do invite you to cook up a quick implementation, RiskBuzz). I will observe, however, that this table could make it simple to turn around and ask a slightly more involved question, like "ok, so what are the three likeliest outcomes and their probabilities?"

iron coder

2007-05-06T07:39:00.000+02:00

There ought to be something like Iron Chef for programming. Naturally, the Iron Coders would each represent a different paradigm: functional, imperative, object oriented, declarative. Actually, I have no idea how this would work and suspect it probably would not very entertaining at all. I just like the idea of summoning IRON CODER FUNCTIONAL! The guy rises with a platform and it's got a little lambda on it...

haskell wikibook now featured

2007-05-01T09:09:00.000+02:00

Spot the lambda?

Haskell :: Functional Programming with Types

Haskell :: Functional Programming with Types is now a featured wikibook, which means that it will be rotated on to the front page on a regular basis.

There are around 40 books on the site that have this distinction, so if you don't see the Haskell book on the front page, try again in another hour or so.

Write Yourself a Scheme in 48 Hours

Note that the wikibook version of Write Yourself a Scheme in 48 Hours has also been featured for quite some time. Yay, Johnathan! It's not on the front page yet, but should be making its way into the rotation at some point. Its blurb is
as follows:

In this advanced Haskell tutorial, we will implement a significant subset of Scheme together. We assume no prior knowledge; however, we will be going fast. So if you're feeling ambitious, why don't you Write Yourself a Scheme in 48 Hours?

le gentle introduction

2007-04-24T08:45:00.000+02:00

Je viens de voir sur Haskellwiki que la traduction du Gentle Introduction to Haskell est terminée. Félicitations et merci à Nicolas Vallée et TuTuX.

I just noticed that the French translation of A Gentle Introduction to Haskell has been complete. Thanks and congrats to the authors. Now how about a YAHT translation or a wikilivre on Haskell?

Haskell wikibook blurb

2007-04-20T13:13:00.000+02:00

The wikibooks project has a 'featured book' concept, in which the best books of the project are prominently displayed on the front page. The Haskell wikibook has been nominated to be listed as one of these featured books. Votes are positive so far, but nothing official yet. In the meantime, all wikibook authors are being encouraged to put together an advertising blurb for the front page.

Here is my attempt. Can you make it better? Please feel free to post ideas on this blog, or on the wikibook talk page.

Haskell is a functional programming language with a state of the art type system. This book will introduce you to computer programming with our language of choice. It is friendly enough for new programmers, but deep enough to challenge the most experienced. Come stretch your mind with us!

(I'm not over-selling am I? I'm always worried about that)

Oh and the minimalist logo is just something I threw together (public domain, of course). Not married to it, just looking for something better than the current one.

pleac revamp - thanks!

2007-04-19T11:30:00.000+02:00

Just wanted to say thanks to whoever it was that started the PLEAC revamp. Looks much more like Haskell now. The old notation-abuse version is now called haskell-on-steroids, which annoys me somewhat, but now we have less risk of confusing newbies. Will be interesting to see how they compare, the Haskell PLEAC and the wiki cookbook.

congrats to hg!

2007-04-13T22:22:00.000+02:00

I certainly do not speak for the darcs project as whole, but as a contributor and an enthusiastic fan. In any case, congratulations to the Mercurial team for adoption by Mozilla! I'm very happy to see Mercurial being adopted by both mutt and Mozilla, two projects that I also use. For me, it means that we're slowly starting to move on from centralised version control. There may be some pain involved, bugs to shake out and what not, but overall it's for the greater good. (That being said, I'm also pleased to see people upgrading from CVS to SVN, just climbing the ladder in general).

As for darcs, well, I hope that Jason's work will lead to the resolution of bug numero 1. No pressure, or anything. Just a little bit of progress, some useful new insights into the problem would be nice already.

Now if only I could work out all those Cabalisation issues...

instant dada

2007-03-31T22:56:00.000+02:00

Just a small project idea for in case anybody is bored: write a small web toy that generates random blog entries in the style of a given blog.

My mental image would be basically to have a form in which the user can paste a URL. What the user should get back is a formatted blog post (maybe even using the right templates), and also html code that they can paste back into their own blogs.

You'll also want to figure out how to retrieve the individual blog posts, perhaps by using a feed-discovery mechanism. The result should be a single post with title, text and maybe even comments. Haskellers, especially newbies, might be particularly interested because it'll give you a chance with the Haskell XML stuff for parsing the RSS, PFP (for the text generation?) and HApps (for the web front-end). Many of the pieces are there. I remember seeing an RSS parser and a random text generator floating about, so it might just be a matter of cobbling things together.

If you're sucessful, you might even kick off one of those memes where people dada their own blogs as a post.

reading and writing UTF-8 in Haskell

2007-02-02T20:49:00.000+01:00

We were discussing on the Haskell fr what it would take to work with UTF-8. Here is an example I cobbled together. Do whatever you want; it's going in the public domain. Note that I've also posted it to the wiki, so please make corrections there if you can.

Note that I don't really know what the best practices are wrt to reading and writing UTF-8, but here's what works for me.


> module Main where
 
> import Control.Monad (mapM_)
> import Data.Word (Word8)
> import Foreign.Marshal.Array (allocaArray, peekArray, pokeArray)
> import System.Environment (getArgs)
> import System.IO (hFileSize, Handle, hGetBuf, hPutBuf, openBinaryFile,
>                   IOMode(ReadMode, WriteMode))

We're going to be using the 2002 UTF-8 implementation by Sven Moritz Hallberg. I don't know if this is the best one, but it's what darcs uses. (I do wonder though, if it's worth turning it into a little library, something like Data.Char.UTF8)


> import UTF8

We perform the demonstration on a list of files, specified as command line arguments. What we want to show is that we can both read and write UTF-8, so the demonstration will be of reading a file in, reverse every one of its lines, and writing it back out with the extension '.reversed'


> main :: IO ()
> main =
>  do args <- getArgs
>     mapM_ reverseUTF8File args
 
> reverseUTF8File :: FilePath -> IO ()
> reverseUTF8File f =
>   do fb <- readFileBytes f
>      case decode fb of
>        (cs, []) -> writeFileBytes (f ++ ".reverse") $ encode $ reverseLines cs
>        (_,  xs) -> fail $ show xs
>   where
>     reverseLines = unlines . map reverse . lines

For this to work, we need to have some helper functions for reading and writing [Word8]. I don't know if this is the right way to go about it.


> readFileBytes :: FilePath -> IO [Word8]
> readFileBytes f =
>   do h <- openBinaryFile f ReadMode
>      hsize <- fromIntegral `fmap` hFileSize h
>      hGetBytes h hsize
>
> writeFileBytes :: FilePath -> [Word8] -> IO ()
> writeFileBytes f ws =
>  do h <- openBinaryFile f WriteMode
>     hPutBytes h (length ws) ws
 
> hGetBytes :: Handle -> Int -> IO [Word8]
> hGetBytes h c = allocaArray c $ \p ->
>                   do c' <- hGetBuf h p c
>                      peekArray c' p
>
> hPutBytes :: Handle -> Int -> [Word8] -> IO ()
> hPutBytes h c ws = allocaArray c $ \p ->
>                      do pokeArray p ws
>                         hPutBuf h p c

think of a monad...

2007-01-31T11:39:00.000+01:00

Think of a monad as a spacesuite full of nuclear waste in the ocean next to a container of apples. now, you can't put oranges in the space suite or the nucelar waste falls in the ocean, *but* the apples are carried around anyway, and you just take what you need. - Dons

Thought an illustration might be useful:

(Yes, yes, I know I have better things to be doing with my time)

yaht latex now up to date with wiki

2007-01-17T02:21:00.000+01:00

YAHT has two versions: a LaTeX one, which produces a nice PDF including margin notes, an index, etc; and a wiki(book) one, which provides an HTML view and instant editing.

The latex (darcs) version was lagging a bit behind the wiki one for a while, but it has now caught up. I'm quite pleased with the results. I have received 3 darcs patches so far, which may not be huge, but is still better than nothing. More importantly, I have observed that people still like going to the PDF version a lot, so we know that keeping that up to date is important. Having YAHT on the wiki has certainly been useful as well. In the two and half months since its wikification, you have not only corrected countless crappy-script-induced-deformations, over a dozen factual errors, and twice as many typos or spelling/grammatical errors. Good work, Haskellers!

Note also that keeping the LaTeX version up to date should now be a lot easier, thanks to (1) a cleaned up and less noisy conversion script (2) stupid darcs tricks. I've got three branches: the (noisy) auto-convered version (latex-to-darcs), a "production" version, which has less noise than the prior, and an MVS version, which pulls in changes from the wiki. The noisy version has some gunk from my scripts that I just have to revert. The MVS version has some changes which only affect noise (errors in where I put my

tags, for example), but I can just record these as local patches.  Now if somebody modifies the wiki, I just have to do:


mvs update
darcs whatsnew

This tells me right away if there any changes I need to incorporate in the latex version. We'll have to see how this turns out in practice, but for now it seems we have now averted the danger of forking the YAHT (whew!). As usual, you can either send me patches to the latex version (darcs get http://darcs.haskell.org/yaht) or just edit the wikibook.

House runs under Parallels

2007-01-16T23:30:00.000+01:00

House is an operating system written in Haskell. Ever since CSE380 (titled "Operating Systems") and my many many segfaults, I have deeply impressed with those I call "people who know how computers work". So whenever somebody says "I wrote an operating system!"... I simply marvel.

Anyway, was just sort of fiddling with my computer trying to see if I can get it to run, but I was having the toughest time! The thing boots fine under Q (the qemu front end), but nothing I did could make it work on Parallels. Tried renaming it with .hdd extension, but then I just got a grub prompt with no idea how to proceed. Renaming it to .fdd and connecting it as a floppy wouldn't work, so my first reaction was "oh! it must be some kind of crazy Parallels-specific format", so I kept searching the web and asking on IRC, trying to see if anyone know how to produce Parallels-compatible bootable floppy images...

Well you know what the problem was?

Next to the floppy drive icon [connected to hOp-0.8.fdd] are two checkboxes, Enabled and Connect on Startup. Connect on Startup was unchecked.

Yes eric, you have to put the disk in the drive for it to work... good job!

What a dolt.

Edit 2006-01-17

Here are a couple of screenshots, as requested. The first is just the Parallels window:

And the second is of the whole desktop (probably a little gratuitous):

If you're having trouble getting this to work on your machine:

Rename hOp-0.8.flp to hOp-0.8.fdd
Add it as the floppy disk drive to your virtual machine
Set the boot sequence to start with the floppy (or at least, have the floppy somewhere)
Remember to click that Connect on Setup button next to the floppy config

Parallels might crash the first two times you do this. I have no idea why, but it seems to work after that.

nous sommes huit (#haskell.fr)

2007-01-07T20:26:00.000+01:00

Le nombre d'utilisateurs sur #haskell.fr (irc.freenode.net) a encore doublé. On est 8, maintenant, 8 êtres humains. Je vais noter chaque passage à 2X jusqu'à 32. J'imagine que à partir de cette taille, le channel n'aura plus besoin d'encouragement.

Ick. Eric-French is scary. There's now 8 of us in #haskell.fr (or rather, 8 max) and that's bona fide human beings too. I'm going to note each of these doublings until there's 32 of us. At that point, I figure that we're not going to need anybody blowing on the fire.

Francophone Haskeller? Come on in! And stick around; just put it in your automated login. If #haskell is too busy for you, consider maybe #haskell.fr as a quieter alternative. We want newbies, experts and curious hangers-on alike. It would be especially good if experienced Haskellers hung around, because newbies really should not be taking advice from yellow-belt Eric. Oh, and unless they kick me out, non-native speakers bienvenues.

haskell wikibook print version

2007-01-06T01:50:00.000+01:00

Just a quick note to mention that the wikibook's print edition is now automatically generated by a script. 90ish lines of what I hope to be reasonable-looking Haskell code. It should make it easier to keep that print version synched up.

distributed chiming in

2006-12-31T15:26:00.000+01:00

It seems that distributed version control has become a somewhat hot topic lately. The latter two posts make the case that being able to work offline is extremely useful, both for road warriors and for users with less than ideal Internet access. Yes, this does seem like a pretty good motivation for distributed version control. Indeed, it was my laptop and my dialup connection (aside from sheer curiosity) that first got me using darcs two years ago. But now I have one of these fancy ADSL connections and a less need to travel or hack offline while doing so. Yet I continue to use and love darcs. I'm sure this is something that bzr, git, mercurial, etc users can attest to: yes, offline versioning is indeed a great feature, but there is something more.

Warning: this is a rather long post. My apologies to planet haskellers and other busy readers.

one mechanism - many benefits

The thing that attracts me to a system like darcs is its conceptual elegance. From one single mechanism, you get the following features for free:

Painless intialisation
Offline versioning
Branching and merging
Easier collaboration with outsiders

These are all the same thing in the darcs world, no fanciness at work whatsoever. I suppose it's not very convincing to sell simplicity in itself, so in the rest of this post, I'm going to explore these four benefits of a distributed model and discuss what their implications might be for you the developer.

Painless initialisation

Getting started is easier because you don't have any central repositories to set up. That might sound awfully petty. After all, setting up a central repository is nothing more than a mkdir and cvs checkout. But it's much more of an inconvenience than you might think.

Setting up a central repository means you have to think in advance about putting your repository in the right place. You can't, for instance, set something up locally, change your mind, deciding that you want a server and switch over instantaneously. You COULD tarball your old repository, move it to the server, and either fiddle with your local configuration or checkout your repository again. But why should you? Why jump through all the hoops? The steps are simple, but they add friction. How many times have you NOT set up a repository for your code because it would have been a pain (a 30 second pain, but a pain nonetheless?). How many times have you put off a repository move because it was a pain? Painless initialisation means two things (1) instant gratification (2) the ability to change your mind. I would argue that such painlessness is crucial because it brings down the barrier of inconvenience to the point where you actually do the things you are supposed to do.

Branching and merging

A well thought out distributed version control system does not need to have a notion of branching and merging? Why? Because a branch can simply exactly the same concept as a repository, as a checkout. No need to learn two sets of concepts of operations or two views of your version control universe. Just think of them as one and the same. Now, you might be worried about say, the redundancy of all this (gee! wouldn't that mean that branches take up a lot of space?)... but eh... details.

For starters, disk space is cheap, at least much cheaper than it was in the past. There might be cases where you are trying to version very large binary files, but for many programming jobs, we are only shuffling text around, so why worry? Besides, branches are supposed to be disposed of one day or another (merged), right? It's not like they're going to be that long lived, otherwise it's just a fork. Moreover, worrying about disk space is the version control system's job, not yours. You could have a VCS that tries very very hard to save space. For example, it could try to use hard links whenever possible, in much the same manner as "snapshot" backup systems. Disk space is not what you the programmer should be worrying about. It's similar to the case being made for having a second or third monitor: programmer time is more valuable than disk space.

Offline versioning

Previous posts have discussed this at length. It's still useful, even if your Internet connection is superb. It's useful because it lets you hold on to things that aren't quite ready for the central repository, but worth versioning until you're more confident about your code.

Collaboration with outsiders

Open source and free software projects thrive on contributions from outsiders. For example, in the past year, 80% of the 360 patches to the darcs repository have come from somebody other than David Roundy, the original author of darcs. I'm cheating a little bit because many of these patches are also from known insiders. Then again, all of the known insiders were outside contributors at some point. The switch from outsider to insider has been for the most part informal: you send enough patches in and people eventually get to know you. And that's it; very little in the way of formal processes.

Outsider collaboration is made easier for two reasons, offline versioning and decentralisation.

By offline versioning, I mean that people can make their modifications locally, retrieve changes from the central repository and still have their modifications intact, versioned and ready to go. Consider a popular project, like the mail client mutt. Some mutt users have patches that are useful for a few people, but not appropriate for the central repository. So they make their changes available in the form of a Unix patch. If you're lucky, the patch applies to the version of mutt that you've downloaded. If you're not so lucky, you've got some cleaning up to do and a new set of patches. I'm not talking about merging or conflict resolution, per se. Assume the conflict resolution requires human intervention. You've fixed things so that it compiles against the new version. What do you do exactly? Make a patch to the patched version? "Update" the original patch so that it works with the new version of the repository? And what about the original author, what does s/he do with your patch? These kinds of things are not so hard in themselves, but they are a major source of friction. They gum up the works of free software development, or any large project, open source or closed.

If you are a project maintainer, having a tool that handles offline versioning means that it is easier for you to accept changes from outsider contributors (zero insertion force - no need to apply patches and re-commit them).

If you a contributor, having an offline versioning tool means that it's easier for you to submit modifications to the project. You don't have manually create patches: you don't have to keep around a clean and working copy of the project, you don't have to worry about where you do your diffs (so that the patch --strip options come out right), you don't have to worry about what happens when the central repository changes and your patch no longer applies. Again, I'm not referring to conflict resolution. If there are conflicts, somebody will have to resolve them; but the resolution and versioning of these conflicts should involve as little bureaucracy as possible. For extra credit points, some version control systems even implement a "send" feature in which you submit your local modifications via email. The maintainers of the repository can then choose to apply the patch at their leisure. These aren't regular Unix diff patches, mind you, they are intelligent patches with all the version-tracking goodness built in.

Offline versioning adds convenience to the mix, a technical benefit. If you flip it around and look at it in terms of distributed controls, you can see some pretty subtle social consequences as well. Since there is no need for a central repository, there is a lot less pressure for the central maintainer to accept patches or reject them outright because you know that the outsider contributors can get along fine with their modifications safely versioned in their local repositories. Worst come to worse, the outside contributors can place their repositories online and have people work from there instead. It sounds like a fork, which sucks... but sometimes, fork happens. Look, sometimes you get forks for differences in opinion, disagreements between developers, or general unpleasantness. But sometimes you get more innocent forks, for example, the main developers suddenly got a new job and is now working 60 hours a week. S/he is still committed to the project, but to be honest s/he hasn't been looking at patches for the last month. No big deal, the rest of us will just be working from this provisional repository until the main developer gets back on his/her feet. There's a social and a technical aspect to forking. Distributed version control greatly simplifies the technical aspect, and that in turn mellows out the social one. Distributed version control means that life goes on.

simplicity and convenience

I'm really only making two points here. Simplicity matters. It reduces the learning curve for newbies and removes the need for experienced users to carry mental baggage around. Convenience matters. It reduces the friction that leads to put off the things you could be doing and it removes some of the technical barriers to wide-ranging collaboration. I could always be mistaken, of course. Perhaps there is some bigger picture, some forest to my trees; and upon discovering said forest I find myself deeply chagrined, getting all worked up over something so silly as patches. But until that time, I will continue to use darcs and love it for how much easier it makes my life.

the Haskell metatutorial

2006-12-27T15:44:00.000+01:00

Thanks to ndm for so clearly articulating the idea. Here is my attempt at implementing the Haskell metatutorial. Please add your stuff or even just expand the guide tree. Embrace the explosion of tutorials. Make something useful of it.


15:05:09 < AStorm> YAHT leaps a bit too far for me. I'd like something complete but less steep.
15:05:47 < metaperl> YAHT is probably as good as it gets INHO
15:05:49 < metaperl> IMHO
15:06:01 < uccus> there should be a grant unification project for Haskell tutorials
15:06:02 < metaperl> the "algorithms" book is not bad either
15:06:26 < kowey> uccus: the wikibook attempts to remix heavily
15:06:28 < uccus> *grand [blushes]
15:07:01 < kowey> we've got yaht, write yourself a scheme, jeff newbern's stuff, some original content, all mashed up and duct-taped together
15:07:03 < ndm> what i would like is a meta-tutorial
15:07:14 < ndm> a list of questions about haskell, what does this do, do you understand this etc
15:07:26 < ndm> and if you say no, it points you at a tutorial which explains it
15:07:28 < uccus> well, mashed up and duct-taped is not good
15:07:41 < ndm> is there a tutorial on pattern matching, for instance?
15:07:44 < uccus> aah. yes. I agree with ndm
15:07:47 < kowey> we could use some heavy editing
15:07:48 < ndm> which covers ~, !, @ etc
15:08:12 < kowey> right, me too... it's like the malcolm gladwell stuff
15:08:17 < uccus> the wikibook can do that
15:08:31 < kowey> many "right" flavours of coffee, pepsi; extra-chunky tomato sauce, etc
15:08:40 < uccus> it's divided into sections... they can contain links to complete tutorials
15:08:59 < uccus> everyone has a different style of tutoring you know...
15:09:05 < kowey> i agree
15:09:16 < kowey> the wikibook right now is newbie-oriented
15:09:28 < kowey> but we could steer it towards choose-your-own-adventureness
15:09:43 < uccus> kowey: the wikibook right now has different steams for newbie/advanced(?)
15:09:45 < kowey> comments on the discussion page on how we could implement this would be quite welcome
15:09:55 < kowey> we have two tracks, newbie and advanced
15:10:16 < kowey> although the advanced track assumes you've basically just gotten through the newbie track... it tries to be a friendly "advanced"
15:10:19 < uccus> yes, but shouldn't there be more?
15:10:39 < uccus> tracks?
15:10:41 < kowey> well, it's got two tracks in terms of material, one track in terms of style
15:11:04 < kowey> what ndm is talking about is having multiple tracks in terms of style (well... style, level)
15:11:10 < uccus> I mean, the grand Haskell wikibook should contain things that are really advanced
15:11:25 < uccus> like tutes for gtk2hs...
15:11:37 < ndm> kowey: i more meant accepting there will be loads of tutorials, but trying to point people at those which will teach them something new
15:11:42 < uccus> *that* should be called advanced
15:11:44 < kowey> i tend to suspect that's more the Haskell wiki's job
15:12:13 < kowey> although there is http://en.wikibooks.org/wiki/Haskell/GUI
15:13:07 < uccus> aaah. thanks kowey. that's enough I think.
15:13:33 < kowey> ndm: i think we're saying the same thing, although i'm speaking horribly imprecisely

unraveller

2006-12-25T16:41:00.000+01:00

I wish unravelling code was easier. For me, coming to grips with a large software project often consists of undoing bundles of yarn, the big chain, multiple dependencies coming in, multi dependees going out. You want to find out where function foo is used. Well, it turns out that it's used by bar, so now you have to find out where bar is used, and oh it's used by quux... and... oh boy.

I wish there was some tool that would let me start from the center of the big old yarn ball and swim out to the surface. I'm not sure how something like this would work either. Ultimately, all the low-level stuff is used in your main, right? So how can you display this kind of information without just overwhelming the user?

Ctags/Hasktags aren't too useful here. They solve the opposite the problem, in which you've got some top level function and you want to drill down to the guts. Graphviz-ing your modules just gives you a thicket, something to marvel at, but not a great source of insight. Dep trees in some hypothetical browser don't sound that useful either (sigh, click the triangle, expand the subtree, whoah! too much! collapse! collapse!).

The mental image I get is that you run it on your code:

unravel Population.applyToPop *.lhs *.hs

and you'd get a browseable, filtered view of the source code, the relevant bits highlighted, useful links in the right places, but not so many that you feel overwhelmed. I want it to be a standalone tool (like a graphical diff utility), something I can run without having to learn how to use your favourite IDE. Likewise, I want it to be quick and easy (again, the standard for ease being a graphical diff). I don't want to set up a project folder or whatever just so I can run the tool.

This is one annoying aspect of being a user, hell a consumer of any kind. I know I want something, but I don't know exactly what, except that I'd probably recognise it when I see it. The ideas in my head are muddled. If somebody sat me down for a user-interview, I'd just sort of ramble on incoherently for 10 minutes, get confused and wander off. Dear whoever, please build me an unraveller. Think like an Apple UI engineer here. Build the tool from user-experience-in, not from functionality-out. Don't listen to my precise demands, because I don't really know what I want. Just help me understand this code.

Anybody know what it is I'm looking for? (If you do have such a tool, a good exercise would be to run it on darcs. Can it help you figure out what a Slurpy is? How about a Population?)

Denotational semantics

2006-12-22T21:04:00.000+01:00

Apfelmus from IRC just wrote what looks like a very nice chapter on denotational semantics. I am reading it now and learning lots. Why not check it out and post comments on the talk page? I'm sure he'll appreciate any comments the Haskell community might have, or anyone with an interest for that matter.

Thanks, afpelmus; what a great Christmas present!

It's alive!

2006-12-17T22:07:00.000+01:00

Hier sur #haskell.fr, on était plus que 2 personnes pour la première fois. Grace aux iniatives de Nanar et cie, il est possible qu'un veritable communauté francophone d'Haskelliens (?) verra le jour.

Sorry for that attempt at parlezing français. I'm pleased to see that the French Haskell community has now increased two-fold to an amazing four people (on the #haskell.fr at least). Venez nombreux or something. It'll be fun. Thanks to Nanar for kickstarting things with a wiki/mailing list/channel creation initative.

rewriting PLEAC Haskell?

2006-12-10T16:42:00.000+01:00

One complaint I have heard about the Haskell cookbook is that it is rather unidiomatic and thus not very helpful for people trying to learn Haskell. For example, one particularly shocking thing the implementation does is to shadow (.) operator to make more "object-like": o . f = f o (reverse dollar? euro?). This leads to snippets of code which, as one of the wikibook commentors put it, are barely recognisable as Haskell:

s''' = s1.split " ".reverse.join " "

PLEAC Haskell in its present form is not very suitable for educational purposes, but what if the Haskell community ran through and cleaned it up? Only the first two chapters are implemented anyway, so it doesn't seem to be all that much; the only substantial thing to rewrite perhaps being in soundex code. I personally cannot invest any time in this, being already behind in other projects like darcs and wxhaskell, but it might be a fun project for Haskell enthusiasts, or even yellow-belt Haskellers trying to come to grips with the language.

If interested, you should probably subscribe to their mailing list and maybe bounce around some ideas on the Haskell café. Another thing to consider is contacting the original author Yoann. It would be good to get him on board, maybe with a little gentle persuasion. I mean, he probably thought it was a good idea to make the language more recognisable to newcomers. Nice thought... but maybe he would now agree that newbies would be better off with more idiomatic Haskell.

wxhaskell unicode testing

2006-11-28T20:10:00.000+01:00

One of my reponsabilities is to make sure that the wxhaskell Unicode stuff is working. I'm still not too sure what the best way to go about this is. I suppose that any such test would require a majour eyeball component, although it would probably be prudent to add a bit of automation and randomness to things. Here's my first attempt anyway:

I have a small UTF-8 text file with text in a bunch of different scripts. Combined some code from one of the haskell mailing lists to read a file into [Word8] with some an UTF8 encoder/decoder from darcs (this particular code claims to be from the Haskell i18n working group and is BSD licensed). I read the file, break it down into lines and display a copy of each widget for each line in the file. I've only got checkboxes, haven't figured out how I want to go through all the widgets yet.

haskell / vim indentation

2006-11-15T18:51:00.000+01:00

Thanks to motormean for his (or her) haskell indentation vim script. It solved that annoying problem with birdtracks being indented. Now I can break up a line without having to deal with the ">". No word yet on how this words with latex-style literate haskell, but I am grateful already.

yaht now pure gfdl

2006-11-13T21:49:00.000+01:00

Oops. I realised a couple of weeks ago that dual licensing Hal/GFDL on the PDF meant that I couldn't back-propagate wikibook patches into the LaTeX version. So now the PDF is GFDL only. Nice and simple.

(I dropped Hal an FYI and invited him to yell. Since it's _already_ GFDL anyway, I saw no reason that he would mind, which he does not seem to, cf no yelling)