Formatting git patches for partially transplanting a repository

on Friday, March 08, 2013

So I wanted to move a subdirectory inside a git repository into its own repo, keeping all the history of the changes in the original repository in the new one. With git, copying partial history around is as easy as using git format-patch --root -o directory/to/put/patches/in -- path/to/subdirectory, which will create a numbered patch file for every commit that touched the subdirectory in question. Applying all the patches in the new repository is just a questions of doing git am -3 *.patch.

The problem is, format-patch skips merge commits, which means that there might be missing changes in the patches, which sorta makes things not work.

The alternative way is then to do git log --pretty=email, which outputs a commit in the same format and actually handles merge commits properly. But, of course, I need to do that for every commit that I want to export (and there's a bunch), and I hate doing things by hand.

To that effect, here's a few lines that do the job properly, exporting a list of commit hashes in the proper order and then going through them one by one and exporting each one to a directory, numbered appropriately so they're correctly sorted:

Export the list of interesting commits in the correct order (older to newer)

git log --oneline --reverse -- path/to/subdirectory|cut -d' ' -f1>../patches/list

Create a patch file for each commit on the list

c=`wc -l ../patches/list|cut -d' ' -f6`;for j in $(eval echo {1..$c});do n=`printf "%04d\n" $j` && a=`head -n $j ../patches/list|tail -1` && git log -p --pretty=email --stat -m --first-parent $a~1..$a -- path/to/subdirectory >../patches/new/$n-$a.patch;done

Apply all the patches in the new git repository

git am -3 -p3 ../patches/new/*.patch

The subdirectory I'm taking things out of is 3 levels deep in the original repository, and I don't want to keep the parent directories, so I'm passing -p3 to have git (and the patching process) remove them when applying.

If git am fails to apply a patch, it's very likely that this patch is a merge commit with changes that are already applied. I can check this by doing patch -p3 .git/rebase-apply/##, where ## is the failed patch number reported by git am. Patch will either apply the change or report that a change has already been applied (and do I want to revert it? just say no). If any changes needed applying with patch, I can then add the changed files with git add and do git am --resolved, which will create the commit and continue applying the patches. If there are no changes to be applied, I can just skip it with git am --skip (which is most likely to happen) and continue.

Gnome Developer Experience Hackfest 2013

on Monday, February 11, 2013

The Aftermath

After finally getting rid of a really bad cold, here I am reporting about the DevX hackfest that took place right before FOSDEM, at the Betagroup Coworking Space, a very nice coworking place in Brussels with excellent access and great facilities. The hackfest, organized by Alberto Ruiz (thanks!) and sponsored by the Gnome Foundation, had the goal of improving the application developer experience on the desktop, and lasted for three days, with plenty of discussions on a variety of topics, from tooling and IDEs, documentation, languages, libraries to bundling, distribution and sandboxing (and more).

It was a pretty interesting experience, there were a lot of people participating and due to the nature of the facilities (i.e., we were all in one room together), and lot of discussions bounced around the room and spilled over from group to group. My goal for the hackfest was to work on (and hopefully finish) the tooling required to create Mono bindings for the Gnome desktop in an automated way, so that packagers and developers can make bindings available for any library that supports gobject-introspection. By the end of the hackfest, the bindings tool (called bindinator) was able to bind Webkit with no user intervention, and with gstreamer bindings 95% done  (two bugs still pending), things are looking good for automated C# bindings.

Between hacking and sneezing, we discussed tooling and IDEs, particularly what an IDE should have in terms of features, and what features a language should have to better support an application development environment; i.e., in the case of dynamic languages, a built-in AST is a very good thing to have, since you really want good code completion in your IDE, especially when you're starting on a new platform and aren't comfortable with the available libraries and APIs. Other useful features that went on the list for an IDE would be syntax highlighting (a must on any good code editor), responsive UI, good build infrastructure integration (preferably hiding away the specific build tool details and possibly with its own project format that's independent of specific build tools (looking at you autotools)), debugger support, modularization (for user extensibility). And, preferably, being built with the same tools and languages that are recommended for the platform (dogfooding++).

The language discussion was really *the* topic that dominated the three days. There was a lot of back and forth over the merits and demerits of Python, Javascript, Vala, C and C# throughout the days and into the evening activities. Which language would be the easiest to integrate? What tools are available for each? Debuggers are important and hard to do, code completion is harder in some languages than others; if one were to code an IDE from scratch, what language would be better for the UI, the logic, systems integration? Would floating point fuzziness affect someone doing an accounting app? What type of developers are the target, and what type of apps? Widgets and applets that just create a quick UI over existing libraries? Bigger apps? How many developers are there for every language, and how many things are missing and/or need to be fixed in Vala, or Javascript, or any other language? Should there be a single language recommendation? Two languages? All these and more were put forth and discussed extensively, and will probably continue to be discussed over time (as there is rarely a right answer for most of them). No matter how people feel about the decisions that came out of this hackfest, they can be assured that they weren't taken lightly, or without a fight.

All in all, it was a great hackfest, three days of very productive discussions and hacking. Kudos to Alberto Ruiz for a great job organizing everyone, and thank you to the Gnome Foundation and Andrea Veri for the sponsorship and assistance.