A small Fosdem wrapup

on Tuesday, February 16, 2010

The other weekend I was in Brussels for FOSDEM. As you know, this year we had a Mono room on sunday, thanks to the amazing efforts of Ruben Vermeersch and Stéphane Delcroix. The conference was great, as it always is, although as usual as didn't get to see much of the talks on saturday - busy preparing my own talk about Moonlight, and meeting people, which is one of the parts I enjoy most at FOSDEM. Sunday was awesome, full of Mono talks in a nicely packed room. People were very interested, we had great feedback, and everything went very well, including my demos - it was a very good day, and all in all, a great event. On monday we had a special Mono hackday, where we got together and, well, hacked. I sat down with Lucas Meijer of Unity and we went through some of the issues they have embedding Mono, similar to what Moonlight has to do. Lucas decided to stay an extra day just for the Mono hackday, after a lot of chatting and quite a few beers the day before, and I'm so glad he did, it was a very productive, if somewhat short, day.

Over the three days of the event I had the pleasure of meeting, remeeting and chatting with a lot of wonderful people, whom I usually only get to talk to online - Jo Shields, Mirco Bauer, Alan McGovern, Jeremie Laval, Jim Purbrick, Michael Meeks, Mans Rullgard, David "Lefty" Schlesinger, Rob Taylor, Bertrand Lorentz, Massimiliano Mantione, just to name a few and not in any particular order (I just know I forgot a ton of people!). Also got to meet a bunch of portuguese people, like Vânia Gonçalves, Miguel Azevedo, Paulo Trezentos and more - some of them I only get to see at FOSDEM these days, for some odd reason... weird country this is :)

All in all, it was great, I missed the interaction and the chats and the dinners and the talks and the general merryness and learning that is to be had when you're surrounded by a thousand geeks. I hope to see you all again soon!

PS: I somehow got Jérémie's name confused with a known beer brand... which might, or might not be, a good sign! Fixed... :)

Solving the gcc 4.4 strict aliasing problems

on Tuesday, February 02, 2010

A couple of days ago Jeff Stedfast ran into some problems with gcc 4.4, strict aliasing and optimizations. Being a geeky sort of person, I found the problem really interesting, not only because it shows just how hard it is to write a good, clear standard, even when you're dealing with highly technical (and supposedly unambiguous) language, but also because I never did "get" the aliasing rules, so it was a nice excuse to read up on the subject.

Basically, the standard says that you can't do this:

int a = 0x12345678;
short *b = (short *)&a;

I'm forcing a cast here, and since the types are not compatible, they can't be "alias" of each other, and therefore I'm breaking strict-aliasing rules. Note that if you compile this with -O2 -Wall, it will *not* warn you that you're breaking the rules, even though -O2 activates -fstrict-aliasing and -Wall is supposed to complain about everything (right??). Apparently, this is by design, though why would anyone not want warnings in -Wall for something that will obviously break code is beyond me. If you want to be told that you're not playing by the rules, make sure you build with -Wstrict-aliasing=2, which will say:

line 2 - warning: dereferencing type-punned pointer will break strict-aliasing rules

So now you know you're being naughty. Of course, if you did try to access the variable, even just with -Wall it will complain at you - this more complete snippet will give you several warnings with -Wall:

int a = 0x12345678;
short *b = (short *)&a;
b[1] = 0;
if (a == 0x12345678)
  printf ("error\n");
  printf ("good\n");

line 3 - warning: dereferencing pointer ‘({anonymous})’ does break strict-aliasing rules

The problem gets ugly when you're dealing with structs and pointers to them - then -Wall is completely silent about possible issues, and only -Wstrict-aliasing=2 will work, like in this little snippet:

typedef struct type {
  struct type *next;
  int val;
} Type;


Type *t1, *t2, *t3;
t1 = t2 = NULL;
t1 = (Type*) &t2;
int i;
for (i = 0; i < 2; i++) {
  t3 = malloc (sizeof (Type));
  t1->next = t3;
  t1 = t3;
if (!t2)
  printf ("error\n");
  printf ("good\n");

This doesn't emit any warnings on -Wall because the loop makes it slightly fuzzy for gcc to tell whether things are getting assigned or not. -O2 will optimize away the assignment to t1 on line 3, which will make things not work later on.

So how to fix this? The attribute may_alias allows a type to bypass the aliasing rules, just like character types do (character types are allowed to alias any other type, according to the c99 standard). Changing the definition of Type to the following will make the compiler happy:

typedef struct type {
  struct type *next;
  int val;
} __attribute__((__may_alias__)) Type;

One final note: if you mix up code with aliased types and non-aliased types, gcc will not enforce aliasing optimizations on your non-aliased-possibly-broken code... i.e., if you define this type two times, one with the attribute, one without, and then do the loop above with both types (separately mind you, with separate variables, the code just happens to be in the same method), the non-aliased type won't fail. Aren't optimizations fun?

Update: People have pointed out that the first statement short *b = (short *)&a; is totally legal and has nothing to do with aliasing.

Yes, that's true, I should have been more precise. The statement is perfectly legal. It's when you try to access the data via the pointer that was assigned on that line that breaks the standard. So when your code blows up, it blows up accessing the data, but that's not the cause, that's the consequence. The cause of said explosion is that optimizations + strict-aliasing look at that (totally legal) statement and say "oh, dude, come on, this is bogus" and throw it away while munching on scooby snacks. Well, not sure about that last part.

Anyways, where was I? Oh yes, so, two things: if you don't want to change your code, you can use may_alias , gcc will say "that's so awesome" and everyone will make merry. Or something. The second thing is, and let me add a little emphasis to this part, because I'm sometimes a bit too subtle, and apparently some things should be said *very clearly*: when a statement is perfectly legal, and yet it IS removed via a combination of default flags with NO warnings whatsoever, something is WRONG, and in my opinion, the problem here is lack of warnings.

And that, as someone said, is that. Or not, whatever tickles your fancy. Hmmmm, tickles...

Chrome and Moonlight, or how to deadlock a browser

on Friday, January 22, 2010

It's no secret that Moonlight works best on Firefox at the moment - it's our baseline browser, after all - but we've had many requests to add Chrome support, and since it supports NPAPI just like all browsers out there, it should really work out of the box, requiring only some extra code to implement/hackify stuff that Chrome/WebKit doesn't expose and that we need - basically, DOM support and some downloader tweaks.

After some initial positive reports of Chrome loading the Silverlight Chess sample successfully, I decided to run some tests and start working on the WebKit bridge code... only to find out that I couldn't make Moonlight load properly on Chrome on my laptop at all. Even the simplest of test pages would hang forever on our initial splash animation, and killing Chrome would dump stacktraces all over the place. Clearly it wasn't happy about Moonlight.

My first instinct was "I must be doing something wrong", so I tried on another machine. Same thing. Built a Chromium debug build and tried it - even worse, I hit symbol conflicts all over the place. It seems the Native Client plugin is included inside Chromium by default, and it exports all the NPAPI symbols publicly. Any plugin (like Moonlight) which uses a loader and dynamically loads the real plugin from another location will get its calls intercepted by the Native Client plugin, and things will fail badly. After fixing this, it still kept hanging on the splash animation. Asked other people to test it - same thing. 99.8% of the time it deadlocks completely, and in only 0.2% of the time will it actually load properly. I guess the positive reports were just really, really lucky.

Next course of action - debug the thing. Following the instructions on how to debug Chrome on Linux, I learned about the Renderer and the Plugin processes that get spawned (and the Zygote, too :P), and how to debug them. Only it didn't work (of course not, I hear you say, that would have been way too easy), due to a missing condition on an if on the Chrome loader (I'm guessing nobody actually debugs it on Linux? :P) Patch the thing, and yay, we're debugging.

To keep plugins from blowing up and/or generally misbehaving and giving the browser a bad reputation, Chrome runs them on a separate process that communicates with the main rendering process via IPC. This, of course, is a terrain rife with potential race conditions and reentrancy issues, and that's exactly what's happening with Moonlight. Fortunately, unlike most race conditions, the problem was very reproducible under gdb as well, and I was able to get traces of both processes in the middle of the deadlock.

So what is deadlocking? Well, it's actually very simple: the renderer process calls NPP_SetWindow on the plugin, and also does a blocking call at the same time. In NPP_Setwindow, we do NPN_GetValue and NPN_GetProperty, which call back into the renderer process and block... oops.

I wasn't very confident that I could reproduce this without all the Moonlight code, but just in case, and because I wanted to have a nice clean skeleton NPAPI plugin around, I built one, which does nothing but stub out all the required methods to get an empty plugin going. When it gets to NPP_SetWindow, it calls NPN_GetValue and NPN_GetProperty - and it deadlocks pretty much 100% of the time.

I opened issue #32797 on crbug.com, with the small splash plugin test case, if you're curious. Hopefully this will get fixed fast. With all the calls to the browser that we do during execution, I really really hope we don't hit this again... but it's more likely than not that we will :/

While the idea of keeping the plugins under control by shuffling them to the side is a good one, browser devs should keep in mind that, with all the limitations that a plugin is subjected to, with NPAPI being very far from perfect, with browsers implementing it differently, OS differences that plugins have to deal with as well, it's already so difficult to have a performant plugin (and believe me, the last thing we want to do is stall the brower), we shouldn't have to be worrying about potential reentrancy issues and race conditions when doing such simple things as querying the browser for a property value.

Pretty please?