Getting Vidyo running on Archlinux & Plasma 5

I recently upgraded to Plasma 5.2 based upon this glowing review. While the transition was smooth for the most part (apart from minor KWin issues), Vidyo started to segfault. The Plasma system tray no longer supports older system tray protocols. I'm not aware of the details, but VidyoDesktop would complain about not being able to set a system tray icon and segfault. The fix is to install the sni-qt package from the extra repository.

ServiceWorkers in Firefox Update: July 26, 2014

(I will be on vacation July 27 to August 18 and unable to reply to any comments. Please see the end of the post for other ways to ask questions and raise issues.)
It’s been over 2 months since my last post, so here is an update. But first,
a link to a latest build (and this time it won’t expire!). For instructions on
enabling all the APIs, see the earlier post.

Download builds

Registration lifecycle

The patches related to ServiceWorker registration have landed in Nightly
builds! unregister() still doesn’t work in Nightly (but does in the build
above), since Bug 1011268 is waiting on review.
The state mechanism is not available. But the bug is easy to fix and
I encourage interested Gecko contributors (existing and new) to give it a shot.
Also, the ServiceWorker specification changed just a few days ago,
so Firefox still has the older API with everything on ServiceWorkerContainer.
This bug is another easy to fix bug.

Fetch

Ben Kelly has been hard at work implementing Headers and some of them have
landed in Nightly. Unfortunately that isn’t of much use right now since the
Request and Response objects are very primitive and do not handle Headers.
We do have a spec updated Fetch API, with
Request, Response and fetch() primitives. What works and what doesn’t?

1. Request and Response objects are available and the fetch event will hand
   your ServiceWorker a Request, and you can return it a Response and this will
   work! Only the Response(“string body”) form is implemented. You can of
   course create an instance and set the status, but that’s about it.
2. fetch() does not work on ServiceWorkers! In documents, only the fetch(URL)
   form works.
3. One of our interns, Catalin Badea has taken over implementing Fetch while
   I’m on vacation, so I’m hoping to publish a more functional API once I’m
   back.

postMessage()/getServiced()

Catalin has done a great job of implementing these, and they are waiting for
review. Unfortunately I was unable to integrate his patches into the
build above, but he can probably post an updated build himself.

Push

Another of our interns, Tyler Smith, has implemented the new Push
API! This is available for use on navigator.pushRegistrationManager
and your ServiceWorker will receive the Push notification.

Cache/FetchStore

Nothing available yet.

Persistence

Currently neither ServiceWorker registrations, nor scripts are persisted or available offline. Andrea Marchesini is working on the former, and will be back from vacation next week to finish it off. Offline script caching is currently unassigned. It is fairly hairy, but we think we know how to do it. Progress on this should happen within the next few weeks.

Documentation

Chris Mills has started working on MDN pages about ServiceWorkers.

Contributing to ServiceWorkers

As you can see, while Firefox is not in a situation yet to implement
full-circle offline apps, we are making progress. There are several employees
and two superb interns working on this. We are always looking for more
contributors. Here are various things you can do:
The ServiceWorker specification is meant to solve your needs. Yes, it
is hard to figure out what can be improved without actually trying it out, but
I really encourage you to step in there, ask questions and file
issues to improve the specification before it becomes immortal.
Improve Service Worker documentation on MDN. The ServiceWorker spec introduces
several new concepts and APIs, and the better documented we have them, the
faster web developers can use them. Start
here.
There are several Gecko implementation bugs, here ordered in approximately
increasing difficulty:

• 1040924 - Fix and re-enable the serviceworker tests on non-Windows.
• 1043711 - Ensure ServiceWorkerManager::Register() can always
  extract a host from the URL.
• 1041335 - Add mozilla::services Getter for
  nsIServiceWorkerManager.
• 982728 - Implement ServiceWorkerGlobalScope update() and
  unregister().
• 1041340 - ServiceWorkers: Implement [[HandleDocumentUnload]].
• 1043004 - Update ServiceWorkerContainer API to spec.
• 931243 - Sync XMLHttpRequest should be disabled on ServiceWorkers.
• 1003991 - Disable https:// only load for ServiceWorkers when Developer Tools are open.
• Full list

Don’t hesistate to ask for help on the #content channel on irc.mozilla.org.

ServiceWorker implementation status in Firefox

ServiceWorkers are a new web platform feature that allows sites to use JavaScript to provide a better offline experience, and eventually various other 'background service' capabilities. Jake Archibald has a good article on his blog explaining them further.

I've been working on the Gecko implementation for a while now, helped by several other Mozillians and some of the first patches are now beginning to land on Nightly. Interest in trying out a build has been high within both Mozilla and outside. I've been procrastinating about a detailed hacks article, but I thought I'd at least share the build and what Firefox supports for now and what it doesn't.

1. Download the build
2. Run it using a clean profile.
3. Go to about:config and set "dom.serviceWorkers.enabled" to true.

Supported:

• Registration, installation, activation, unregistration. Update() is not according to the spec, the scripts themselves are not cached offline, so don't actually try to make offline apps yet.
• Intercepting navigation and fetch events and replying using a simple SameOriginResponse object.
• Intercepting Push API events.

Not supported:

• Cache API
• fetch() - You'll have to use XHR in the worker.
• ClientLists and postMessage() to talk to windows
• Persistence - ServiceWorker registrations are currently not stored across restarts.
• Anything that throws an error :)
• No devtools support right now.

SameOriginResponse is a relic of an earlier version of the spec. Similarly the Request object received from the fetch event is not exactly up to speed with the spec.

I have two sample applications on Github to try out the build. Note that ServiceWorkers require HTTPS connections. For development, please create a new boolean preference in about:config - "dom.serviceWorkers.testing.enabled" and set it to true. This will disable the HTTPS requirement.

Push API Test - Remember to run the Firefox build from the command line since this demo prints stuff to the terminal. Click the register button to sign up for Push notifications. To perform the push, copy the unique URL and run

curl -vX PUT 'URL'

The ServiceWorker will be spun up and a 'push' notification delivered.

Simple Network Interception - This example will load an iframe with some content. Refresh the page and the iframe's content should change and the browser should show an alert dialog. These changes are done by the ServiceWorkers (there are 2, one for the current scope and one for the sub/ scope) intercepting the requests for both "fakescript.js" and the iframe and injecting their own responses.

Contributing

We could always use help with the specification, implementation and testing.

The specification is improved by filing issues.

The tracking bug for the Gecko implementation is Bug 903441. There is a mercurial patch queue for patches that have not landed on trunk.

I (nsm) am around on the #content channel in irc.mozilla.org if you have questions.

Your Jabber ID as your Persona identity

(This is NOT an official Mozilla project and does not in any way reflect the views of my employer.)

Mozilla Persona is a way for users to use their e-mail ID as their identity on the web. While cool, it will only really take off when existing services that people use become Identity Providers. XMPP (Jabber) is a widely deployed IM protocol whose IDs look like e-mail and it is a secure, federated system in alignment with Persona’s goals. I thought it would be really cool if I could log in to Persona enabled sites using Jabber IDs. I’d like to announce browserid-xmpp which does just that.

It should work with any XMPP server that supports components and BOSH. That said I have only tested it on my VPS (with Prosody, ejabberd and Openfire), so any issues and pull requests are welcome, as is a quick comment if you deploy it on your server. You’ll also need a relatively sophisticated web server like Apache or nginx to serve the browserid file with the right Content-Type. CheckMyIdP is a great way to check if everything is setup properly.

browserid-xmpp is two things. The first is a XMPP component that can plug into any XMPP server and answer a certificate signing query. This is a fork of the “official” browserid-certifier with an Jabber-RPC front-end rather than a web service.

The second is the set of provisioning and sign in pages that can be re-used by any domain. The authentication is handled as a two stage process using BOSH. This was my first experience with BOSH and it is ridiculously cool how it works and supports session hand-off to another page, without which this would not be possible. On the sign in page, an XMPP stream is established and authentication is done using standard XMPP authentication. The established BOSH stream has a session ID and every message sent has an incrementing request ID. On successful sign in, the sign in page sticks these two, along with the JID into sessionStorage. The provisioning page reads these out and ‘attaches’ to the existing BOSH stream. Due to the unpredictable nature of the SID and RID, there is a reasonable guarantee that someone who attached to the stream successfully knew about the stream before. The provisioning page then makes a Jabber-RPC call over the same stream to the XMPP component. This call is performed on behalf of the JID and a certificate is sent back to the browser. You are now signed in!

P.S. I’d like to thank Cory Benfield for an excellent guide to writing an IdP.

P.P.S. This post was published right before a 12-hour plane ride, so I’ll be back for tech support in a while.

Push notifications for the Open Web

I’m excited to announce a new WebAPI that Mozilla has been working on for the past few months – Push Notifications for Web applications. Push Notifications let web applications be notified that something has changed on the server and that the application should refresh its data. For example, a calendar application can use Push Notifications such that whenever a new event is added on the server, the calendar application gets started in the background. The calendar app would then add the event to its local agenda and shut down, all without the user’s intervention. When the (pleasantly surprised) user looks at his phone, it has a current copy of his agenda.
Thanks to Push Notifications, Web applications are freed from repeatedly polling for updates, leading to a better experience for everybody. Users get better battery life and more responsive applications. Developers don’t have to re-implement polling logic in every application they write. Mobile devices benefit from intelligent scheduling of notifications to reduce network usage and further improve battery life.
There were several challenges involved in making a push notification system for the open Web:

1. Developers shouldn’t have to pre-register their applications with a push notification provider. Native app ecosystems require this; for example, you have to get a token from Google to use Cloud Messaging for Android. But for Web apps, which can be self-hosted instead of being distributed by an app store, such restrictions would only limit the audience. In addition, anybody should be free to run their own push server for their devices. Mozilla can run a server for Firefox users. Carriers can run their own servers, on which they can use cellular infrastructure to wake up Firefox OS devices. In fact, an individual user can even run their own push server if they choose.
   
2. Users shouldn’t have to log in to a third party to use push capabilities. Note: Users may still need to log in to the app so the app knows what information to push to the device. For example an email app isn’t very useful without logging in.
   
3. Protect the user’s privacy. This means that the push server should not receive any private or user-identifying information that is specific to the application. In keeping with this, the push server does not receive any application-specific data. It can only act as a “shoulder tap” or carry a number specifying a “version” or similar identifier.
   
4. Make Push Notifications an application wakeup API and not a “user notification” API. The act of displaying a badge or popup notification is not covered by Push Notifications. This decoupling means that applications can use push for all types of tasks, from dealing with IM updates to syncing information across devices without interrupting the user. Applications that do need to notify the user can do so by using the Notifications API with Push.
   

We believe we now have a system that can satisfy the goals while still being secure and scalable.

How do I make my Web application push enabled?

It is very simple for web applications to use Push Notifications. In fact, in only about 100 lines of code, I modified Gaia to support “Push To Install”. Once you’ve logged in using Persona on your phone, you can install an application to your phone from any browser on any device, just by pushing it to the phone.
The basic flow for Web application developers is:

1. Your application calls navigator.push.register() to acquire a unique URL, called a push endpoint. This endpoint usually refers to a push server. Your app can call register() multiple times for different uses. For example, an email application could have one push endpoint for every account it tracks.
   
2. Your application uses something like XMLHttpRequest to notify your application server of this push endpoint. The application server is maintained by the application developer. For example, if you write a microblogging application, you will have a server where posts are aggregated. You would then associate the push endpoint with the particular user of your application.
   
3. When your application server believes something interesting has happened that your application should be notified about, it makes an HTTP PUT request to the push endpoint.
   
4. The push server and the user agent (Web browser/Web runtime/Firefox OS device) communicate and deliver a notification to your application. If your application is not running, it will be started in the background and receive the notification. The notification is delivered using System Messages.
   
5. When it receives a notification, your application should connect to the application server and download the newest data. Since Push Notifications is a signaling system, it does not carry data in the notification.
   

I’ll do a blog post with a detailed Push Notifications tutorial soon.

Status

On March 28, 2013 the first pieces of code to enable Push Notifications landed on mozilla-central and are enabled for Firefox OS builds. Set the pref services.push.serverURL to wss://www.simple-push.com:9999 and the APIs should be available. You can also run your own server.
The protocol is unlikely to change, but client bugs are still being fixed. As such, this is an experimental API. Push Notifications also obsoletes the older Push Notifications API.
Push Notifications is only available to Web applications at this point. There is agreement that web pages loaded in tabs should also be able to use the API, but various technical issues need to be resolved before this can work properly. This is the next major issue the team will address.

A new adventure

(This is a long overdue post).

I remember when I used to have homework, then for a few years in college I had no work. Last month, after a 4 month vacation, I began ‘grown-up life’. Unlike other giant leaps, mine didn’t start with a small step. It started with a 13000km trip to Silicon Valley.

I am very happy to have started a new (and first!) job at Mozilla. After last year’s incredible internship, it feels just like home. I have joined the Platform team and begun working on some upcoming APIs which I’ll blog about as they shape up. Being able to work with extremely talented people and to work on a class of problems that very few engineers in the world get to toy with is very satisfying. It does come with its share of problems, like having to get to grips with a decade old, sparsely documented code-base, but damn, when that code executes correctly after hundreds of iterations, there are so many snacks to celebrate with :) But seriously, joining Mozilla at a time when Web architecture and APIs are undergoing massive upgrades, and we get ready to launch a mobile operating system, is very exciting. Oh and employees get to be guinea pigs test it.

Couple this with managing your whole life for the first time. No longer do parents or the dorm automatically pay electricity bills or ensure your Internet connection is working. House hunting, paying rents, moving, buying furniture, managing finances, cooking, everything has been a first. Smooth for the most part, but far too salty sometimes.

Couple this with the constant technical, cultural and recreational melange that is the Bay Area and the regret about not having enough hours in the day and it’s overwhelming. In a month I’ve discovered rock climbing, seen a trapeze act, won second prize in a hackathon and gone to various meetups. Every day when I snuggle into my comforter-sleeping bag combo (you see, I still haven’t bought a mattress) get into bed, I’m exhausted, but every day is also a very satisfying adventure. I can’t wait to have more of them.

P. S. The libuv book has suffered due to this, but I’ll get back to it soon.

Interning

DA-IICT 3rd year students usually do an internship in the summer. Good industrial internships are always hard to come by, although the situation is way better in IT than in other, more resource-intensive disciplines. 10 interviews, a hundred or so e-mails and lots of forms later this is part guide, part my internship search story. The first thing you’ve to decide is what kind of company you want to intern at. Most companies will have menial code jockey jobs which you don’t want. This is my first blog post typed out on the n900 for the most part while waiting for a flight home at Ahmedabad airport. What a keyboard!

Why not a GSoC again?

Having over two years of FOSS contribution and 7 years of usage and having done a GSoC previously, I could have done one again. While a GSoC is an invaluable experience, it is not the same as an internship. GSoC is excellent at improving e-mail communication skills and long distance collaboration and giving a feel of the open source development style. But it is not the same as going to office, talking to people, having lunch together and hanging out. In addition internships usually will be in a city or country other than the one you live so it can be a great excuse to explore a new place. So if you’ve already done a GSoC, getting a different sort of experience is way more important in my opinion. So I didn’t apply at all this year. Besides I was confident that my experience would serve me just as well to get into the kind of companies I wanted.

Have a good CV

Spell-check, design it well and put in only relevant things. If you are a good singer don’t put that in just to have stuff to show in a IT company. On the other hand positions of responsibility should always be highlighted. FOSS contributions top the charts in startups and other ‘cool’ or cutting-edge companies.

Start early

By luck or resolve my best move was starting to look for internships in December for an intern to start in May. Keep in mind that HR departments are busy places, resumes can take time to process and sometimes do get lost. Wait for a week for a reply when you submit your CV, then ping them aggressively on IRC, Twitter and e-mail to ensure you aren’t forgotten. Interview procedures can take upto a month and for international interns there are visa procedures that take time. Finally you are likely to get rejected by the first few and you should have time to apply for more. In any case companies with established internship programs have information available on their websites and start each season with prior planning.

Decide what you want to work on

The first two companies I applied to were Google India and RethinkDB. The RethinkDB folks were very positive about international interns. I had two interviews in early January which went ok. I was rejected, which in hindsight I know was due to me not really being passionate enough about what they were doing. So I narrowed down to the two specific interests I currently have.

• Javascript engines and low level APIs
• concurrency, distributed networking and web-scale computing.

and decided that I would only approach companies based on these work areas.

Stick to a few companies.

Interviews are always stressful because you have to think on the spot. If you are trying internationally they will be at very bad times (most of mine were at 6am and 10pm). Companies hiring solely on phone interviews will usually take atleast three interviews. In addition you will have to prepare atleast a bit for them. This adds up to a lot of things to do. So stick to a few companies at a time. Prioritise which company you want to work for if hired by multiples companies. When you agree for interview times, make sure you convert timezones properly and that you don’t have any appointments more important than the interview at that time. Finally, remember you have a life too :) I gave one of my interviews at conf.kde.in, and another during Synapse. At such times be very careful with scheduling.

Research

Off-campus internships (ie. ones you find on your own) are the way to go. The college will always aim for what is good for a majority of students, or towards established companies. But if your area of interest is niche you can do a much better job looking on your own.
Try to find out as much as you can about what interns do in the company. With some searching you can usually find blog posts of former interns which can be very informative. Talk to people in the domain. If you are a FOSS contributor ask fellow IRC users about intern opportunities or experiences.
Based on my areas I finally applied to:

• RethinkDB
• Google India
• Directi
• Opera
• Mozilla

I was very lucky to meet a former Mozilla intern at the MIT Media Lab COEP workshop in late January. Without that I would never have thought of it as I was unaware of the Mozilla Foundation and Mozilla Corporation dichotomy.

Be confident, but ready for rejection.

During the interview what matters most is being able to keep up a continuous stream of conversation going to show that you are capable of thinking. So if you tend to think mentally for 15 minutes and then produce the answer in a flash, it would be good to think out loudly. If you are confused, clarify the question, it does not penalise you. Remember that in interviews no one expects perfect code, and classes and documentation. If you miss edge cases thats fine, performance – not an issue until the interviewer actually asks you for a better algorithm. Graph and string algorithms are a favourite of interviewers. For algorithms the TopCoder tutorials are a good read. For C++, the C++ FAQ is invaluable. In fact I suggest always having that page open during the interview. For C++, templates and virtual functions tend to be a question spot. In addition, know the warts and good points of your favourite language. If you have FOSS projects, be ready to explain what they are, and how you implemented them. One favourite interviewer question is, “What was the hardest part to implement?”. In such a case be prepared to explain in as general terms as possible, since the APIs you use may not be something the interviewer has experience with.
A typical telephone interview will last 30 minutes to an hour. Half of that time will be the technical interview and the other half when you can chat with the interviewer. A full recap of my interviews with each company are beyond the scope of this article, but suffice to say that I was lucky to have extremely nice interviewers. Google insists on algorithmic questions which you’ll usually answer via a shared Google Docs. Remember that in a phone interview it is important to know how to approach the problem. Since you have access to a computer, once you know what to do, you can look up your existing code or use the internet. Just keep discussing the approach on the phone and speak with utmost confidence. The interviewer may try to misguide you. For Mozilla and Opera the interview was purely on former experiences and some C++ stuff. Use the chat time later well. Good questions to ask are about prior interns, what they do, how they find the company etc. If you know their name before the interview, see if you can find out about them on the internet.

Get to know and learn from the interviewers

My most memorable interview was the fourth one with Mozilla when due to some daylight savings confusion I was woken up by the interview call. Desperate to get myself sane, I asked him a few questions before I let him start the technical round. After that I spent 45 minutes discussing spidermonkey and Mozilla’s general plans with Luke Wagner. It was a very humbling experience. If you can show the interviewer that you are passionate about the products and do you homework, there final review is much more likely to be glowingly positive. Finally they can tell you about some implementation features, constraints etc. that can be interesting to know about even if you never join that organization.

Congrats, now get to work!

After all this, I hope you get selected somewhere. I can’t really write the part about how to handle it if you get rejected. So what happened to me?
The response order was:

• RethinkDB reject – January 16, 2011
• Directi reject – March ??, 2011
• Mozilla accept – March 22, 2011
• Google accept – March 25, 2011
• Opera reject – March 28, 2011

based on my interview experiences, stipend and location I opted for Mozilla Corporation! When I got the confirmation on March 22, it was unbelievable but I guess all the hard work paid off. The Mozilla folks have been really nice and punctual throughout the process, and damn they know how to take care of their interns :) I think a series of posts will of course pour out of me regarding the internship in the coming months. I will be working at Mozilla HQ in Mountain View, California from next week to the end of July. My first assignment is typed array implementation improvements in JavaScript so Firefox can perform WebGL, audio/video and binary data better. With Mozilla I found a great, FOSS friendly company, interesting work dealing directly with JavaScript engines, a new city to explore right in Silicon Valley and going to the United States of America for the first time ever. I couldn’t have asked for more :D

---

A note to DA-IICT students specifically! Our placement cell has an odd policy where when you apply for one or more internships through the placement cell, you are bound to accept the offer of whichever company accepts you first. So if a ‘better’ company delays their interviews, and a ‘lesser’ company hires you already, you are screwed.

Posted via email from nikhil's posterous