Multiple improvements in KGet

* This is another GSoC related post, normal users please ignore it.





Abstract
========
KGet is a versatile and user-friendly download manager for KDE.
This project will add various features to KGet to improve its functionality and
usability. These include semantic information via Nepomuk, support for digital
signatures, better Metalink integration and good Plasma support.

Personal details
================

Name: Nikhil Marathe

Email Address: nsm.nikhil@gmail.com

Freenode IRC Nick: nsm

Location (City, Country and/or Time Zone): Mumbai, India ( GMT + 5:30 )



Proposal Name
==============
Making downloading easier, safer, better - multiple improvements to Kget

Motivation for Proposal / Goal
==============================
Kget is a very good download manager, with a crisp interface and nice
integration into KDE. At the same time there is always room for improvements. These include quick renaming of partial downloads, support for
verification of downloaded files and ability to download from multiple sources.
I will attempt to add these improvements.

Goals
-----
As specified in the proposal:
* Add support for a context menu to alter download properties.
* Allow manual addition of URLs to multithreaded downloads.
* Integration of downloads from multiple sources - Bittorrent/HTTP/FTP.
* Integrate KGpg to verify digital signatures.
* Integrate Nepomuk support if available.
* Metalink creation support.
* Support to download an MD5SUMS file from servers if available and verify
downloads. Manual intervention possible if the MD5SUMS is not found.
* In case of a Metalink, attempt to verify PGP checksums using KGpg.

In addition I have the following features I would like to implement:
* Add Plasma applet drop target which can be added to panel - KGet's current drop target tends to cover up screen content. Tucking it away in
the panel seems a good workaround.
* Allow KGet to display transfer details in the system tray tooltip - this
feature ( seen in ktorrent ) is very useful.
* Allow KGet to restart on crashes - it is very annoying to find that the download you left on and went for lunch didn't finish because KGet crashed and didn't restart. This should fix it.

Apart from the motivation of improving KDE, I have a somewhat selfish motivation
for improving KGet since it seems to be the only download manager that can
resume partial files on my computer :)

Implementation Details
======================

NOTE: Wherever UI changes are required, they will be made to the web interface too in case required.

My current implementation plan includes:
* Modify KGet UI for context menu. This also involves adding the appropriate HTML/Javascript/CSS to support the same through the web interface.
* Add KIO operations for moving/copying files when download location is changed
to the KGet core.
* Add support to verify signatures using KGpg command line options ( as KGpg
does not seem to have a D-BUS interface. )
* Add dialog for creation of Metalinks from downloaded files, or local
filesystem files, including automatic MD5 and PGP checksumming.
* Submit data to Nepomuk about download location, download server.
* Allow user to add tags/rating to download while it is going on.
* Discreet option to add multiple download links for non-Metalink downloads.
* For MD5SUMS verification, attempt to guess multiple types of filenames (
MD5SUMS, md5sums.txt etc. ) or allow the user to enter link or checksum
manually.
* Implement plasma drop target which should also have settings for the
following -
Prompt for download by raising KGet window or just downloading to a default location and not breaking the user's workflow.
* Provide a setEmergencySaveFunction and crashHandler so that we can attempt to rescue files from corruption and restart KGet when it crashes.

Tentative Timeline
==================

now - May 23rd : Understand relevant KGet code. Take a look at how to hook into
Nepomuk. Plan the UI and backend design.

May 23rd - June end: Attempt to implement the core functionality of manual URL
insertion, renaming and moving, and integration of multiple download methods.
Integrate KGpg and Nepomuk. Add Metalink PGP verification.
Write plasma drop target applet. Implement tooltip functionality.

July : Implement Rating and Tagging support. Add configuration options.
Implement MD5SUMS verification. Fix bugs, write documentation.

August 1-10th: Fix remaining bugs, wrap loose ends, test test test.

More personal details
=====================
Do you have other obligations from late May to early August (school, work,
etc.)?:

My 3rd semester begins in the last week of July, which means I'll have
a bit less time ( around 25 hours a week ) towards the last two weeks of coding.
Therefore I will attempt to finish almost all
of my proposal by the end of July.

About Me
========

I'm 18 years old. I study in Gandhinagar, India. I'm pursuing
a B. Tech. in Information and Communication Technology at DA-IICT where I'm in
the first year.

I've been programming and using Linux for about 6 years now. I'm fascinated by
all the areas of computing, including compilers, operating systems, graphics and
web design+development. Python is the coolest language for me, although C/++
comes pretty close. I have significant experience in Qt ( including the new MVC
architecture ) - having developed a local network instant messaging client using
it. I am also familiar with the HTTP and FTP protocols and have used PGP
encryption/signing to some extent. In addition I have a decent web development experience, including Ajax and JS effects, which are required for the KGet web interface.

A complete list of my projects can be found at
http://22bits.exofire.net/browse/code

KDE has been a really great piece of software for me ever since I first used
version 3.2. I've always been a fan of its configurability and the momentum and
innovation in the KDE community, and KDE 4 totally took it to the next level. It
has been a kind of
dream to work on KDE someday. Unfortunately I could never participate in GSoC
due to age restrictions. I also didn't have the experience to enter such a huge
project until last year, but now I'm ready to become a full time contributor to
KDE. I have recently patched bug 164137 (
https://bugs.kde.org/show_bug.cgi?id=164137 ) in KWin to remove a redundant
checkbox, which is currently awaiting feedback from KWin developers.

When not in front of the computer I also love playing football, reading and
listening to music.

Proposal for Tiling support in KWin

*Note to my normal blog readers, this post may not be of interest to you

Abstract:
This project will add a tiling layout mode to KWin. Tiling window managers displays all windows on the desktop at once, side by side. This allows easy navigation and allows tasks shared across applications to be carried out effortlessly. Unfortunately it is usually presented as a power user option. This will be an attempt to make it more accessible to new users.

Name: Nikhil Marathe



Email Address: nsm.nikhil@gmail.com



Freenode IRC Nick: nsm



Location (City, Country and/or Time Zone): Mumbai, India ( GMT + 5:30 )



Proposal Name:
Implement support for tiling in KWin



Motivation for Proposal / Goal:

Tiling is a very good way to view multiple tasks/windows together. This is well
suited for tasks like writing reports where you can keep looking at references, or watching multiple pictures/videos and even programming. Fast window
switching via the keyboard also improves productivity and decreases needless
Alt+Tabbing. Although tiling window managers have existed for a long time, KWin
does not have this feature. It has been requested for a long time now too ( https://bugs.kde.org/show_bug.cgi?id=59338 ), and
will be useful for those who want to use KDE yet still want a tiling WM.
In addition tiling WMs have always tried to appeal to power users, leaving a gap for the new user to jump. They eschew the mouse and more often the configurability, while available in abundance, often requires scripting. My belief is that the tiling mode should be an intuitive and friendly user experience and should be offered to the new user as a must-have feature.

Goals:
* Provide an intuitive tiling mode, with different layouts ( like Awesome ) and
support for certain floating windows.
* Expose a D-BUS API for tiling so other applications, including Plasma can hook into it.
* Use compositing and the mouse wherever possible for a fluid experience.
* A feature where windows dragged to the edges will automatically resize themselves to half the screen size as suggested here http://lists.kde.org/?l=kwin&m=122749581132588&w=2 . Of course this could be extended and made more powerful.
* Marking/Selection - Move/tile only certain windows.
* Tiling Stacks - An entire set of
windows could be pushed back, brought in front or moved, each preserving there tiled layout. This could easily be extended to work across multiple monitors.
* Another UI feature that I would like to bring in from 'Present Windows'. The user should be able to select windows by typing a certain filter for the window titles. So a search for Dolphin followed by a keystroke would only tile dolphin instances, leaving xchat floating in the centre.

Implementation Details:

My goal would be to implement a complete tiling solution for KWin. This would
involve:

* Communicate with KWin team and community feedback, along with some help from the KDE Usability team to decide how best it can be made user friendly.

* Add tiling code to the core, including new Placement modes, support for Stacks, session management and screen edge gestures. ( Currently when a window is being moved, edge gestures are still passed on to KWin which rotates the desktop or something else. I'd like to intercept these if the user says so to enable tiling )

* Tiling architecture would be somewhat based on Awesome. Awesome is well engineered and well commented, and some of its principles can be adopted into KWin's tiling mode.

* Ensure tiling plays well with layout commands like Cascade or Unclutter or Present Windows.

* Add marking/selection and relevant key bindings.
Marking would be light weight, relevant only for the current window motion, while a Stack would be more permanent. This would use something like the resize geometry display box or if compositing is enabled, tinting or some similar effect.

* Add D-BUS API.
* Is tiling enabled?
* Available layouts, current layout.
* Switching layouts
* Stack awareness.

* Add configuration options for movement, shortcuts to layouts.
This would be intuitive. The shortcuts would only be active in tiling mode and not clash with application shortcuts. For example. in a multimedia keyboard Next would go to next track in Amarok, but Meta + Next would go to next layout or next stack.
Mouse (especially the scroll wheel) would be used extensively to do almost anything the keyboard can.

* Implement panel applet and plasma grouping by editing relevant taskbar code and leveraging the D-BUS API.

* Allow certain windows/applications to preserve their tiling by using KWin's Special Window/Application settings.

* Write user documentation, fix up bugs.

Tentative Timeline:

now - May 23rd : Understand relevant KWin code. Use KWin less, use a tiling WM more. Read KDE documentation, interact with the mentor and
community. Investigate entry points into code. Design the project and plan how the user interface should behave in
tiling mode.


May 23rd - June end: Attempt to implement the core completely. This includes the main tiling
components, the D-BUS API, working well with Plasma and so on. This will include
testing and bug squashing. Have a working tiling mode by the evaluation period.

July : Implement configuration dialogs/options, write user documentation, make the UI
sensible. Implement non critical features such as compositing support, bells and whistles.

August 1-10th: Fix remaining bugs, wrap loose ends, test test test.

Do you have other obligations from late May to early August (school, work,
etc.)?:

My 3rd semester begins in the last week of July, which means I'll have
a bit less time ( around 25 hours a week ) towards the last two weeks of coding. Therefore I will attempt to finish almost all
of my proposal by the end of July.

About Me (let us know who you are!):

I'm 18 years old. I study in Gandhinagar, India. I'm pursuing
a B. Tech. in Information and Communication Technology at DA-IICT where I'm in
the first year.

I've been programming and using Linux for about 6 years now. I'm fascinated by
all the areas of computing, including compilers, operating systems, graphics and
web design+development. Python is the coolest language for me, although C/++

comes pretty close. I have significant experience in Qt ( including the new MVC
architecture ) - having developed a local network instant messaging client using
it.

A complete list of my projects can be found at
http://22bits.exofire.net/browse/code

KDE has been a really great piece of software for me ever since I first used
version 3.2. I've always been a fan of its configurability and the momentum and
innovation in the KDE community, and KDE 4 totally took it to the next level.
Okular and Amarok are the killer applications for me. It has been a kind of
dream to work on KDE someday. Unfortunately I could never participate in GSoC due to age restrictions. I also didn't have the experience to enter such a huge project until last year, but now I'm ready to become a full time contributor to KDE.

When not in front of the computer I also love playing football ( soccer ).
Otherwise I can be found reading or listening to music.

GMail Notifier on an Arduino

The exams are over and I've been hacking a bit on the Arduino today. So I came up with a simple hack which blinks an LED on the Arduino if you've got unread mails in your Gmail inbox. I assume that you're familiar with the basics of Arduino.

Equipment

• A computer with an internet connection


• Python


• pySerial


• Arduino ( I'm using Duemilanove )


• Red LED


• Pushbutton (optional)


• Wires
  

On the computer

A python script will run continuously on the computer, and fetch the Gmail RSS feed every few minutes. pySerial will be used to notify the Arduino of new mails.

Here are our imports and constants

# ~ Gmail Notifier for Arduino
# ~ This file is released under the public domain

import httplib
import getpass
import base64
import re
import time
import serial

INTERVAL = 5 # check every INTERVAL minutes

serv = 'mail.google.com'
path = '/mail/feed/atom'
# ask user name and password and encode them for authentication
auth = base64.encodestring(
                 '%s:%s'%(raw_input('Username: '),
                   getpass.getpass()))

So first, fetching the feed. We'll use httplib. Here is the code:

def getfeed():
  print 'Checking...'
  conn = httplib.HTTPSConnection(serv)
  conn.putrequest('GET', path)
  conn.putheader('Authorization', 'Basic %s'%auth)
  conn.endheaders()
  return conn.getresponse().read()

Next lets get the count. Gmail replies in the following format. In the case of new mails there is more information, but we don't care about that. We're mainly interested in fullcount.

<?xml version="1.0" encoding="UTF-8"?>
<feed version="0.3" xmlns="http://purl.org/atom/ns#">
<title>Gmail - Inbox for nsm.nikhil@gmail.com</title>
<tagline>New messages in your Gmail Inbox</tagline>
<fullcount>0</fullcount>
<link rel="alternate" href="http://mail.google.com/mail" type="text/html" />
<modified>2009-02-22T11:00:33Z</modified>
</feed>

So we'll use regular expressions to get the count.

def count(data):
 matches = re.findall('<fullcount>([0-9]+)</fullcount>', data)
 if len(matches) == 0:
     print 'Error in parsing feed, check user name and password are correct'
     return 0
 return int(matches[0])

We'll need to write this to the serial port.

def writeSer(data):
 try:
     # the best way to find this out is to launch the Arduino environment
     # and see what it says under Tools -> Serial Port
     ser = serial.Serial('/dev/ttyUSB0')
     ser.write(data)
 except serial.serialutil.SerialException:
     print 'Error writing to serial device'
     raise

Now that we're done with the functions, it's time to make them work together

# subtract so that we check first time
last_check = time.time() - INTERVAL*60

while True:
    if time.time() - last_check < INTERVAL*60:
        continue
    last_check = time.time()
    msgs = count(getfeed())
    print msgs,'mails'
    writeSer(str(msgs))

Thats the computer part.

On the Arduino

The circuit :



The LED is on pin 13 and goes to ground. The button takes 5V through the power pins on the analog side, via a 220 ohm resistor. The other leg is grounded. Pin 4 can be used to read the state of the button. The wire from pin 4 connects to the 5V leg of the button.

The code is dead simple and so is presented together.

int ledPin = 13; // connect led to digital pin 13, or use default small one
int bPin = 4; // connect button to digital pin 4
boolean blink = false; // holds our current state

int INTERVAL = 200; // led blink rate in milliseconds

void setup() {
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
}

void loop() {
  if(Serial.available() > 0)
    blink = Serial.read() > 48;// 48 is 0 is ASCII

  if(digitalRead(bPin) == LOW) // button pressed
    blink = false;

  // blink is true if we got serial input
  // or we had got serial input and button hasn't been pressed yet.
  if(blink) {
    digitalWrite(ledPin, HIGH);
    delay(INTERVAL);
    digitalWrite(ledPin, LOW);
    delay(INTERVAL);
  }
  else
    digitalWrite(ledPin, LOW);
}

The button is used to switch off the blinking once you've noticed that you've got mail and don't want it to keep blinking till the next check.

Running

Verify and Upload the code to the Arduino. You can try entering numbers in the Serial Monitor to check that the circuit works. Remember to switch off the Serial Monitor.

Now start your script, python mailarduino.py. Enter authentication details. Now sit back and relax... oh wait, you've got mail.

Full python script ( mailarduino.py )

import httplib
import getpass
import base64
import re
import time
import serial

INTERVAL = 5 # check every INTERVAL minutes

serv = 'mail.google.com'
path = '/mail/feed/atom'

auth = base64.encodestring(
                '%s:%s'%(raw_input('Username: '),
                getpass.getpass()))

def count(data):
 matches = re.findall('<fullcount>([0-9]+)</fullcount>', data)
 if len(matches) == 0:
     print 'Error in parsing feed, check user name and password are correct'
     return 0
 return int(matches[0])

def getfeed():
 print 'Checking...'
 conn = httplib.HTTPSConnection(serv)
 conn.putrequest('GET', path)
 conn.putheader('Authorization', 'Basic %s'%auth)
 conn.endheaders()
 return conn.getresponse().read()

def writeSer(data):
 try:
     ser = serial.Serial('/dev/ttyUSB0')
     ser.write(data)
 except serial.serialutil.SerialException:
     print 'Error writing to serial device'
     raise

last_check = time.time() - INTERVAL*60 # subtract so that we check first time

while True:
    if time.time() - last_check < INTERVAL*60:
        continue
    last_check = time.time()
    msgs = count(getfeed())
    print msgs,'mails'
    writeSer(str(msgs))

Techfest 09 and the Arduino

I was in mumbai over the weekend, attending Techfest 2009 at IIT Bombay. Techfest has been a large and great event for many years now, bringing cutting edge technology to students and holding great lectures by iconic people in technology.

Me and three of my friends went specifically to attend the Physical Computing workshop by Keyur Sorathia, an interaction designer. This was my first foray into hardware hacking, mainly because hardware components are not so accessible in India.

On day one, we hacked a keyboard, figured out which pins pressed which key and connected a tilt sensor to do the same. Then we popped them into an helmet. Unfortunately, soldering wasn't our strong point and although the helmet worked, we couldn't wear it on our head. Keyur also showed us videos about various which aim to make computer interaction more like interacting with real objects.




Day two, and we finally got our hands on the Arduino. The Duemilanove is the latest iteration of Arduino and we had fun programming LEDs to light up on key presses. Of course we didn't get to see all of the Arduino's magic in a two day workshop. But we did get infrared sensors, peizo sensors and buttons to mess around with, which I'll start using soon.


On day three, the workshop unofficially over, we first saw Robot Wars, which was a bit of a disappointment because one of the machines got stuck and didn't move much for the remainder of the match. Then we attended an interesting lecture by Rhythm and Hues. Directi had a booth where they had 3 C snippets and visitors had to predict the output. Two out of the three were pretty tough, but I managed to answer 2/3 and got myself a T-shirt that says <geek> on the front and </geek> on the back.

Nissan concept car Pivo2

Due to our train and other schedules, we were unable to attend Technoholix on any of days, which was bad. So next year I'm gonna take off more time and stay at Techfest and attend all the great events.

First flight

I went paragliding again last week, to finish my Elementary Pilot course. I had my first solo flight from a height of 45m on 24th Dec 2008. It was an exhilarating feeling, although short. After 3 days of solo top to bottoms, I now have 15 min of total air time. I'll put up the pictures soon. The sight is Shinde Wadi, around 125km from Mumbai.






I also had great fun at the Native Place, home to Nirvana Adventures. Hoping to go back in May and make progress towards Club Pilot.

Filtering QTreeView

While working on my newest project, I had a miserable time figuring out how to filter sub-items in QTreeView and its associated models (like in kopete/pidgin). The first thing to do is to use QSortFilterProxy as a middleman. The problem is that QSortFilterProxy will only search top level tree nodes.
The way to implement custom filters is to subclass QSortFilterProxy and reimplement filterAcceptsRow(). But I couldn't figure out any way to quickly access sub elements using QModelIndex, nor could I find anything on the 'net. So after quite a lot of fiddling around, I managed to get this, which works. So to save someone else some time, here is the code

bool MemberFilter::filterAcceptsRow(int sourceRow, const QModelIndex& sourceParent) const
{
    if( sourceParent.isValid() && sourceModel()->data(sourceParent).toString().contains(filterRegExp()) ) return true;

    QString data = sourceModel()->data(sourceModel()->index(sourceRow, 0, sourceParent)).toString();

    bool ret = data.contains(filterRegExp());

    QModelIndex subIndex = sourceModel()->index(sourceRow, 0, sourceParent);
    if( subIndex.isValid() )
    {
        for(int i = 0; i < sourceModel()->rowCount(subIndex); ++i)
        {
            ret = ret || filterAcceptsRow(i, subIndex);
        }
    }
    return ret;
}

On a side note, this has been a post after a long long time. This is because I've decided only to post important things or code related stuff. Right now I finished one semester of college, and will be home for a month.

Installing Arch linux on the HP DV6910TX

This is a record of my attempt to put Arch (2008.06 Core Dump) on the HP DV6910TX.
Here is my system configuration

• Intel Core 2 Duo T5750 @ 2.00GHz
• Ram : 3GB
• Nvidia GeForce 8400M GS with 256Mb dedicated memory
• 320Gb harddisk

Partitioning

Here was my original partitioning scheme:

• C: 308Gb
• D: (Recovery) ~10Gb
  

Make sure you create your HP Recovery Discs first!

Since the arch install CD doesn't have ntfsprogs ( it has the packages, but not as part of the setup boot ), I booted from dreamlinux (you can use any liveCD) and used ntfsresize to shrink the C: (/dev/sda1) to 35Gb.

Then I used gparted to create a proper partition layout. Here is how it goes

• /dev/sda1 - ntfs - 35Gb (C:)
  
• /dev/sda2 - ntfs - 10Gb (D:)
• /dev/sda3 - swap - 4Gb ( to hibernate, you need atleast as much swap as your RAM )
• /dev/sda4 - extended
• --- /dev/sda5 - ext3 - 20Gb (/)
• --- /dev/sda6 - ext3 - 50Gb (/home)
• --- /dev/sda7 - ntfs - 180Gb (/shared)

After committing these changes I began the installation.

The rest of the settings were normal Arch installation procedures, which can be found elsewhere.

Hardware

Display

The dv6910tx has an nVidia 8400M GS with 256Mb of dedicated memory. Installing the nvidia drivers ( pacman -S nvidia nvidia-utils ) and running nvidia-xconfig generated the required xorg.conf, and the card worked perfectly.

Touchpad

Installing the synaptics package and merging the required changes into xorg.conf ensured that the touchpad worked too, including scroll areas and locking.

CD/DVD drive

Works perfectly without any configuration.

Ethernet

Arch correctly used the r8169 driver

Wifi

Arch was able to identify the card, but there is no hotspot here to test the connection.

Card reader

The 6910 has a 5-in-1 reader from Ricoh. But I could test only for SD cards.

Firewire

Detected fine,, but no device to test.

Power management

Installing Guidance (pacman -S guidance-power-manager) will allow you to quickly view battery performance and perform suspend or hibernate ( I'm using pm-utils ).
To enable different power modes ( governors ), install cpufreq and follow the Arch Wiki Page on CPUfreq.

A special entry has to be made to mount/unmount NTFS partitions. Put the following in /etc/pm/sleep.d/66ntfs:

#!/bin/bash                                  

MOUNTS="/windows/C /shared"

function 66ntfsmount() {
   for i in $MOUNTS   
   do                 
       mount $i       
   done               
}

function 66ntfsumount() {
   for i in $MOUNTS
   do
       umount $i
   done
}

case $1 in
   hibernate)
       66ntfsumount
       ;;
   suspend)
       66ntfsumount
       ;;

   thaw)
       66ntfsmount
       ;;
   resume)
       66ntfsmount
       ;;
   *)
       echo "BAD!"
       ;;
esac

Now make it executable ( chmod +x /etc/pm/sleep.d/66ntfs ).

Webcam

Not tested yet.

NTFS partitions

I installed ntfsprogs and used the fuse.ntfs system to mount ntfs in userspace as read and write. Here are the additions to /etc/fstab

UUID=...  /windows/D  fuse.ntfs  defaults 0 0
UUID=...  /windows/C  fuse.ntfs  defaults 0 0
UUID=...  /shared     fuse.ntfs  defaults 0 0

NOTE: The arch shutdown scripts kill the fuse ntfs system rather than unmounting the drives. To fix this issue edit /etc/rc.local.shutdown and insert the following:

. /etc/rc.conf
. /etc/rc.d/functions

# Unmount NTFS partitions
stat_busy "Unmounting NTFS partitions"
umount /shared
umount /windows/C
umount /windows/D
stat_done

Bluetooth

Device detected. I could scan for my cell phone, but I've yet to figure out how to exchange files.

Hotkeys

If you use KDE4, go to System Settings -> Regional and Language -> Keyboard Layout. Enable keyboard layouts and change the keyboard model to Hewlett-Packard Pavilion ZT11xx. All the keys EXCEPT Quickplay and DVD work after that. The mute key does not turn red, but it does work. Of course it is your job to actually configure application shortcuts to use the keys.

Remote

The volume keys respond. Other keys should be configured to work with the respective applications using application specific settings.