Root Cause Analysis for library.nu

Indeed, this is one of the best articles I have read in a very long time! The author takes a deep look at the shutdown of the library.nu website (formerly GigaPedia) and analyzes the real causes for the flourishing of such a 'pirate' society in our modern times. Long live Open Access to Knowledge!
http://www.aljazeera.com/indepth/opinion/2012/02/2012227143813304790.html

Compiling and Installing WFDB version 10.5 on Debian Squeeze

Today I tried to compile and install the WFDB tools from the PhysioToolkit (PhysioNet) on my Debian Squeeze system. I am documenting this experience here!

1. Download the source package from http://www.physionet.org/physiotools/wfdb.tar.gz
2. Install the needed Debian packages first. I needed the following Debian packages
• libcurl4-gnutls-dev (for NETFILES feature support in WFDB, which allows you to work on portions of the PhysioBank data records online, without having to download the entire record on your computer)
• libexpat1-dev (for XML parsing, some files did not compile successfully while installig the WFDB library because this package was missing)
3. Follow the steps under "if you download the sources (recommended):" in the official quick start guide. If you are on a 64-bit system (like me!) then do NOT install the WAVE application now
4. If you installed the WFDB library in a non-standard path (as I did), then probably you will need to adjust the $PATH and $LD_LIBRARY_PATH environment variables. I added the following snippet to my ~/.profile, (this will be automatically executed every time you login with your account -- run 'DOT SPACE ~/.profile' to execute the profile script immediately!) -- I also needed to manually copy the 'man' folder from the "(path to WFDB installation)/share/man" to '/usr/share/man' for manual enteries to be installed into the system!

# add WFDB binaries to PATH

PATH="(path to WFDB installation)/bin:$PATH"




# add WFDB libraries to LD_LIBRARY_PATH

LD_LIBRARY_PATH="(path to WFDB installation)/lib64"

5. Now you are ready to install the optional WAVE application in case you are running a 64-bit system. For this, I followed the instructions in this post. Finally, you will need to further modify your ~/.profile to append the following

# add WFDB-32 binaries to PATH

PATH="(path to WFDB-32 installation)/bin:$PATH"

# add WFDB-32 libraries to LD_LIBRARY_PATH

LD_LIBRARY_PATH="$LD_LIBRARY_PATH:(path to WFDB-32 installation)/lib"

Research by Dr. Moustafa Youssef

Today I had the opportunity to attend a session given by Dr. Moustafa Youssef on his research. The key point in his talk was about detecting the presence of humans inside a wireless network area by detecting variations in the wireless signal strength (Received Signal Strength Indicator, RSSI) at the receiver side. Dr. Moustafa introduced his Horus system based on this concept! Imagine the scenario in which your home wireless network acts as an intrusion detection system while you are sleeping at night. Dr. Moustafa and his group developed an all-software solution that provides this functionality by utilizing the hardware of the deployed wireless network and your laptop! If you are really interested, you can watch his Google Tech Talk here. The same concept is eventually being applied in hospital settings as well as discussed in this paper.

The Adaptive Noise Canceller as a High-Pass Filter

In 1975 Widrow et. al. introduced a scientific paper about Adaptive Noise Cancellers (ANCs). Since then, ANCs have been used extensively in different applications. This article is about the use of an ANC as a simple, yet powerful, high-pass filter.



As shown in the figure, a reference DC signal x(n)=1 is fed to a single-coefficient ANC. The noise estimate y(n) is the multiplication of the DC signal x(n) times the single coefficient, or weight, w(n). The noise estimate is then subtracted from the input signal d(n) to obtain a noise-free sample e(n) which can be considered the output of the ANC. e(n) is also fed back into the Least Mean Squares (LMS) Algorithm block, which calculates a new value for the weight w(n+1) based on the correlation between the reference signal x(n) and the error signal e(n) using the equation

w(n+1) = w(n) + mu * e(n) * x(n)

where mu is the adaptation step, usually a very small factor in the order of 0.01.

Widrow et. al. proved in their 1975 paper that this structure is equivalent to a high-pass filter with a corner frequency determined by mu.

The ANC is indeed a very simple, yet powerful, structure that has many applications in different disciplines. Recently, I've used the ANC as proposed by Widrow et. al. with some modifications for adaptive cancellation of power-line interference signals from the electrocardiogram (ECG), as shown in this paper.

Physiological Signals Datasets

In this post I will be gathering some links about the physiological signals datasets that are publicly available on the web. This post will be updated from time to time!

Physiobank
One of the largest platforms that makes it easy to contribute physiological signals is the PhysioBank platform. Following are some of the interesting datasets available under PhysioBank. The easiest way to access these datasets is through the PhysioBank ATM web interface.

ANSI/AAMI EC13 Test Waveforms
The ANSI/AAMI EC13 standard for electrocardiographic monitors specifies those test signals for evaluating the performance of the electrocardiographic monitors.


Multi-Parameter Databases
Those databases include different physiological signals typical of a patient monitoring device, such as ECG, Respiration, Continuous Blood Pressure, Plethysmogram, ... etc.

An important subset of those databases are the Polysmnographic databases useful for sleep staging. Physionank contains the MIT-BIH Polysomnographic Database, the Sleep-EDF Database, the Sleep Heart Health Study Polysomnography Database, and St. Vincent's University Hospital / University College Dublin Sleep Apnea Database.


MIT-BIH Arrhythmia Database
The most famous ECG arrhythmia database!

Worker Threads!

Many times you need to call a function/method that does some really heavy work. However, you don't want this function call to block your main application. You want to maintain a responsive GUI, for example!

This famous problem has a solution using multi-threading. From my background in the C++ Qt framework, I know the basic principles involved. And there is a very famous Qt example that explains how this "Worker Thread" solution can be implemented: The Madelbrot Example

Today I wanted to solve the same problem but in the C# (.Net Framework) context. Fortunately, the MSDN had a similar example that illustrates the solution to this problem, making use of the BackgroundWorker class:
http://msdn.microsoft.com/en-us/library/hybbz6ke.aspx

Reading and Writing .Net Applications Settings

Sometimes you have to deal with the .Net framework in a way or another. I was forced to use a .Net C# closed-source library. And that library was automatically reading file paths from ConfigurationManager.AppSettings[]. So, I needed a way to read and modify those settings. Fortunately, this post saved my life!

To read:

string value = ConfigurationManager.AppSettings["oldPlace"];

To modify/write:

System.Configuration.Configuration config =ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);

config.AppSettings.Settings["oldPlace"].Value = "3";
config.Save(ConfigurationSaveMode.Modified);
ConfigurationManager.RefreshSection("appSettings");

More information can be retrieved from the MSDN:
http://msdn.microsoft.com/en-us/library/system.configuration.configurationmanager.appsettings.aspx

Precision and Recall

One of the most important metrics used in evaluating the performance of binary classifiers is the Precision-Recall curve.

From Wikipedia:

It is possible to interpret precision and recall not as ratios but as probabilities:
* Precision is the probability that a (randomly selected) retrieved document is relevant.
* Recall is the probability that a (randomly selected) relevant document is retrieved in a search.

This page contains some illustrations to better understand the definitions of Precision and Recall:
http://newadonis.creighton.edu/hsl/searching/Recall-Precision.html

Also, this paper relates Precision-Recall curves to the Receiver Operator Characteristic curves:
http://www.biostat.wisc.edu/~page/rocpr.pdf

Finally, this page is further explaining the concepts and elaborating on the F-measure metric that tries to combine Precision and Recall into one measure:
http://streamhacker.com/2010/05/17/text-classification-sentiment-analysis-precision-recall/

Splitting a video into frames using ffmpeg

I needed a quick way to split a video sequence into its individual frames and save those frames as images. FFmpeg seemed like a good handy solution. And the short answer is to do this:

ffmpeg -i inputfile.avi -f image2 image-%07d.png

This was a solution proposed in this thread. And for more information you can refer to the detailed documentation of FFmpeg.

PS: FFmpeg is available on Windows and you can get the installers here. If you just need the executable then download one of the static builds.

Deadline Email Notifications using Bugzilla Whining Feature

Bugzilla is a great project. Yet, customizing it to your needs is an overwhelming task!

I was assigned a task that our Bugzilla server should automatically notify the task assignees upon the deadline. After a lot of search and reading, here is what I reached at...

First, since our ISP did block our Sendmail server, we needed another way to send emails from Bugzilla... and this post saved my life:

http://www.dawood.in/bugzilla_alerts_using_gmail.html

The perl module Email::Send::SMTP::TLS was installed using the CPAN command:

perl -MCPAN -e 'install Email::Send::SMTP::TLS'

Then, from the administration, the email service was set to SMTP::TLS as per the instructions in the previous link.

Finally, the bugzilla code was modified as per the instructions, as follows

/usr/share/perl5/Bugzilla/Mailer.pm

if ($method eq "SMTP" || $method eq "SMTP::TLS") {

my $smtp_server;

my $smtp_port;

($smtp_server,$smtp_port) = split /:/,Bugzilla->params->{"smtpserver"};

push @args, Host => $smtp_server,

#username => Bugzilla->params->{"smtp_username"},

User => Bugzilla->params->{"smtp_username"},

#password => Bugzilla->params->{"smtp_password"},

Password => Bugzilla->params->{"smtp_password"},

Hello => $hostname,

Debug => Bugzilla->params->{'smtp_debug'};

push @args, Port => $smtp_port if($smtp_port);

}

Now I have a Bugzilla system that can send emails... well, what about automated notifications? The whining feature came into play...

The whining feature was used to automatically notify the users about deadlines. The was inspired by the instructions in the following links (instructions of Marc):

http://markmail.org/message/2y67gfpa2ocu7ewa

http://groups.google.com/group/mozilla.support.bugzilla/browse_thread/thread/87159cba2d617553

The whining feature relies on Saved Searches. A 'saved search' is to be created and the search should be done for every member of a group. To execute the saved search for everybody, a group should be created that contains all the users of Bugzilla. This could be done by using the user RegExp '.*'.

Then, the Save Search 'Tasks due soon' was created. It searches all the products for bugs that are open (NEW, ASSIGNED, REOPENED). Then the advanced search (boolean charts) was used to search for only the bugs for which the current user is assigned (Assignee 'is equal to' '%user%') or cc'ed (CC 'is equal to' '%user%'). This was done using the bugzilla Pronoun Substitution feature (http://www.bugzilla.org/docs/2.20/html/query.html). In addition, the bug should have its deadline due soon or overdue (Deadline 'is less than' '+3d'). The final saved search looks like this:

Finally, from the Whining feature in administration, all the pieces were put together as follows:

In sum, the saved search entitled 'Tasks due soon' should be run 'Each day' 'at 15:00' for every member of the group 'everybody' and the search results should be sent 'one message per bug' to the '%user%'.

One question may arise, since we are running the search for all the users, how did we guarantee that the email will be sent to the bug assignee only and not to everybody? This was done using the Pronoun Substitution feature of Bugzilla. Basically, '%user%' is being automatically substituted for by the current user for which the search is done. So, if the current user is not the assignee or on the CC of the bug, there will be no search results, and consequently, there will be no emails sent!

We are not done yet! One final thing needs to be done. Add the whine.pl to your hourly cron, so that cron would automatically run the whine.pl script and Bugzilla start whining. This was done by adding a script 'bugzilla3' under '/etc/cron.hourly/' on a Debian Lenny system. The script was adopted from the 'bugzilla3' script under '/etc/cron.daily/'. Its code is:

#!/bin/sh

set -e

umask 022

doit()

{

if [ -f /usr/share/bugzilla3/lib/whine.pl ] ; then

su www-data -c /usr/share/bugzilla3/lib/whine.pl

fi

}

# process the main configuration: /etc/bugzilla3

doit

exit 0

That is all!

Sound in GNU Octave on Debian Lenny

I was very disappointed by the poor sound capabilities of the GNU Ocatve project compared to MATLAB. The playaudio() function was inconvenient with nasty '/dev/dsp device or resource busy' messages. This was until I installed the octave-audio package in Debian and had the powerful sound() and soundsc() functions working.

The nasty thing is that those functions didn't work out of the box. They needed to call a play command from the shell. Finally, I discovered that this command is part of the sox package on Debian. I apt-get install sox and now, sound() and soundsc() are working perfectly!

Installing OpenOffice.org 3.0 on Debian Lenny (stable)

There is no OpenOffice.org 3.0 in lenny-backports till now and I need it! So, I grabbed it from sid (unstable). This discussion helped me a lot:
http://www.linuxquestions.org/questions/debian-26/openoffice-3-in-debian-lenny-679610/

In short, you need to

1. add to your repositories (/etc/apt/sources.list)
   

   deb http://www.backports.org/debian lenny-backports main contrib non-free

2. Also add to your repositories
   

   deb http://http.us.debian.org/debian sid main

3. To stay with the stable and not install packages from unstable by default, we need to add the following to /etc/apt/apt.conf APT::Default-Release "stable";
   
   
4. Then do aptitude install openoffice.org/unstable

Voila! Hope it does not break your system!

UPDATE: OpenOffice.org is now available in Lenny-backports. Now you do not need step 2. So, instead type aptitude install openoffice.org/lenny-backports

The long-waited Qt 4.5 Released :)

Qt 4.5 is now officially released.
http://www.heise.de/english/newsticker/news/133888
http://doc.trolltech.com/4.5/

With the LGPL v2.1 licensing option added, I expect the widespread adoption of Qt into commercial applications!

Qt 4.5 is even better suited for the medical field, combining reliability, performance, flexibility, nice look-and-feel, among other advantages.

Qt 4.5 now has -graphicssystem command-line option to switch the graphics engine of Qt applications to raster or even OpenGL without any change to the code. Yes, your application can draw itself using OpenGL without requiring you to change a single line of code, check:
http://labs.trolltech.com/blogs/2008/10/22/so-long-and-thanks-for-the-blit/

Moreover, you can have a custom widget plotting itself using OpenGL without requiring the rest of the application to use OpenGL, check:
http://labs.trolltech.com/blogs/2008/06/27/accelerate-your-widgets-with-opengl/

The LGPL v2.1 has one drawback regarding the use of templates in C++, check:
http://lab.obsethryl.eu/content/lgpl-21-qt-45-and-c-templates
However, it is stated on the Qt Labs Blogs that Nokia is working on an LGPL exception to overcome this problem. Let's wait and see :)

With a framework so powerful and flexible like Qt, one can ask: Are we watching the dawn of Qt and the sunset of Java and .Net??

Adding PDF headers, footers, and watermarks in GNU/Linux

Recently I needed to add a header/footer to an existing PDF file. With some help from this link, I figured it out:
http://coding.derkeiler.com/Archive/Perl/comp.lang.perl.misc/2005-04/msg00151.html

First, use OpenOffice.org Writer (for example) to generate a single-page PDF file that has the needed header/footer/watermark.

Second, use pdftk's background option. Quoting from the man page of pdftk:

pdftk in.pdf background back.pdf output out.pdf

the back.pdf is the file you created that has the needed header/footer/watermark. The in.pdf is the file you need to apply header/footer/watermark to. The result, of course, is saved into out.pdf.

You can also use the dual option stamp instead of background. This will overlay the back.pdf on top of every page of in.pdf.

Debian Lenny is out... at last!

Debian 5.0 (codenamed "Lenny") is out. Check:
http://www.desktoplinux.com/news/NS4199843436.html

It appears that Debian 5.0 (aka "Lenny") will soon take its big binocular eyes out into the wider world. The Debian project has completed the first release-candidate of Lenny's installer, which features much-improved support for ARM-based devices, along with much faster installation from "live" CDs

The next stable release of Debian is codenamed "Squeeze". Check:
http://www.debiantoday.com/debian-lenny-successor-gets-named/

Prepare yourself for Squeeze, that’s right the little green guy. Continuing with the Toy Story names the Debian project has decided to name the next release in the line up after the three eyed green alien who fears “the claw” at Pizza Planet in the movie Toy Story

Free Books on "Design Patterns in C++" and Qt

Today is my lucky day :)

I found two free books about Qt and Design Patterns.

The first book is
C++ GUI Programming with Qt 4

The first edition of this book could be downloaded _legally_ for free here:
http://www.qtrac.eu/C++-GUI-Programming-with-Qt-4-1st-ed.zip
http://www.qtrac.eu/marksummerfield.html


The second book is a magnificent one. It is the first book I see to explain C++ and Design Patterns in a Qt-ish style!

The book is free as in freedom! It is released online under the Open Publication License (OPL) here:
http://oop.mcs.suffolk.edu/opensource/

This is NOT a book, it is a complete PROJECT that I think people should actively contribute to... The bugzilla server for this project and many useful materials could be found here:
http://cartan.cas.suffolk.edu/moin/OopDocbook


For convenience, you can download the whole book compiled into chm format here:
http://lawand.wordpress.com/site-map/computer-science/e-books/introduction-to-design-patterns-in-cpp-with-qt-4/

Enjoy the cute Qt!

An open source self-balancing robot

What a nice project!

http://www.mtoussaint.de/tobb/index.html

Interesting: The Evolution of Tech Companies’ Logos

I just got this from a friend :)
http://www.neatorama.com/2008/02/07/the-evolution-of-tech-companies-logos/

union, struct, and bit fields in C and C++

It took me some long time to comprehend the issues involved in using unions, structs, and bit fields in C and C++.

First, for the definition of unions and structs, take a look at this link:
http://publib.boulder.ibm.com/infocenter/comphelp/v8v101/index.jsp?topic=/com.ibm.xlcpp8a.doc/language/ref/strct.htm

Unions could be used together with structs to reinterpret the same location in memory with different interpretations (similar in spirit to the C++ reinterpret_cast). For examples on this see:
http://www.anyexample.com/programming/cplusplus/example_of_using_union_in_cplusplus.xml

C++ extends the unions and structs, for example see:
http://www.yolinux.com/TUTORIALS/LinuxTutorialC++Structures.html

Bit fields are used to define special-sized integers. This is useful for packing data (e.g. flags) together into one memory location. This, however, is a bad habit and should be avoided as the compiler-generated code to access bit fields is usually inefficient! For more examples on this, see:
http://en.wikipedia.org/wiki/Bit_field
http://publib.boulder.ibm.com/infocenter/macxhelp/v6v81/index.jsp?topic=/com.ibm.vacpp6m.doc/language/ref/clrc03defbitf.htm
http://www.cs.cf.ac.uk/Dave/C/node13.html
http://www.daniweb.com/forums/thread55765.html
http://blogs.msdn.com/oldnewthing/archive/2008/11/26/9143050.aspx
http://osr507doc.sco.com/en/tools/ANSI_F.3.9_Structs.unions.html

Finally, unions and structs could be anonymous! You do not have to give a name to a union. In this case its members become directly accessible within the same scope. Wikipedia calls this "untagged union". For an example see:
http://www.velocityreviews.com/forums/t286529-union-with-anonymous-struct.html

Debugging, Profiling, Memory Leaks and more!

I have been searching for open source tools to assist in detecting memory leaks and application profiling. Also, static code analysis was another goal.

For static code analysis I found:
CppCheck (http://en.wikipedia.org/wiki/Cppcheck)

I used it to its full power by invoking (at the top level of the project):

cppcheck --recursive --all --errorsonly --style

Also, SPlint (http://en.wikipedia.org/wiki/SPlint) seems promising, though it is for C code only (no C++).

A comprehensive list of static code analysis could be found in Wikipedia:
http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis

About application profiling, the GNU Profiler (gprof) does a good job. It is light-weight and has support in GCC. This is an excellent article about using gprof:

• Programmer's Toolkit: Profiling programs using gprof(http://linuxgazette.net/100/vinayak.html)

Of course, you can best visualize the output of gprof using the (kprof) KDE front-end.

Memory leak detection has many options. Of the best is using Valgrind(http://valgrind.org/). However, I found an interesting and promising feature that has been added to the gcc. This feature is called Mudflap. It is easy to use as a compiler flag and adds memory debugging support directly into your application using libmudflap. More interesting information could be found here:
http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging
http://gcc.fyxm.net/summit/2003/mudflap.pdf
It is so promising that it will be integrated into the QNX IDE:
http://qnxdevtools.blogspot.com/2008/02/mudflap.html
It can print the memory leaks on program exit via the (-print-leaks) option

Memory debugging tools are so many. There is a list on Wikipedia article:
http://en.wikipedia.org/wiki/Memory_debugger

The task of memory leak detection is achievable using lightwieght tools. These tools basically replaces the default C library malloc() and free() with special debugging implementations. Notable is mtrace(), the memory debugger built in the GNU C library:
http://en.wikipedia.org/wiki/Mtrace

A good article about memory leak detection in C++:
http://www.linuxjournal.com/article/6556

And finally, all what you can dream of are collected in one article, in the GRASS wiki here:
http://josef.fsv.cvut.cz/WIKI/grass-osgeo/index.php/GRASS_Debugging


Hope you find this useful!