Ubuntu21 May 2012 04:03 pm

In Ubuntu we receive a multitude of package installation failures due to people misconfiguring their /etc/default/grub file. During this release cycle we plan on putting some work into helping people resolve these misconfigurations and subsequently we need to understand what changes people make to /etc/default/grub.

Fortunately, the grub apport package hook includes /etc/default/grub in the information that it gathers and uploads to Launchpad. So we just need to get all those attachments from Launchpad. Sometime ago I did this for the X org team with xorg.conf files and used the Launchpad API directly. However, now we have lp-grab-attachments, part of the lpltools package, which can make this much easier:

lp-grab-attachments --package grub2

Now I have a bunch of /etc/default/grub files on my local system and I can use grep to search them locally. The most common errors all related to quoting as we had expected. People either neglect one of the quotes in the pair or use unicode quotes instead of ASCII ones.

Ubuntu01 May 2012 03:56 pm

A short time ago I ran across a couple of bugs with the same title reported by the same Launchpad user something like:

package xyz failed to install/upgrade: subprocess post-installation script returned error exit status 139

Upon further investigation I discovered that those two bugs were duplicates but it made me wonder how many other Launchpad users had reported the same bug more than once.

I figured that the easiest way to query this information would be to use the Ultimate Debian Database, which also helpfully contains Ubuntu bug information. Using my local copy, I queried for Ubuntu bugs with the same title and reported by the same user about the same source package. The vast majority of these were package installation failures, some of which actually had consecutive bug numbers and others where the user just kept trying to upgrade over time without resolving the original issue.

I didn’t keep great stats but the quantity of apport-package bugs decreased by at least 175 when I got through marking duplicates.

Ubuntu26 Mar 2012 02:23 pm

I find bug heat a useful way to determine the impact of a bug as it takes into account the number of duplicates, number of subscribers and quantity of people affected by the bug.  Additionally, as a defect analyst for a team that is interested in bug reports about a large number of packages, I’d like to be able find out which of our bugs (across all our packages) have the greatest heat in Launchpad. It would be helpful to see this data in a chart so I can visually distinguish differences in heat and also have the bugs grouped by package rather than having them in a list ordered only by bug heat. (The chart will also help because Launchpad times out when creating the report for the foundations-bugs team.)

To this end I’ve collected data on the five hottest bugs for each package to which my team is subscribed. I’ve used it to make following chart which only displays packages where the package’s hottest bug is greater than 10% of the hottest bug overall.

Foundations’s Hottest Bugs

The chart will display the bug number and title when you mouse over a point on it. It is also possible to control which of the five hottest bugs are shown. Again, I’ve created these for the desktop, server, foundations and ubuntu-x-swat teams. There are also charts for every package set in Precise.

Launchpad and Ubuntu09 Mar 2012 03:45 pm

In the Ubuntu Bug Squad we frequently talk about how to perform tasks on IRC but for some people it is a lot easier to learn something by seeing it done. Subsequently, I’ve created a video of how to confirm a bug task in Launchpad. There are actually three different ways to do it and the best is the last one in the video.

I created the video mostly following the instructions at the Canonical design blog. I recorded separate screencasts for each way of confirming the task, that way they can be easily replaced if Launchpad changes. Text slides, instead of audio, were used to make production easier and also to allow for easy translations.

Ubuntu22 Feb 2012 04:41 pm

I work with a team, the Ubuntu Foundations team, that has an interest in a lot of source packages and naturally bugs reported about those packages. To track all these packages we have setup a team in Launchpad, the foundations-bugs team, and subscribed them to the packages we care for and setup a mailing list to receive bug mail. Launchpad also provides us with a report that shows us some bug numbers, but this just shows information about where the bug tasks are as a whole. I wanted to find out which packages are receiving bug tasks right now and if that bug volume is abnormal for the package. (Some packages will always be receiving lots of bug reports.)

To this end I’ve created created a chart, using the packages the foundations-bugs team are subscribed to, that displays the quantity of bug tasks opened in the past 7 days and the past 14 days. Additionally, the bug tasks for a package are classified by their reporting method. (When a bug is reported by apport it is tagged differently depending on the type e.g. if it is a crash the bug is tagged apport-crash.) This information allows us to tell if a particular package is receiving a lot of crashes or package installation failures, which is more important than a lot of tasks tagged ubuntu-bug.

Screenshot of chart

Looking at the above screenshot we can see that casper is having a spike in bug reporting activity as the bar length for the 7 day and the 14 day period are equal. This likely has something to do with the 10.04.4 testing that was going on but is worth investigating. grub2 also seems to have a large number of apport-package bugs which also should be investigated. The chart makes this really easy as each section of the bar takes you to a Launchpad search for all bug tasks about that package with the relevant tag. Additionally, the package name is also a link for all the bugs about the package.

I’ve made the same chart for the desktop, server and ubuntu-x-swat teams. If you have a team that is subscribed to Ubuntu packages I’m happy to make one for you too. Additionally, there are also charts for every package set in Precise so one of those might have a list of packages in which you are interested.

Ubuntu14 Nov 2011 05:36 pm

As an Ubuntu user I end up encountering some bugs and as I’ve been using Ubuntu for some time now I’ve reported quite a few of them, 83, which are still open.

As a defect analyst and bug triager I know developers and triagers can’t get get to every bug report and some may end up sitting in one state or another. However, one way I can improve this situation as a bug reporter is by reviewing the bugs I’ve reported after a new version of Ubuntu has been released.

For example, today I looked at the bugs I reported and discovered that some were about packages no longer available in Ubuntu, they also were not eligible for a Stable Release Update, so I set those bug reports to Invalid. I also found a bug report that was fixed in Ubuntu 11.10 and a bug report with new information from upstream that I added to the Ubuntu bug report.

You can find bugs you’ve reported about Ubuntu by going to bug reports about Ubuntu in Launchpad and clicking on “Bugs reported by me” in the portlet on the right hand side. I encourage you to review the bugs you’ve reported and see if they are still relevant (are they fixed or no longer valid?) or could benefit from more information (updated steps to recreate the bug?). Additionally, if the bug’s status is New you could ask a fellow Ubuntu user to recreate the bug and set it to Confirmed.

Ubuntu08 Sep 2011 08:38 am

I’ve talked about preventing bugs from being reported by apport and creating duplicate signatures so the apport retracer can mark bugs as duplicates of another. There is another process for preventing bugs from being reported and that is writing bug patterns something I’ve given a class on.

When apport begins the bug reporting process it first checks to see if the bug being reported matches a pattern and if the bug does it redirects the reporter to the bug, or wiki page, matching that pattern. Additionally, the apport retracer helps us identify bug reports that would benefit from a pattern (those with 10 duplicates) by tagging them bugpattern-needed and subscribing the Ubuntu Bug Control team to the bug report.

After we write a bug pattern we should do three things to the bug report:

1) Remove the tag bugpattern-needed
2) Add the tag bugpattern-written
3) Unsubscribe Ubuntu Bug Control

While modifying the tag is essentially one operation in the Launchpad web interface there are still two steps involved here. This was two too many in my opinion! I wanted to write a bazaar plugin to modify the Launchpad bug report appropriately after the bzr branch with the pattern was pushed to Launchpad. I became extremely motivated yesterday when I saw about a dozen merge proposals from Vadim Rutkovsky (thanks!) for bug patterns.

So now there is a bazaar plugin in the bug patterns branch that will properly take care of a bug report once a pattern is written for it. To use it you need to have python-launchpadlib-toolkit installed, for working with the Launchpad API, and create a symlink to bugpattern_written.py in your ‘~/.bazaar/plugins/’ directory. Enjoy!

Ubuntu01 Sep 2011 09:43 am

One thing that Launchpad doesn’t do very is searching of bugs for text. Luckily, having access to a database of all Ubuntu bugs makes this quite easy and the Ulitmate Debian Database is just such a database and accessible to everyone. I keep a local copy for cases just like this one.

Steve Langasek recently asked me about package installation failures like bug 837525 and was wondering if there were more like it. Looking at that particular bug I realized I wanted to do a search of all bug titles with “error writing to ‘‘: No such file or directory”. Using this in a Launchpad search returns zero results. This isn’t very helpful when I know there is at least one!

To the Ultimate Debian Database we go:

udd=# select bug, title from ubuntu_bugs where title like '%standard output%directory%';

I cheated a bit there as I noticed there were some translated errors in the bug titles but saw directory was common amongst all of them. This produces the following results:

bug | title
584085 | package pidgin-libnotify (not installed) failed to install/upgrade: error writing to '<standard output>': No such file or directory
643255 | package xsane-common 0.996-2ubuntu3 failed to install/upgrade: errore nello scrivere su "<standard output>": Nessun file o directory
768866 | package gcalctool 6.0.1~git20110421-0ubuntu1 failed to install/upgrade: errore nello scrivere su "<standard output>": File o directory non esistente
815008 | package screen 4.0.3-14ubuntu7 failed to install/upgrade: errore nello scrivere su "<standard output>": File o directory non esistente
815268 | package soundconverter 1.4.4-2 failed to install/upgrade: error writing to '<standard output>': No such file or directory
815606 | package freepats 20060219-1 failed to install/upgrade: error writing to '<standard output>': No such file or directory
823123 | package gawk 1:3.1.7.dfsg-5 failed to install/upgrade: errore nello scrivere su "<standard output>": File o directory non esistente
824113 | package linux-headers-2.6.32-33 2.6.32-33.72 failed to install/upgrade: error writing to '<standard output>': No such file or directory
825911 | package html2text 1.3.2a-15 failed to install/upgrade: error writing to '<standard output>': No such file or directory
829625 | package ntpdate 1:4.2.6.p2 dfsg-1ubuntu5.1 failed to install/upgrade: error writing to '<standard output>': No such file or directory
834329 | package krusader 1:2.3.0~beta1-1 failed to install/upgrade: error writing to '<standard output>': No such file or directory
837327 | package zeitgeist-fts-extension 0.0.7-0ubuntu1 failed to install/upgrade: error writing to '<standard output>': No such file or directory
837525 | package PACKAGE failed to install/upgrade: error writing to '<standard output>': No such file or directory
(13 rows)

The point being that the Ultimate Debian Database can be a quite valuable resource when working with Ubuntu bugs. I’d like to say a big thanks to the people who have worked on building and providing this database as a resource.

Ubuntu29 Aug 2011 02:49 pm

Over the past six months or so I’ve been looking at package installation failure bug reports and started noticing that we have allowed the reporting of a lot of bug reports that we really shouldn’t have.

As one example dpkg reports multiple types of failures when it is unable to deal with a local deb file. Encountering a short read while working with the deb file is the most common error and is indicative of a hardware or memory issue on the system. While we could provide the bug reporter support on how to determine where the issue lies and how to test the hardware this is not a bug report and does not belong as a bug report Launchpad.

To prevent further bug reports like this I’ve updated apt, which creates most apport-package bug reports, to not report the issue if it is due to a short read. Additionally, I’ve cleaned up most of these bugs that were already reported in Launchpad and as a part of that process I tagged this type of bug ‘short-read’. Querying Launchpad today there are approximately 202 of these short read bug reports that we really didn’t need and were cluttering up our bug lists.

This is just one example though – there are many more cases like this…

210 bug reports where the package installation failure was due to a segfault in an underlying part of the package installation process. (These are better reported as a crash on that underlying part.)

153 package installation bug reports where there was an fsys-tarfile error.

94 package installation bug reports where there was an I/O error on the file system containing ‘/’, ‘/usr’ or ‘/var’.

82 bug reports regarding package installation failures where a local deb file was inaccessible likely due to hardware or memory issues.

53 bug reports regarding package installation failures where the deb file was corrupted again likely due to hardware issues.

27 bug reports regarding package installation failures where there was a failure when writing the contents of the package to disk.

13 bug reports regarding package installation failures where there was a failure to read the contents of the package.

At least 40 bug reports where people were trying install a package on a Live Media build of Ubuntu and needed a new version of update-initramfs for the package installation to work.

So far that’s a total of 874 package installation failures that we didn’t really need reported in Launchpad. But wait there’s more! I’ve very recently been looking at ubiquity bug reports and come to find out a lot of these are due to media or hardware errors too. I’ve tagged some of these ‘squashfs-error’ and others ‘media-error’ depending on which type of error message appeared in the system’s syslog. The total of these two tags is 204 bug reports.

This brings our grand total of unnecessary bug reports, for this experiment, to 1,078 and this doesn’t include bugs with an extraordinary number of duplicates like a ubiquity bug affecting the behavior of debconf with hundreds of duplicates.

The point being that when we look at bugs that have been created in Launchpad by an automatic process we should look at whether or not we really need this specific type of bug report and if not prevent it from being reported at all. If you aren’t sure how to stop bugs from being reported or clean up hundreds of bugs at once I’m happy to help.

Ubuntu11 Aug 2011 10:01 am

As a part of the Oneiric development cycle I wanted to work on dealing with the overwhelming number of package installation failure bug reports. These are identifiable by the tag apport-package and a ProblemType of Package in the bug description. These reports are currently created when a package fails to install on a development release of Ubuntu or a stable release. This is useful since we have more than 15,000 packages in the archive there are likely some that are not used by people who use the development release. However, this is problematic for stable and development releases when there is no automatic duplicate detection of these bug reports as you end up in a situation where the same failure is reported multiple times and your bug list becomes cluttered.

Thankfully Martin Pitt added in support to apport for a duplicate signature for any type of apport bug report. I’ve now uploaded a version of apport in Oneiric that creates a DuplicateSignature for package installation failures using information from the dpkg terminal log file that will then allow the retracer to automatically mark duplicates of package install failures! This is even more useful now that Michael Vogt has modified dpkg so that apport receives the dpkg terminal log untranslated. I’ve also made it so that when the bug is reported if there is a conflict between two packages trying to use the same file the bug report is tagged ‘package-conflict’.

What about all those existing package installation failures? Well I happen to have a local cache of all the DpkgTerminalLog file attachments from apport-package bug reports. I used the same method used in apport to create the duplicate signature for all those existing apport-package bug reports. Then using the md5sum, thanks Kees, of those duplicate signatures I was able to identify approximately 200 bug reports that were duplicates of other package installation failures! And of course I went through and tagged all the already reported ‘package-conflict’ bug reports I could find.

« Previous PageNext Page »