Fix Samsung Captivate (Galaxy S) battery life after update to Android 2.2 (Froyo)

The Update

Finally, Samsung has released the Android 2.2 Froyo update for the Galaxy S (AT&T Captivate). For reference, before the update I was getting about one-and-a-half days of battery life with normal usage (wi-fi and sync always on). Android 2.2 touts a 2x-5x improvement in CPU performance and improved memory management, both contributing to faster and smoother performance.

Sold! Where do I upgrade?

The Aftermath

Unfortunately, shortly after installing the update I notice an unacceptable battery drain rate. From a full change and with, at most, 10 minutes of use, my battery was dead about eight hours later. Wait a second, I thought performance was supposed to be better! Did this improved performance mean I had to sacrifice to the battery gods? This is not at all what I signed up for.

Maybe the first charge cycle was a fluke, I told myself. Well, it wasn’t.  Two more charge cycles came and went, and I was still experiencing no more than about eight to ten hours of battery life.  At this point, I’m searching Android forums for solutions and finding many similar battery life horror stories.  It wasn’t just Samsung Galaxy phones either. Reports about other HTC, Samsung, etc. phones were also prevalent. However, I could find no definitive solution to the problem.  Suggestions ranged everywhere from get a new battery to perform a factory reset to turn off wi-fi and many, many more.  At this point, I’m seriously considering reverting back to Android 2.1 (or rooting and going to 2.3.4), because less than half a day’s worth a battery is no way to live.

The Solution

Then, there was one suggested solution I came across that actually seemed plausible.  I’m sure as hell not going to reset the phone and spend the time reloading all my data and apps, and nothing is wrong with my 6-month old battery as it was working just fine before the update to 2.2.

The suggested solution was to reset the battery calibration table. The theory here was that the software estimates the battery usage and life, and is what tells the phone what percentage the battery is at and when the phone should shut down due to critically-low remaining capacity. By reseting the calibration table, the software would relearn the battery capacity.  OK, I buy that, so how do I do this? Well, the suggestion was to install the Battery Spy app, open the app, then follow: Menu button -> Battery Profile -> Reset calibration table -> Yes. Afterwards, you must let the battery drain all the way down, then charge it back to full (I did this with the phone off), and then let it charge all the way down again.

The Numbers

Once changed to full, I performed a couple more actions: I turned off auto -sync and also changed the wi-fi sleep policy option to turn off wireless when the screen is inactive (Menu -> Settings -> Wireless and network -> Wi-Fi settings -> Menu -> Advanced -> Wi-Fi sleep policy -> When screen turns off).  The last option here is a default policy that was changed in Android 2.2; previously, “When screen turns off” was the default, but in 2.2 the default changed to “Never”. From what I could gather, the “When screen turns off” setting was a problem because if the phone sat idle (e.g. while you were asleep) it would switch to 3G (even if it had excellent wi-fi signal) and potentially rack up unintended data usage. Another reason for the settings change, again from what I could gather, was that the wi-fi transmitter uses less power than the cell radio, and less power = more battery life.

To my astonishment, 4 days 22 hours and 14 minutes later the phone was down to 2% battery, but still on despite normal usage and not ever going near a charger.  Wow, this is unbelievable!  Nearly five days of charge, that was more than twice the battery life I’ve ever experienced on this phone; but, was this due to the battery calibration or the settings I changed?  Time to test. I next charged the phone up to 100% again, then turned auto-sync back on.  1 day 20 hours and 45 minutes later, battery is at 5%.  OK, big hit there, but still better than I had ever experienced before the upgrade to Android 2.2.  Next, I changed fully once more, and then set the wi-fi sleep policy back to “Never”. That time, 2 days 6 hours and 11 minutes later, it’s at 4% charge.

Conclusion

Your mileage may vary, but it seems that the battery calibration is what did the trick. At over two days of average battery life now, I’m once again a happy Android user and am enjoying a bit faster performance to boot.  Before I wrap this up, I’ll leave you with one last tip: the CPU Spy app. This nice little app gives you a breakdown of how much time the CPU has spent in its various clock speeds (e.g. 1000, 800, 400, 200, and 100 MHz, and deep sleep), and the far majority of the phone’s time should be in the deep sleep mode.  If it’s not, then there’s likely some app or background process that is keeping the phone from entering this low-power state.  Find out what’s keeping it out of deep sleep, and your battery will thank you.  Hope this was helpful, and I would definitely try the battery recalibration before taking more drastic measures that involve losing all your apps and data.

Drawing Petri nets

I recently went looking for tools to easily draw Petri nets, a modeling language for distributed systems, and here’s what I found:

• petri-nets LaTeX package: A LaTeX package written by Franck Pommereau specifically for drawing Petri nets that makes use of PSTricks as a basis for the actual creation of the drawings. I didn’t actually try this package out because, after skimming through the documentation, it seems that nodes must be positioned using the Cartesian coordinate system. Not really what I’m looking for since, ideally, I’d like to be able to simply define the graph’s structure and not have to worry about explicitly positioning each object.
• PGF (“portable graphics format”) LaTeX package: A LaTeX package for creating graphics, similar to PStricks. Unlike PStricks, this package has built-in support for Petri nets. It provides out-of-the box support for defining place and transition nodes, drawing edges, and placing a defined number of tokens within a node.  If interested, Section 3 of the PGF manual provides an excellent and detailed tutorial on building a Petri net graph. While the syntax for describing a graph seems easier than the petri-nets package above, there are still cumbersome aspects to the definition. For example, positioning is still a bit of a manual process. You must specify “left of”, “right of”, “above of”, and “below of” layout commands for each node, though at least you do not have to deal with coordinates. Now granted, precise layout might be exactly what’s needed in a LaTeX document, but I’m looking for something easier and faster to define.
• Graphviz: visualization software that can auto-generate directed graphs a specification given in its DOT language. It’s easy to create the place and transition nodes using subgraph declarations, and connecting the nodes requires no manual positioning at all. Below is an example graph:

  File barber.dot:

  digraph G {
      subgraph place {
          graph [shape=circle,color=gray];
          node [shape=circle,fixedsize=true,width=2];
          "customer waiting";
          cutting;
          "idle barber";
          "customer paying";
      }
      subgraph transitions {
          node [shape=rect,height=0.2,width=2];
          enter;
          "start cutting";
          "finish cutting";
          exit;
      }
      enter -> "customer waiting";
      "customer waiting" -> "start cutting";
      "idle barber" -> "start cutting";
      "start cutting" -> cutting;
      cutting -> "finish cutting";
      "finish cutting" -> "customer paying";
      "finish cutting" -> "idle barber";
      "customer paying" -> exit;
  }

  When run through dot:

  dot -T png barber.dot -o barber.png

  …you get an image that looks something like this:

  

  However, there doesn’t seem to be a very good way for drawing the tokens within the place nodes. Nodes can define text labels, but using periods for tokens isn’t very readable. Alternatively, since nodes can specify a background image, you could generate a few images with different numbers of tokens and use the correct image to represent the desired number of tokens, while still having a text label too.

  If you intend to put your graph into LaTeX, check out the dot2tex tool (written in Python) which will convert Graphviz’s xdot output to LaTeX using PSTricks and PGF. Not only does this allow you to create beautiful graphs in your LaTeX documents, but it also allows your labels to be typeset by LaTeX (including the use of math mode, which is very nice).

• Finally, there’s your favorite drawing or image manipulation program. It’s quick-and-dirty, but perhaps the easiest solution I examined.  While it might be a pain to edit or reconnect nodes, it’s certainly easy to set the positioning you want. If your program supports snapping edges and labels to nodes, such as OODraw, then moving or reconfiguring portions of the graph are even easier.

If you’ve found a better tool for drawing Petri nets, please share your experiences.

Dude, where’s my color picker?

It’s not something I use too terribly often, but today I found myself missing the KDE color picker applet, a tiny applet that gives you an eye dropper you can use to quickly sample any color on your screen and copy its hex value to the clipboard. I could have sworn it was available in Kubuntu Gutsy by default. Turns out that, in Hardy at least, it comes as part of the kicker-applets package.

Eclipse security component errors

Just got through installing eclipse on an Kubuntu Hardy box, and got the following error during the splash screen:

Could not initialize the application’s security component. The most likely cause is problems with files in your application’s profile directory. Please check that this directory has no read/write restrictions and your hard disk is not full or close to full. It is recommended that you exit the application and fix the problem. If you continue to use this session, you might see incorrect application behaviour when accessing security features.

The solution seems to be to create the directory: ~/.mozilla/eclipse

The corresponding bugs where the above solution was found:

• Ubuntu bug
• Eclipse bug

Bill Gates at UT

Bill Gates was on The University of Texas at Austin campus a couple weeks ago to talk about "Software, Innovation, Entrepreneurship, and Giving Back." While unable to get a ticket to see the talk in-person, I did watch the live webcast (archive now available). Below is my summary of his talk with a couple of my own comments mixed in.

Bill opened his talk with the mentioning of the incredible advancement of computing. How computers are getting smaller, how the number of transistors on processors are doubling, how more natural computing interfaces are on the way. He talked about how TV delivered over the internet is going to revolutionize that field with a level of personalization and interactivity not seen before. He mentioned how the IT field will be changing as much of the system administration work done by humans today will be taken over by software systems that monitor and fix the systems automatically.

Bill talked about how the software industry is still in its infancy and that while user interface tools, such as touch screens and tablet pens, are cheap, the software to effectively make use of these tools is still very far behind. Now, if you are a fan of Ray Kurzweil and The Age of Spiritual Machines, as I am, this might not surprise you. In his book, Ray details the exponential advancement of computers and mentions that while we might have computers with the computing power equal to that of a human brain by 2020, the software to utilize that power will likely lag by at least a decade.

Bill also talked about how computing and software is becoming a more necessary tool for all branches of science, and played a short video showing visualization of mouse-brain scans. Research today is requiring increasingly large amounts of data and computation, and Bill mentioned that he would like to get more of Microsoft's software into the hands of university researchers. He announced a program called DreamSpark, which allows students free (as in beer) access to Microsoft development software, including Visual Studio and Windows Server. However, at the end of the talk when a student asked Bill about the openness and availability of Microsoft's 3D visualization software to the world of academia, Bill responded that Microsoft's products might not fit all of their needs but that Microsoft wants to make sure that all of their software available to researchers. So Bill says that he wants to make this software available, yet if I go visit the DreamSpark website I see only mainstream products like Visual Studio and nothing at all that looks aimed at helping university researchers.

Another interesting topic was Bill's discussion about Microsoft's research department and how the Bill & Melinda Gates Foundation is helping people in poor countries. Bill talked about how he would like to see computing become available to everyone, including the poorest two billion people of the world. Traditional products and business models for the rich countries of the world are no good in the poor countries of the world. For example, personal computers, no matter how cheap you sell them for, are useless in places with zero literacy, training, networking, and electrical power. Starting this summer, these kinds of issues are what Bill will be focusing his attention on, as earlier in the talk he announced that he will be shifting his work schedule to work full time for the foundation and part time for Microsoft.

Bill stated that medical research for issues faced by the poor countries of the world is currently in an extremely sad condition. There are millions of children per year that die from diseases that are non-existent in the rich third of the world's countries, yet no one is willing to help because there is no profit to be made. For example, ten times more money goes towards researching baldness than towards researching Malaria, which, unlike baldness, kills millions each year. Bill mentioned that with no money driving innovation in these areas, it is going to take government policies, philanthropy, and people with values to take the initiative.

Bill went on to talk about a specific instance of how groups at Microsoft are making a difference. Ideas such as "Digital Green" where Microsoft is going around to small towns making videos of the successful farmers. TVs and DVD players are then brought to the town centers where others can learn techniques from the successful farmers. Bill mentions that this "Farmer Idol" has created a healthy competition between the farmers, where everyone wants to outdo the others so they can be included in the video. The program has resulted in significant productivity gains for these communities.

Bill concluded his speech by telling the audience of computer science and electrical engineering students to help keep their awareness of the world's poorest third of the population and that with some special effort, they can help with the problems those people are facing.

Afterwards his speech, Bill answered several questions about open source software, the decreasing number of people entering the computer science field, the work of his foundation, software in micro devices, and Microsoft’s bid for Yahoo.

Again, the video of the speech is now available, so check it out if you are interested.

Installing the latest version of Bazaar and tools on Gentoo

Currently, the bzr-svn and bzr-gtk plugins are not available in Gentoo’s package directory. Fortunately, there is exists a bzr Gentoo overlay. If you’ve never used an overlay before, it’s a way to override or add packages to Gentoo’s standard package directory. In this overlay there are a couple common plugins, as well as a patched version of Subversion that is needed for the bzr-svn plugin.

Current packages in the overlay:

• bzr
• bzrtools
• bzr-config
• bzr-dbus
• bzr-gtk
• bzr-rebase
• bzr-svn
• bzr-vimdiff
• paramiko
• subversion (patched version needed for bzr-svn)

Steps for using the overlay:

1. As usual, update your portage tree
   

   emerge --sync
   

2. Make sure you have the following line in your /etc/make.conf
   

   source /usr/portage/local/layman/make.conf
   

3. Get a copy of the overlay from launchpad
   

   bzr branch lp:///bzr-gentoo-overlay
   

4. Make sure you have the path to the overlay set in /usr/portage/local/layman/make.conf
   

   PORTDIR_OVERLAY="/path/to/bzr-gentoo-overlay"
   

5. Now you should be able to update your system, pulling in updates to any currently installed packages from the new overlay.
   

   emerge -uav world
   

6. emerge any of the new packages you want that are now available from the overlay. Note: If you install bzr-svn, make sure to re-install Subversion so that the patched version is used from the overlay.
   

   emerge -av bzr-gtk bzr-svn subversion
   

Texas Python Regional Unconference

Just a quick announcement for all the Houston and surrounding area Python lovers out there. There is a Texas Python Regional Unconference being held in a couple weeks (Sept. 15 – 16) at the University of Houston main campus. Registration is free and as simple as adding your name to the registration wiki page. I’ll be there to talk or hack Django with anyone interested. See you there!

Hacking Django, how Bazaar

A short while ago, I went on a mission to find a better way to hack Django. I needed a way to keep my patches organized and updated. So, in this article I discuss my experiences with a tool I found to help me achieve these goals…

Enter Bazaar, one of the several distributed version control systems that have been getting some hype recently. With so many to choose from, why did I choose Bazaar? Well, maybe it was because of the cool name, or maybe it was the pretty website, or maybe just because it’s written in the language I love — Python. If you are familiar with Subversion, then you will feel right at home with Bazaar. This is because Bazaar uses most of the same commands as Subversion (e.g. svn ci = bzr ci, svn st = bzr st, etc.).

Following this guide for hacking an open source project using bzr, I created a django directory to stash my django hacks. After using svn to check out the latest Django code to a directory named upstream, and turning upstream into a bzr repository too, it was time to start the real work. To re-iterate in command line speak, here is what I did:

Update: The steps below have been updated to reflect changes with bzr commands (versions >=0.15rc2).

mkdir django
cd django
bzr init-repo .            # use "bzr init-repo --trees ." here instead if using a version <0.15rc2
svn co http://code.djangoproject.com/svn/django/trunk/ upstream
cd upstream
bzr init
echo ".svn" > .bzrignore   # do not store svn metadata in our bzr repo
bzr add
bzr ci -m 'Initial import of Django.'

Usually, when you want to add a new feature to some code, you create a branch. The purpose of these, so called, feature branches is to keep the features separated. Trying to create multiple features in the same branch would just be a big mess. Creating branches in a Subversion repository is easy enough, you just svn cp the code to a new location. It’s the keeping your branch in sync with the trunk that’s not as easy.

With Subversion, you have to keep track of the revisions that have already been merged to your branch and make sure to only merge the changes that haven’t already been merged. Not so with Bazaar; Bazaar keeps track of this for you. A simple bzr merge will pull in upstream changes that have not been merged yet.

So, when I wanted to start working on a patch for ticket #2473, I created a branch:

bzr branch upstream in-and-empty-list-2473

After cd‘ing into my new branch and committing my changes, I created a diff to attach to the ticket with the command:

bzr diff -r branch:../upstream

So far, working with Bazaar has been a breeze. Without it, my Django holiday hacking wouldn’t have been nearly as fun.

is_authenticated() vs. is_anonymous()

A warning for those who might not of noticed the change. About a month ago (in [3360]), an is_authenticated method was added to the User and AnonymousUser classes. These are the classes used for Django’s default authentication system.

Previously, the template code used for displaying content based on whether or not a user had authenticated went something like:

{% if not user.is_anonymous %}
Content for logged in users.
{% else %}
Content for non-logged in users.
{% endif %}

The problem with this code is that if, somehow, the user template variable did not get populated, your template would treat the requesting user as if they were logged in.

Notice that this sort of behavior will be seen for all negative if statements. The template code {% if not variable %} will always evaluate to True if variable doesn’t exist in the context. This is because Django uses the settings.TEMPLATE_STRING_IF_INVALID value for non-existent template variables; by default, TEMPLATE_STRING_IF_INVALID is '', which evaluates to False (and not False is True).

Notice also that even using the template code:

{% if user.is_anonymous %}
Content for non-logged in users.
{% else %}
Content for logged in users.
{% endif %}

(without the "not" ) will not work in this case because if the user variable is non-existent, the requesting user will still be treated as if they were authenticated.

The new recommended way for checking that a requesting user has been authenticated in your templates is:

{% if user.is_authenticated %}
Content for logged in users.
{% else %}
Content for non-logged in users
{% endif %}

Here, if the user template variable were to not exist your template would treat the requesting user as non-authenticated, just as we would want.

Using CrackLib to require stronger passwords

Let’s face it, humans are not well adapted to memorizing strings of random characters; and hence, the average computer user is not very good at creating secure passwords. Most users create passwords made up of easy-to-remember words, like the name of a favorite sports team or maybe the name of a significant other. In this article I will show you how you can make Django require better passwords from your users.

Here are the packages you will need:

• CrackLib
• python-crack

Installing the required packages

Step 1: Install CrackLib.
CrackLib is a library for checking that passwords are not easily crackable, or in other words, it makes sure that a password is not based on a simple character pattern or on a dictionary word. CrackLib is a common package that you should be able to find in your distribution’s package manager (or, quite possibly, it could already be installed). Alternatively, you could download the source and follow the installation instructions in the README file. By default, CrackLib installs a python package named cracklib, but it does not have as many features as python-crack. (Redhat does not include the cracklib python package in its cracklib package.)

Step 2 (optional, but recommended): Install extra word dictionaries.
Packaged with CrackLib is a file name cracklib-small. (Some distributions, like RedHat, don’t include this file in their cracklib package, in which case keep reading…) This file is a dictionary of words, simply a long list of words with one word per line. These 50,000 words are a good start, but we can do better. On the CrackLib download page, there is also a package named cracklib-words. After downloading and extracting the package, you will have a single file containing 1,648,379 “words”. Many distributions also have a cracklib-dicts or cracklib-words package that maybe the same or similar to the cracklib-words file on the CrackLib website.

For even more dictionaries, take a look at these word lists. It might also be a good idea to create your own list of words. For example, if you work for a company in the financial industry, it would be a good idea to make a list of words and slang specific to that industry. If you will be dealing with non-english speaking users, it would be a good idea to find some dictionaries in other languages.

Step 3: Create the word indexes that get used by CrackLib.
Now that we have our dictionaries, we need to compile them into an index that CrackLib uses. So, put all your dictionaries in one directory (/usr/share/dict/ is a common place). Depending on your distribution, there will be different commands to run. On gentoo (or if you installed from source), the following (as root) should do the trick:

create-cracklib-dict /usr/share/dict/*

And on RedHat:

mkdict /usr/share/dict/* | packer /usr/lib/cracklib_dict

These commands should create the following files:

/usr/lib/cracklib_dict.hwm
/usr/lib/cracklib_dict.pwd
/usr/lib/cracklib_dict.pwi

Step 4: Install python-crack
If your distribution’s package manager doesn’t include python-crack, then you will have to download and install it yourself. Neither Gentoo nor RedHat include this package, so I will walk you through the install. Download and unpack python-crack. Now, cd into the directory and run

./configure

By default, configure will setup python-crack’s files to be installed to /usr/local. This means that you will have to add /usr/local/lib/python2.4/site-packages to your PYTHONPATH. Alternatively, you could add a --prefix=/usr to the end of the configure command to install python-crack’s files to the usual /usr/lib/python2.4/site-packages directory.

If you get this error:

checking for prefix of the default cracklib dictionary database… unknown
configure: error: crack.h does not define CRACKLIB_DICTPATH. Please use DEFAULT_DICTPATH, see ./configure –help

then set the DEFAULT_DICTPATH variable when running configure, like so

./configure DEFAULT_DICTPATH=/usr/lib/cracklib_dict

Now run

make
make install

Now that we are finished installing, lets test out python-crack

$ python
Python 2.4.3 (#1, Jun 28 2006, 23:41:37)
[GCC 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import crack
>>> crack.VeryFascistCheck(‘foo’)
[snip]
ValueError: it is WAY too short
>>> crack.VeryFascistCheck(‘foobar’)
[snip]
ValueError: it is based on a dictionary word
>>> crack.VeryFascistCheck(’3#hsad2U>u2u’)
’3#hsad2U>u2u’

VeryFascistCheck() raises a value error and prints an explaination if the password is not strong enough, otherwise it echos back the password.

Creating the Django Manipulator

from django import forms

class NewUserForm(forms.Manipulator):
    def __init__(self):
        self.fields = [
            forms.TextField(field_name="username", length=20, maxlength=20, is_required=True),
            forms.PasswordField(field_name="password", length=20, maxlength=50, is_required=True,
                validator_list=[isStrongPassword]),
            forms.PasswordField(field_name="confirm_password", length=20, maxlength=50, is_required=True,
                validator_list=[forms.validators.RequiredIfOtherFieldGiven("password", "You must confirm password"),
                                forms.validators.AlwaysMatchesOtherField("password", "Passwords did not match"),]),
        ]

def isStrongPassword(field_data, all_data):
    """Test the password with cracklib to make sure it is strong."""

    import crack
    # Increase the number of credits required from the default of 8 if you want.
    #crack.min_length = 11
    try:
        crack.VeryFascistCheck(field_data)
    except ValueError, message:
        raise forms.validators.ValidationError, "Password %s." % str(message)[3:]

Notice that I added isStrongPassword to the password field’s validator_list. Now you are ready to put the NewUserForm manipulator to use in your you views.py. After running the manipulator’s get_validation_errors(), bad passwords will generate errors like “Password is based on a dictionary word” or “Password does not contain enough DIFFERENT characters.” For help on using manipulators in your view, see the Forms, fields, and manipulators documentation.

If you want to make Django’s built-in authentication require stronger passwords, then you could add validator_list=[isStrongPassword] to the password field of django.contrib.auth.models.User.

Now, go forth and require strong passwords.