Application Performance

Submitted by eric on Tue, 08/18/2009 - 21:22

There have been quite a few questions and forum posts related to DoggCatcher performance, so I thought I would take a moment to summarize some of the issues and recent developments.

The goal is to provide a good experience when using DoggCatcher and at the same time not detract from the experience of other mobile applications.

One of the things that has been done to improve the DC experience is to ensure that DC only uses resources that it needs. The feed parsing and UI display have been optimized many times since DC's first release in November 08 and I feel that in this area, we are getting the performance that we are looking for.

The other significant area related to performance is the application's lifecycle. DoggCatcher is the type of application that can benefit from running in the background when doing things such as feed updates, media downloads, and playing audio.

Up until now, it has been configured to always try to stay in memory. Android provides some mechanisms for applications to set a priority that get evaluated when resources are low. Android can then evict the applications with the lowest priority. Not surprisingly, DC is not the only application that is taking advantage of this high priority. When multiple applications use the same technique, and resources get low, the phone can become sluggish because multiple apps are resisting eviction at the same time.

I think the best thing to do now is to be a better citizen and change DC such that it only retains the high priority when it is actively doing something important (updating feeds, downloading, and playing audio). When it's not busy, we'll let Android evict it from memory. The drawback is that it will take a few seconds to start DC back up after it's been evicted. It will still update feeds at the configured interval by restarting itself.

The beta testers and I have been using DC in this configuration for a few weeks now and I think the change is the right way to go. You can enable this in the preferences starting with version 825, it is called keep-alive. The old behavior is the default because I try to always reproduce existing behavior and provide configuration to enable new behavior. Probably down the road, I'll change it so new installations use the new relaxed keep-alive.

Update: as of version 868, keep-alive for new installs defaults to 'while in foreground or busy in background'

I encourage you to try it, and let me know how it's going or if there's any way I can improve it.

Comments

DoggCatcher service running all the time

Submitted by jdc on Thu, 12/16/2010 - 11:25.

I'm using 1.1.1740 on an HTC Dream running CM5.0.8 (Android 2.1). RAM is a serious constraint on these phones, so I like to avoid unnecessary background processes. I have DoggCatcher set up to update feeds once a day at 8 am, and have Keep Alive set to "While in foreground or busy in background". Despite this, DoggCatcher always seems to be running when I check the running services screen. Here's part of the output of procrank:

PID Vss Rss Pss Uss cmdline
1675 22684K 22684K 10592K 7992K com.snoggdoggler.android.applications.doggcatcher.v1_0

So it's using more than 8% of my RAM. When memory gets tight, Android seems to kill other apps, such as k9 mail which uses IMAP IDLE for push mail. It would be much better for me if DoggCatcher was killed instead, and I thought that's what the Keep Alive setting would accomplish.

Any thoughts?

(And can the RAM usage be tightened up at all?)

Thanks for a wonderful app!

Service

Submitted by eric on Thu, 12/16/2010 - 21:29.

Android decides which services to kill based on a variety of things (which change with each android release). The only time DC should remain in memory when memory is tight, is when it's updating or downloading items. This is intentional.

Sometimes what happens is that DC starts up on the alarm for a feed update, updates feed, downloads some stuff, and then goes idle. Android will kill it after a while an then another doggcatcher service will start back up (at the request of the android framework). This new service doesn't have any dc code running and will just hang around. That's probably what you are seeing.

I also would not trust those numbers. I've seen numerous lengthy posts from android engineers explaining a lot of complicatedness about calculating the memory used by an app. The only way I've been able to do it is dumping the heap...and the heap size rarely matches what the OS reports. I think this mainly has to do with how android attempts to share some resources between processes to minimize memory usage, so it's hard to calculate the shared aspects.

The app itself is just a few megs, all the rest is the feed content, so it will vary depending on how many feeds/items you've got set up.

Service

Submitted by jdc on Thu, 12/16/2010 - 22:01.

Thanks for the quick response.

For some reason, Android kills other services quite frequently, but almost never kills DoggCatcher. For example, when memory is very tight, k9 gets killed and restarts over and over again, and in more normal situations it still gets killed every 30 to 60 minutes, but DoggCatcher usually runs all day without being killed, even when memory is tight. (The Running Services screen shows how long it has been since each service was restarted.) It's not just k9; other services also get killed regularly.

So it seems there is something different about DoggCatcher.

Is there a way to hint to Android that it's ok to kill DoggCatcher? I've heard about some kind of priority setting that an application can use.

Keep-alive

Submitted by eric on Fri, 12/17/2010 - 23:21.

As far as I know, you can't hint to android which processes to kill but you can hint which ones not to kill. That's what the keep-alive preference does. DoggCatcher also does this when it's updating feeds or downloading media. Any time an app is doing this, you'll see some kind of notification in the notification pane...that was required as of android 1.6 or 2.0 (can't remember which).

As long as you don't see a notification, then android should kill dc when it needs memory.

Keep-alive

Submitted by jdc on Mon, 12/20/2010 - 14:16.

Thanks for the explanation. But, like I said, it doesn't seem to be working for me, as DoggCatcher almost never gets killed even when I am very low on memory. And on the rare occasions when it gets killed, it gets restarted right away, which is no help. I'm willing to help debug this, e.g. with logcat output or whatever else would be useful.

Or, an alternative would be changing the Keep Alive setting to have three options: the current two as well as an option that would make DoggCatcher completely exit when idle (i.e. not register a service at all). Instead, it could set up a timer to get restarted when needed. (I have only a vague understanding of how Android handles these things, but I know there are other apps that do periodic things but show no ongoing service running.)

service death

Submitted by eric on Mon, 12/20/2010 - 23:28.

I understand what you're saying. I see lots of services laying around on my phone and it give me an uneasy feeling. I wish I had some control over making them go away.

The problem is that it's android that's stopping and starting them. Apps are not supposed to exit themselves, even though some do, including doggcatcher. The android engineers expect android to manage the removal of apps when resources are needed for other apps. So if you use a task killer to kill a service like doggcatcher, and you see it start back up, that's android starting it back up. Occasionally you'll even see android kill a service and then start it back up. That's part of the cycle it goes through to minimize memory usage. Sometimes you'll also see doggcatcher start back up when you exit it from the menu...that's also android starting it back up.

Having a doggcatcher service is required for some features, such as keeping the app from getting evicted when it's updating feeds and downloading media, so it isn't going to be possible to remove the service. The other apps you are seeing may not be using services and android may be handling them differently, such as not restarting them when they died. I think it may be the default behavior for android to restart a service when it dies.

I wish I had a better answer for you but I don't think I can do much here. It's pretty difficult to go against the android design (and probably for good reasons that I don't understand)...it's relentless and it will win :-)

Thanks for the complete

Submitted by cboluda on Wed, 08/19/2009 - 15:25.

Thanks for the complete explanation. I also think this is the way to go.
This version 825 is a huge step ahead.
The phone feels more responsive now.

Improved start-up

Submitted by jc on Wed, 08/19/2009 - 11:40.

I support this idea of DC behaving generously and allowing it to be dumped from memory when the user is not actively using it and is trying to launch other programs.

I just wanted to add that the next step is to improve the time it takes for DC to start so users can get back into DC quickly. Can you share with us why it takes so long and what is happening behind the "Starting Doggcatcher, please wait"?

Update on improved startup

Submitted by eric on Sun, 08/30/2009 - 23:17.

Some of the time that I trimmed during startup had to go back in for the new feature of relative times on feed/item updated dates.

However I was able to remove an http request that checks for app updates, which should make an improvement when you startup while on a cell network.

I improved this

Submitted by eric on Fri, 08/21/2009 - 23:37.

I improved this a bit. It was taking between 6-7 seconds before and it's down to 2 1/2 now.

Thanks for the idea.

This will be in the next release.

Great point

Submitted by eric on Wed, 08/19/2009 - 20:54.

I'll definitely take a look at that. I'm pretty sure there are some things I can trim. There are also some thing that I need to do, such as opening the database, loading configuration settings, and some other initialization stuff.

I created an issue for this.

http://mantis.snoggdoggler.com/view.php?id=198