Tom Finnigan's Tech blog

Very Sleepy Profiler

2009-06-26T10:13:00.000-07:00

So, lately at work I've been working on the performance of our product. One of the first steps to improving speed is to run a profiler.

I've been completely spoiled by shark, the free profiler available on OSX. It's a sampling profiler, but it also does stack sampling, and works great. You just attach to any running program (or launch a program) and it runs at roughly the original speed, but gives you lots of great information.

I was looking for a shark-like profiler for windows and I tried lots - VTune, Intel's PTU, Windows Performance Toolkit, GlowCode, and a few others. Eventually I ended up with a profiler called Very Sleepy.

Very Sleepy has several advantages. I don't want an instrumented profiler, because with a UI-heavy app, instrumentation makes profiling unfeasibly slow. You either have to program a UI-using robot, or be very very patient in using your UI. Since we're a plugin, the UI robot would be much more complicated than just using a sampling profiler.

Next, it provides exactly the information I want to know - what's the hotspot? How is it being called? What does it call? For each function, how much does it represent, by itself and with children?

Finally, it's open-source, so it's free, and I can modify it to meet my requirements. I was having lots of problems getting it to properly load the symbols from our symbol server and over the internet, but because I had the source, it was simple to dump debugging information. There are a few other small customizations that I've been able to make to help our usability (i.e. only look at processes with an image starting with the letter R) that are only really beneficial to me and my coworkers.

The main hiccup I found after the debugging is that the symbol server that I was using had compressed symbols, and I needed to download and install a new dbghelp.dll and srmsrv.dll from Microsoft. I still haven't been able to get it to download symbols to my local store from Microsoft and McNeel's public symbol servers in the profiler itself, but that's not a huge deal, as I can get VS2005 to download them for me, and very sleepy reads them just fine out of the symbol store.

"I'd just google it" as an interview answer

2009-06-11T05:25:00.000-07:00

Recently read an interesting post about whether "I'd just google it" is an acceptible interview answer.

I agree that it's usually not a good idea as a final answer. However, if the question is something that I would just google, I'd probably say something like: "If I actually had this problem, I'd probably research it on the internet. However, assuming that I couldn't find anything, here's what I'd do...". This way you show that you're practical, but you also answer the question as asked. I've worked with people that were great at solving problems by themselves, but horrible at deciding when to use a library vs. when to do it themselves.

I gave an interview for a system admin at the graphics lab once. For the technical part of the interview, I asked him what he would do if there was a server running redhat linux, which he had physical access to, but nobody remembered the root password. He said he didn't know. I pointed him to a nearby machine said that the question was open-internet, and left him to search.

After a few minutes he came back and showed me that there were a few different approaches, explained their pros and cons, and in which order he'd try them.

He made a great admin.

Keyboard use is overrated

2008-11-18T08:43:00.000-08:00

I was just reading a post on codinghorror about how good programmers should necessarily be good typists.

I feel like maybe I should be submitting this to the daily WTF, but I knew a programmer who had a keyboard tray, and he kept his keyboard stowed. While programming.

He was a VB programmer, and I guess it's somehow possible to program in VB using only your mouse. I sat near him, but I never asked him how exactly he was able to do it.

My guess is that he first laid out his UI, which auto-assigned names. Double-clicking on elements allows event handlers to be created. He also spent a lot of time in really long drop-down lists, so I'm guessing there was a list of functions, and a list of variables, and he was just filling in the blanks with selections from his list.

Every once in a while he'd get stuck and have to pull out his keyboard to type something, but he was always annoyed about it.

Build order

2008-07-19T21:03:00.000-07:00

So, I do a lot of compiling at work. The project is large enough that I think twice before modifying header files that would cause most of the project to rebuild, such as defs.h or point.h.

One problem I have is that when I'm modifying such a file, compiling the project (F7) results in a complete recompile, to bring the project up to sync with the modified header. This is a significant amount of effort, but necessary.

However, sometimes I've made a mistake when making changes. Sometimes I need to edit the header file again. For example, maybe I've forgotten to make functions const that need to be const. After finding the error, I need to fix it and re-start the compilation process.

So, that's the problem. Here's a proposed fix: rather than compiling all the files in a given project alphabetically, compile them from most recently edited to least recently edited. There are a couple of places where an error would be detected, but it's more likely going to show up in a file I was recently editing, rather than one that was last touched 4 years ago, but starts with the letter A.

I'm not even sure if this is possible to set up with the current tools, but why isn't this the default? Is there a downside I'm not seeing?

Vista's UAC at work

2008-05-22T12:14:00.000-07:00

So, most of my day-to-day programming is a plugin to a windows program.

There are various build configurations that produce output, and as a post-build step, we run a silent installer that makes sure that the output of the current configuration is the one installed in the host app.

This worked great until Vista. With UAC, every time I finish a build, I have to click Allow for the installer. This does not help my workflow.

Perhaps the simplest solution is 'blessing' the installer executable with admin rights, so that it knows I trust it, similar to the SUID bit on unix/osx. Apparently SUID was considered too dangerous for Vista, which I understand, given the amount of legacy software that would be insecurely ported in this way.

The other solutions I can think of involve disabling UAC or installing XP, which sort of defeats the purpose.

Anyone have any better ideas?

iPhone background processing

2008-03-18T12:53:00.000-07:00

After reading Craig Hockenberry's post about background processes and notifications on the iPhone, I realized that there's one other option that's a possibility for dealing with background processes.

Basically, you have an application like growl that centralizes notifications, and other things like network usage, etc. Applications can register callbacks to the notification system, so that their background processes start when you run the notification app, when something is using the network they get notified, etc.

I'd also build in a process manager. And solitaire. And a web browser. Oh not.. :)

This means that the user can decide when they want notification, how often they want the network to be used, etc.

This would add complication and control, and is likely overkill for most users, but I can see how such a framework would be handy for a user with multiple applications that need to be running at the same time. There's really no other way for multiple applications to run at the same time besides using a jailbroken iPhone.

Whether the security model allows for it is another issue.

iPhone background apps, and laptop sleep

2008-03-15T20:04:00.000-07:00

So, there has been some discussion on daring fireball about the iPhone and running background apps.

There's not much reason to run background apps on the iPhone - it is essentially a single-tasking device. There are some applications on the iPhone that run background apps, such as iTunes and file transfer. However, you can still do most things, even tricky things like IM, completely as a foreground app.

The trick is to use an intermediate server to buffer communications. When the user is logged into an IM, the server logs in. When the user switches to a different app, the server remains logged in, maybe switches status to 'away', and queues the messages. When the user switches back to the app, it knows it was logged in, and downloads any queued messages.

What would this prevent? Well, it would prevent the app from popping up when an incoming message is received, which might be desirable anyway. It would also prevent apps that need to continually send out data without being in the foreground. Finally, apps that require a lot of computation would need the user to stay in the app while it computes, so no encoding video in the background on your iPhone. While it's arguable that these applications should be possible, most of what people want to do can be accomplished by using an intermediate buffering server, or forcing the user to keep the app in the foreground.

What are advantages of no background apps? Well, if the iPhone has a security flaw, or even in the case of trojan programs, the user would be unaware if the iPhone became a zombie, and the user has none of the tools necessary to even notice such behavior, nevermind track it down. There is no process viewer, and it goes against the single-tasking design of the device.

Another issue that people have run into is the iPhone data problems while roaming. Right now there is only one app that downloads data in the background, mail. People have had problems in the past with mail massively running up their roaming data bill.

The potential issues with battery life have already been discussed.

I think Apple's stance on this is similar to their stance on sleep. I've seen lots of people walking around with windows-based laptops open, going between classes or meetings. All applications on windows are notified that windows is going to sleep soon. Any application can cancel or delay the sleep request, and enough do that the feature is basically worthless on most PCs.

Sure, there are certain applications that would legitimately fail if the machine went to sleep, and might have good reason to cancel sleep requests. There are hacks on OSX that allow this kind of behavior, but it took a while for them to become available. In the meantime, most apps were written to behave well under the default configuration.

The end result is that when I close my mac, it goes to sleep. When I open it back up, it comes back up. It's fast, and it always works. The common case works well, and the uncommon case works eventually.

Distributed tinyurl

2007-12-10T12:17:00.000-08:00

TinyURL seems to suffer from a few problems:

Single point of failure
Not human readable
Not reversible

How could you combat these problems?

The goal is to have a large URL replaced by a small URL. Simple compression or hashing of the URL isn't going to work. You need to iterate through possible values, or the output will be unusually large.

I normally come to solutions iteratively, and this one still feels half-baked. But it's at least half-delicious.

Standardize on a shortened URL scheme. Each shortened URL is a small hash of the actual URL, plus a unique identifier. Entries are propagated through a system of distributed hosts, and lookups can be achieved by querying the closest server.

Standardization allows you to integrate functionality into webbrowsers and similar tools to lookup the URLs before clicking, and display the actual URL right where it's normally displayed.

Distributed hosts allow you to avoid the single point of failure.

And using the hash of the URL plus a unique identifier allows you to have confidence that your local server is returning a valid URL, and the correct URL. If you use a hash that's cryptographically strong, you get cryptographic guarantees against tampering with the database.

I'm not sure of any cryptographic-strength hashes that would work well on input data as small as URLs, or produce small enough output, so that's a potential problem. Also, I imagine the sysadmins of the world wouldn't be keen on supporting another big distributed system, especially for such a trivial issue.

How much for a sanctioned custom iPhone ringtone?

2007-11-14T09:24:00.000-08:00

After reading Daring Fireball's frequent posts on Apple's ringtone policies, I was curious about how much it would cost to get a custom ringtone on the phone.

While there are various software hacks and workarounds to get ringtones on the phone, the only Apple-sanctioned method is to buy a song from the iTunes music store and convert it to a ringtone with their custom tool. Because it's sanctioned by Apple, it's the only kind of ringtone you can be sure won't suddenly disappear after a software update.

Unfortunately, there aren't too many clips of my daughter saying "ring, ring, ring" on the iTMS. As you can imagine, I'm pretty attached to my ringtone. If I ever scrape together enough money to buy an iPhone, it would have my ringtone on it, one way or another.

For the Apple-sanctioned approach, the main trick is getting my track onto the iTunes store. After some googling, I found that cdbaby will distribute tracks on the iTMS.

The charges break down as follows:

$35 cdbaby cd fee
$20 cd barcode
$ 1 Buy on iTunes
$ 1 Convert to Ringtone

$57 Total

This is not completely accurate, because you'll also need to burn a CD and mail it to cdbaby, and you'll get some sort of payment back from cdbaby after you buy your own track.

You could also reduce the costs by getting a bunch of people to go in on the same CD.

However, it's pretty obvious that this isn't much of an alternative to the current hacks and workarounds to achieve the same thing for almost free.

Making Wrong Code Not Compile Either

2007-09-30T13:36:00.000-07:00

Every once in a while I feel an impulse to write a longer response to something I read than the comments section of a blog will allow.

Hopefully this blog will help me follow through on those impulses.

In that spirit, Joel Spolsky wrote an interesting defense of true Hungarian Notation.

In general, I agree with the thesis: used properly, Hungarian Notation can make bad code look bad, and increase code quality in general.

Hungarian in this case is simply used to encode meaning into the variable name. That's great - the variable name is a wonderful place to put descriptive information about what the variable is and how it's used.

However, Joel's suggested problem and solution are sub-optimal, and he paints c++'s type system as a disaster compared to c. While c++ can be used badly, the more flexible type system can be used to combat the same problems Joel is using Hungarian to solve, and more robustly. Not that Hungarian is bad - what I'm saying is that there are a couple of different levels at which you can encourage proper usage, and the strongest is through proper typing.

With c++ it's pretty easy to make bad code not only look bad, but not compile at all. As a proof of this, I've written up Joel's safe/unsafe string example using c++. You'll notice that I did include prefixes to my variable names in main. You'll also notice that it would be very difficult for a programmer to use an unsafe string with a function that requires safe strings, and vice versa. If I wanted to make it even harder to misuse, I could use some data hiding.

The advantage of this strong checking is that a new programmer doesn't need to be trained to spot wrong code - the compiler will tell them when they do something bad.

Without further ado, Joel's string example in c++. This is the entirety of the file, designed for brevity. It correctly compiles, but it will not compile either of the incorrect commands.

 
#include <string>
#include <iostream>
using namespace std;

class StringHolder
{
public:
  std::string str;
};

class UnsafeString : public StringHolder {};
class SafeString : public StringHolder {};

UnsafeString request()
{
  UnsafeString retval;
  retval.str = string("This string is unsafe.");
  return retval;
}

SafeString encode( const UnsafeString& unsafe )
{
  string temp = unsafe.str;
  string find_me( "unsafe" ), replace_with( "safe" );

  while ( true )
  {
      std::string::size_type pos = temp.find( find_me );
      if ( pos != string::npos )
      {
          temp.replace( pos, find_me.size(), replace_with );
      }
      else
        {
          break;
      }
  }

  SafeString retval;
  retval.str = temp;
  return retval;
}

void write( const SafeString& safe )
{
  cout << safe.str << endl;
}

int main( int, char** )
{
  UnsafeString us_input = request();
  //write( us_input );  // Doesn't compile
    SafeString s_output = encode( us_input );
  //s_output = encode( s_output );  // Doesn't compile
    write( s_output );
}