First, we wrote a trivial main.c to drive our program:

#include 

// Promise C that we will link against externally defined functions (f and g)
// with these signatures.
extern int f(int i);
extern int g(int i);

// Simple main.
int main(int args, char** argv)
{
  int i = 1;

  printf("i=%d\n", i);
  printf("f=%d\n", f(i));
  printf("g=%d\n", g(i));

  return 0;
}

If you hadn’t guessed, we’re now going to define int f(int) and int g(int). Both of these functions take an integer, increment it, and return it. First, here is int g(int) as generated by an unoptimized gcc call we put in the file g.S:

.global g

g:
push   %rbp             # Push the current base pointer on to the stack.
mov    %rsp, %rbp       # Store the stack pointer value into the base pointer.
mov    %edi,-0x4(%rbp)  # Put argument 1 into the local variable.
mov    -0x4(%rbp),%eax  # Put the local variable into the output reg
add    $0x1,%eax        # Increment the value in eax by 1.
pop    %rbp             # Restore the base pointer of the prev frame.
retq

Looks a little verbose. Here is our hand-optimized version called int f(int) that we put in a file named f.S:

.global f

f:
  mov    %edi,%eax        # Put input value into output register.
  add    $0x1,%eax        # Increment the value in eax by 1.
  retq

They do (roughly) the same thing. Add 1 to the input and return the input.

So, what can we learn from this? Well, notice how we pluck the input from the %edi register and place it in the %eax register? Those calling conventions might differ if you are in another language, like, say, C++. This is why we use extern "C" { #include }. Remember that the compiler generates the machine code from the header, not the linked machine code.

It’s also instructive to notice that the unoptimized gcc call pushes %rbp (the base pointer) to the stack and then points %rbp at its old value. It then accesses a local variable 4 bytes in size above the current top of the stack (it’s -0×4 because execution stacks grow from the top-down).

If we had used a local variable and made another function call the stack pointer (%rsp) would have been decremented so that then ext call to push %rbp wouldn’t clobber our existing data. We would also see a leaveq which quickly pops all of our data off the stack by taking the old %rbp stored in the current %rbp and restoring the old value.

Nothing too earth-shattering, but revisiting the metal-level helps keep perspective and helps me keep one eye on what the stack machine is doing under the higher level language I’m usually using, such as C++.

       --ignore-rpath LIST
              Ignore  RPATH  and  RUNPATH information in object names in LIST.
              This option has been supported by glibc2  for  about  one  hour.
              Then it was renamed into:

       --inhibit-rpath LIST

“Supported for about one hour.” Love it.

One of the major problems I have had is the largely excellent activerecord-oracle_enhanced-adapter doesn’t do everything quite right yet. For example, they don’t provide the code to do structure dumps. Ok, no biggie. They don’t support RAWs yet (I’m lobbying for a pull request to mostly support RAWs) but even if RAWs were supported there is some oddness that when my primary keys are not the default primary key type, they aren’t dumped correctly to the magic Active Record Migration language.

So, what to do? Well, we decided to override the Rake::Task db:test:prepare and after db:test:purge‘ing the database we’ll just re-run the migrations on the test database. Optimal? Not entirely, but you can’t beat the flexibility!

So, how do we do this?

Remove the Old Task / Redefine a New One

So, you can’t just re-define a task. A Task object contains actions. You must clear those actions and then define the task, which appends to the now-empty-task your defined action.

namespace :db do
  namespace :test do |s|
    s[:prepare].clear

    ...

Now define the task like you would normally…

task :prepare do
...
end

The rest is, in some ways details. Invoke the other Rails tasks you want. Be aware that the rake test call switches environments from development to test during it’s normal runtime. When you override the preparation you need to manage that. In our case we want to call Rake::Task[:db:migrate] in the test environment and then do our unit tests.

Also of note, SEQUENCE objects aren’t automagically deleted by the db:purge task, so if you use that method to purge your test database you’ll also want to delete your SEQUENCE objects. You can look those up in the user_objects table.

I’ve been trying to optimize building summary information about about 12 million very-very-very tiny records. This is still technology under development, so please forgive my vagueness. The slowness of the query comes from a few places.

Problems

First, sheer record number. Twelve million of anything amplifies your decisions, and these records are very small. It all fits in memory and it’s nothing but CPU churn.

Second, there is a single join against a label table which, I partly blame Postgres for the slowness of. The labels are assigned by the closest prefix less than the given value. It’s not a clear join and as a result Postgres always scans the table’s index. I assume that Oracle would be able to put some snazzy prefix searching or Trie-type structure in place, but that has yet to be seen.

So, those are the two big problems. I am very proud, though, of what I’m going to term “gorilla optimizations” that our team came up with to shrink the problem down to a size where It’s clear we can continue to fly on Postgres for a while longer.

Solutions!

First, we decided to pull the join lookups out into an external library which was provided by the vendor. This alternate lookup method is faster even when having our application do all the lookups in Ruby code.

Second, we decided to summarize the labeling. Since the reports we are looking at generating are about maximums and upper-bounds it was acceptable to reduce precision on our label lookup by reducing the range of our foreign key to the label. This was done by simply truncating foreign keys and grouping them. The result is that almost all of them still receive the same value, but a few do not. This also resulted in far fewer calls to the label library.

Third, we summarized by a time slice. The database could give us hours very cleanly ( date_trunk('hours', ... ) ) but in situations where we wanted 1/2 days we had the application code examine the raw rows given back by Postgres and aggregate them into the size block desired before storing.

The results of all these optimizations sums up to something we can probably live with for quite a while, and why I share this is that no single optimization could have made this work. The constant picking away at bits of bloat and lag by many minds has resulted in not having to do any sort of major redesign or custom system development, and that’s a nice story worth sharing.

First, add the handler methods to the base ApplicationController as in the last post. Notice that
they are different! They point to a controller named error.

class ApplicationController  :error, :action => :server_error
  end

  def method_missing(id, *args)
    redirect_to :controller => :error, :action => :not_found
  end
end

Now, what I glossed over in the previous post, this error controller will have two actions (and views which
are not shown here). Notice that we also have defined rescue_with_handler in ErrorController
as well as in ApplicationController. In the rare case that we cannot render the error pages, for
example the navigation bar uses a database table and the error is that the database is down, we need a way to
fail back to the static /500.html page. Otherwise we will recursively exhaust the call stack in the
Ruby VM.

class ErrorController < ApplicationController
  def not_found
  end

  def server_error
  end

  def rescue_with_handler(exception)
    redirect_to '/500.html'
  end
end

Finally, add a line in your config/routes.rb file to send anything not already captured to the
ErrorController‘s action note_found.

match ':action', :controller => :error, :action => :not_found

I choose to pepper this sort of thing with logging statments at least until I’m sure it’s working correctly. Enjoy!

Given the extremely constrained time I’m living (happily, I might add) in I offer up to you this solution and encourage feedback!

class ApplicationController < ActionController::Base

  def rescue_with_handler(exception)
    redirect_to '/500.html'
  end

  def method_missing(id, *args)
    redirect_to '/404.html'
  end
end

That’s the best I could dig up given my admittedly limited experience. Better solutions are welcome! It seems the Rails 2 redirect to 404.html inside the public directory isn’t in play anymore. Thanks in advance! Hope this helps someone else Googling like I did!

So above we see two overrides. First, the obvious method_missing override sends any action this ActionController does not know about to the static 404.html page that Rails ships with. Replace this with a 404 controller if you like and get a nicer error page and override that controller’s method_missing method and you have a nice robust chain to handle basic failure and really unexpected failure.

The second method we override is the rescue_with_handler method. When an exception kills an action, this is what seems to handle the drama. Naturally an exception is typically a 500 error. Naturally you can be a little more precise about your error handling and do the nice chaining I mentioned for the 404 page.

Finally, if you haven’t thought of this yourself, make a default route at the end of your config/routes.rb file to ensure that your error controller gets any request that isn’t matched otherwise.

This is what frustrates me about the “NoSQL” movement, and my apolgies to Grant Ingersoll for being the prompt for this post, but as a corporate blogger, I don’t feel so bad picking on this particular post: The movement is packed with non-critical posts designed to gather attention and not promote understanding.

More secifically, when I clicked on the afore mentioned blog post I expected a discussion of ways to estimate inverse document frequency or perhaps an argument that if you shard your search cores you can ignore idf (inverse document frequency) because it is statistically valid for scoring documents.

I would have been happy for a post that just mentioned the word sharding and talked about putting different search cores in different cores on the same servlet container on the same machine vs. distributing those search cores to other machines.
I would have loved to read about memory contention and the idea of floating garbage. I would have even enjoyed a small piece that said that true horizontile scaling is hard because of global values such as idf.

I got none of that. I didn’t even get a link to a technical piece talking about any of the buzz words mentioned.
I know, I know, my fault for clicking. The reader beware. That being the case, I don’t feel so bad that you may have taken 5 minutes to read this. And Grant got a little publicity for Lucid Imagination.

Quickly, go to your VPN URL and get the client software installed. it will probably fail. It does that. Then, we will roughly follow the steps at the blog site cited above plus a few bonuses:

1. Install it in /opt/cisco/vpn (yeah, we did this already)
2. Download Firefox’s 32bit binary and install it at /usr/local/firefox.
3. sudo apt-get install ia32-libs lib32nss-mdns
4. Copy /usr/local/firefox/*so to /opt/cisco/vpn/lib
5. Create the file /etc/ld.so.conf.d/lib32firefox.conf and add the line to it /usr/local/firefox.
6. Run ldconfig to process the config file /etc/ld.so.conf.d/lib32firefox.conf and add those shared libraries to the system.
7. Remove ~/.mozilla from the user that you want to be able to start the VPN.
8. Run firefox and point it at your vpn’s URL so it “sees” the certificate. Once you do that it appears that you can start /opt/cisco/vpn/bin/vpnui and connect without issue.

A few hoops, but well worth it if you can stay in your preferred development environment and connect into work. May I recommend, while I’m at it, to create a user-work account on your desktop to keep a little separation between home and work life? No? Well, I can’t bring myself to do it either.

TCUP (Trivial Callup Protocol) was to be a simple messaging protocol with the capability of establishing calls through nodes in an overlay network. Sadly, life got busy and I never had the time to make the implementation solid. You’ll also probably notice that the PDUs in the protocol document nowhere specify an upper layer protocol. Lesson learned, always include that field. It is a pain coding multiple upper layer protocols and trying to dispatch the traffic to the correct handler.

Still, I love the idea of overlay networks and small independent rings of storage and computation over the internet. Neat ideas coming back from the past.

We recently bumped into a very subtle null pointer exception patterned after this snipped:

public class A
{
    public static void main(String[] argv)
    {
        final Float f = argv.length == 10 ? 0f : getF();
    }

    public static Float getF() { return null ; }
}

No, that’s not the code, verbatim, but the pattern is the same. There was a ternary operator, as above, that returned a float primitive (the 0f) or a Float object from the method call. The output of the ternary operation down-casts both calls to a float primitive, thus if getF returns null as it does in our example, the statement blows up. If you wrap the 0f in a Float.valueOf(0f) then no downcast is performed and you are fine.

Fascinating, no? What I love about this example is how subtle and “correct” the statement looks.