I'd suggest trying the non-vectorized version on PyPy. This code seems very JIT-able, and CPython is nowhere near the state of the art for performance with this kind of code.
On my rMBP, I did a quick back of the envelope comparison using PyPy 2.2.1 vs CPython 2.7.6 on OS X Mavericks and came up with (best of 3 runs):
$ time pypy testcode.py
real 0m1.529s
user 0m1.402s
sys 0m0.121s
$ time python testcode.py
real 0m13.880s
user 0m13.635s
sys 0m0.241s
$ time python testcode_vectorized.py
real 0m0.746s
user 0m0.551s
sys 0m0.139s