Eric Moritz

Learning Python Logging

2012-02-17T17:46:00Z

The Python logging module is often a source of confusion with developers. Often logging is the final thought in a project. When we are finishing up a project, the last thing we want to do is sift through the logging documentation to figure logging out. It is a perpetual cycle that results in some very bad code.

I am going to assume that you have some familiarity with the Python logging module and the concept of logging levels, so I will only address concepts with the logging module that has been a source of confusion in colleagues.

How Python logging should be used in modules

The first source of confusion with the Python logging module is the question of how to create a logger. This is probably the easiest part of Python logging. Here's how you do it:

import logging
log = logging.getLogger(__name__)

That is basically it. That code will create a logger based on the module's fully qualified name. So if we have a module called mypackage.module1.module2, that will be the logger's name.

Keep in mind that __name__ is the string "__main__" if your Python file is ran as a script. Other than that, that is all you need to do.

You do not have to worry about logging levels, or handlers or anything in your modules. How your loggers are configured is an application level setting. Your modules have no reason to worry about how to configure loggers.

Logger names

An often overlooked fact about loggers is that logger names are hierarchical. That is why I choose to use the module name as the logger name. The hierarchy starts with the root logger and descends from there.

The root logger can be accessed by calling:

  rootLogger = logging.getLogger()

Configuring the root logger will cascade down to the other loggers:

  rootLogger.setLevel(logging.INFO)

The same is true if you access a parent of any other logger:

  appLogger = logging.getLogger("myapp")
  appLogger.setLevel(logging.INFO)

All myapp.* loggers will have the level INFO as their default.

Let us say that there are the following loggers used in our application:

redis.connection
mysql.connection
mysql.query
myapp.models.polls
myapp.models.questions
myapp.models.articles

If we only wanted connection errors from our databases, query debug information from MySQL, and warnings from myapp's models; we could configure it as so:

logging.getLogger().setLevel(logging.ERROR)
logging.getLogger("mysql.query").setLevel(logging.DEBUG)
logging.getLogger("myapp.models").setLevel(logging.WARNING)

That is it. This accomplishes what I described. We set the default level to ERROR which causes "redis.connection" and "mysql.connection" to only emit ERROR and EXCEPTION level messages. We set the "mysql.query" logger to DEBUG to emit query debug messages. Finally we set the "myapp.models" logger to WARNING.

Configuring Python loggers

I just showed you how to configure the level of a logger pragmatically. Where should that code exist? If you answered, "In the module", I want you to get up, find the nearest blunt object, and hit yourself with it. Do you not remember me saying that logging configuration is an application level configuration?

Here is a hypothetical script that uses a Redis connection.

import_presidents1.py:

import csv
from redis import StrictRedis

reader = csv.reader(open("./presidents.csv"))
header = reader.next()

client = StrictRedis()

for row in reader:
    key = "president:%s" % (row[0], )
    doc = dict(zip(header, row))

    client.set(key, doc)

Now, let us assume that the redis module used a "redis.connection" logger that logs a DEBUG level message whenever the redis client needs to reconnect to the server. Here is a mock Redis client for this example.

When we run the code as it is, no connection messages will be printed.

Now, let us say our connection is flaky and we have to reconnect to Redis often:

import_presidents2.py:

import csv
from redis import StrictRedis

reader = csv.reader(open("./presidents.csv"))
header = reader.next()

client = StrictRedis()

for i, row in enumerate(reader):
    key = "president:%s" % (row[0], )
    doc = dict(zip(header, row))

    # simulate a disconnect every 3 operations
    if i % 3 == 0:
        client.disconnect()

    client.set(key, doc)

We can configure the "redis.connection" logger to output the connection messages by configuring our script like so:

import_presidents3.py:

import logging
import csv
from redis import StrictRedis

logging.basicConfig()
logging.getLogger("redis.connection").setLevel(logging.DEBUG)

reader = csv.reader(open("./presidents.csv"))
header = reader.next()

client = StrictRedis()

for i, row in enumerate(reader):
    key = "president:%s" % (row[0], )
    doc = dict(zip(header, row))

    # simulate a disconnect every 3 operations
    if i % 3 == 0:
        client.disconnect()

    client.set(key, doc)

For simplicity, we could have replaced:

logging.basicConfig()
logging.getLogger("redis.connection").setLevel(logging.DEBUG)

with:

logging.basicConfig(level=logging.DEBUG)

logging.basicConfig() is a really handy function. I find myself using basicConfig() often because most of the time I simply want to output to stderr or to a single file rather than wanting complex routing using Handlers.

What are Handlers and Formatters?

Apart from Loggers, there are three classes that are used to configure an application's loggers: logging.Handler, logging.Formatter, and logging.Filter.

A Handler defines how a message is handled. For instance, there is a StreamHandler that logs messages to stderr.

There are 13 standard Handlers defined in the logging.handlers module that will send messages to files, syslog, HTTP servers, sockets, what have you.

A Formatter is exactly what it sounds like. It formats a LogRecord. That's pretty much it.

A Filter filters a LogRecord. I have never needed one, so I will let you define your own use case for one.

Let us change our presidential import script to log any exceptions that occur when importing:

import_presidents4.py:

import config
import logging
import csv
from redis import StrictRedis

# We do not want to use __name__ here because __name__ is "__main__"
log = logging.getLogger("presidents.importer")

try:
    reader = csv.reader(open("./presidents.csv"))
    header = reader.next()

    client = StrictRedis()

    for i, row in enumerate(reader):
        key = "president:%s" % (row[0], )
        doc = dict(zip(header, row))

        # simulate a disconnect every 3 operations
        if i % 3 == 0:
            client.disconnect()

        # simulate a failure
        if row[0] == "37":
            raise exception("crook.")

        client.set(key, doc)
except:
    log.exception("dang it.")

Notice that we added "import config", this will let us separate configuration from implementation. For our little one-off script, having a config.py module is overkill and only exists to prove a point.

Here's the contents of config.py:

import logging
from logging import FileHandler, StreamHandler

default_formatter = logging.Formatter(\
   "%(asctime)s:%(levelname)s:%(message)s")

console_handler = StreamHandler()
console_handler.setFormatter(default_formatter)

error_handler = FileHandler("error.log", "a")
error_handler.setLevel(logging.ERROR)
error_handler.setFormatter(default_formatter)

root = logging.getLogger()
root.addHandler(console_handler)
root.addHandler(error_handler)
root.setLevel(logging.DEBUG)

An alternative way to configure your application's logger is to use the logging.config.fileConfig to configure the loggers using an INI file.

Here is the equivalent INI file for the configuration we previously described using Python code:

logging.ini:

[formatters]
keys=default

[formatter_default]
format=%(asctime)s:%(levelname)s:%(message)s
class=logging.Formatter

[handlers]
keys=console, error_file

[handler_console]
class=logging.StreamHandler
formatter=default
args=tuple()

[handler_error_file]
class=logging.FileHandler
level=ERROR
formatter=default
args=("error.log", "w")

[loggers]
keys=root

[logger_root]
level=DEBUG
formatter=default
handlers=console,error_file

To use this file, let us modify our config.py.

config_ini.py:

import logging.config
import os.path
LOGGING_CONF=os.path.join(os.path.dirname(__file__),
                          "logging.ini")

logging.config.fileConfig(LOGGING_CONF)

Finally, here's the updated import script:

import_presidents5.py:

import config_ini
import logging
import csv
from redis import StrictRedis

# We do not want to use __name__ here because __name__ is "__main__"
log = logging.getLogger("presidents.importer")

try:
    reader = csv.reader(open("./presidents.csv"))
    header = reader.next()

    client = StrictRedis()

    for i, row in enumerate(reader):
        key = "president:%s" % (row[0], )
        doc = dict(zip(header, row))

        # simulate a disconnect every 3 operations
        if i % 3 == 0:
            client.disconnect()

        # simulate a failure
        if row[0] == "37":
            raise Exception("Crook.")

        client.set(key, doc)
except:
    log.exception("Dang it.")

For some people INI files leave a bad taste is their mouth. If you do not like the taste of INI files, you can also configure the loggers using a dictionary or with Python code. It is up to you.

There we have it. Python 2's built-in logging module is one of those warts of the standard library that goes against the "Zen of Python" but once you learn the non-obvious way the designers intended to be used it becomes simple.

I hope this helps clear up Python's logging module for you.

Links

Books every self-taught computer scientist should read

2012-02-15T11:16:00Z

There are three books that I believe that every self-taught computer scientist should read. These are books that I often recommend to smart junior developers that learned their knowledge by doing instead of schooling. These are books that will close the gap for people that fall into the category of "or equivalent experience" on a job posting.

K&R

The first book is the venerable C Programming Language book. This book was written by Brian Kernighan (an original Unix hacker) and Dennis Ritchie (the creator of C) and because of that, it is affectionately known as the K&R book.

The K&R book is invaluable for self-taught computer scientists for a number of reasons. Firstly, C is arguably the Latin of modern programming languages. C++, ObjectiveC, Java, Perl, Ruby and Python all have their roots in C.

Second, the K&R book is written by the designer of the language. I read it early in my career and found it very easy to read despite being a manual for a programming language.

This book provides a comprehensive look at all aspects of the C language; from basic statement structure to pointers and elementary data structures. The topics discussed in this book will provide you with a good foundation of programming that is often missed when you learn to program to get the job done.

Mastering Algorithms in C

Where the K&R book leaves off, the Mastering Algorithms in C book picks up. In this book you will learn the about Big-O notation, analysis of algorithm, recursion and fundamental data structures.

The beauty of this book is that it demonstrates these topics a practical way that appeals to the self-taught computer scientist. Instead of discussing these topics in only abstract terms, the author walks you through the implementation of algorithms and data structures in C.

With this book you will be introduced to five fundamental data structures in computer science and their application: Linked Lists, Sets, Hash Tables, Trees, and Graphs.

Once you have a solid grasp of these structures you will be able to model solutions in those terms without having to reinvent the wheel. Do you need to model the relationships between people? That's a graph. Do you need to model a hierarchy of sections on a website? That's a tree.

The Little Schemer

After you have a grasp of basic data structures, it is time to visit the book that had a significant impact on the way I solve harder problems. This modest, but potent book is The Little Schemer. In the world of computer books it's nearly a pamphlet at 195 pages, but just like its language of choice, Scheme, it is concise and powerful.

This book is a step-by-step guide on how to think like a computer scientist. A step-by-step guide may sound dry and boring but this book is surprisingly full of humor. I believe the authors took a page from Douglas Adams and Monty Python when they wrote this book.

The beauty about this book is not just its humor. The beauty is in its compounding process of teaching hard problems by starting with a simple idea and then building on that idea to lead you step-by-step onto more complex, abstract algorithms.

Before you know it, you will be done with this book, thinking like a computer scientist and all the while having a lot of fun in the process.

Update: The Hacker News discussion about this article provides a number of alternative books to my list.

Book Links

Why REST?

2010-10-06T01:42:52Z

Preface

I have to say before you read this that I only have a theoritical understanding of how a web service benefits for being RESTful. Most of the topics I have explored are actually explained in Roy Fieldings section on REST

In the future, I would like to explore and test the assumptions I have made and come up with evidence of thier truthiness.

Why REST?

RESTful web services have been talked about ad-nauseum for a decade. In fact, REST felt like such a hipster's architecture that I pawned it off as a fad and waited this long to only begin to understand it. Prior to understanding a technology, I have to ask myself, "What motivates people to use this technology over others?". I think I may have figured it out.

The key to benefit of REST is the fact that it lives within the confines of HTTP/1.1. Within this world a RESTful web service benefits from all the solutions already solved by the HTTP/1.1 spec. A brief list of standard HTTP solutions that benefit RESTful web services are a uniform interface, caching, asynchronous processing, and security.

A RESTful web service, in particular a resource oriented RESTful web service, starts with a list of URLs that resources live on. Resources are basically entities that live inside your application. A few examples could be a Blog entry, a Forum post, a page hit. These entities don't have to map directly to your applications database, they could merely be logical resources, such as a page hit.

When a client sends a resource to the server, the server will act on that resource. If it's a blog post, it would store the blog post. If it's a logical resource like a page hit, the service would simply increment a counter somewhere.

Each resource has a limited number of verbs that can be done to that resource. The primary verbs are GET, POST, PUT and DELETE. There are inherent restrictions to what these verbs can do. GET never writes on the server; whereas POST/PUT and DELETE always write. Due to these restrictions you can do some simple optimizations to your web service to help it scale.

One of the limitations of SQL is replication and it is a hard problem to solve. In the most basic SQL replication architecture you have a master server and a number of slave servers. The master is written to and the slaves are read from. If your web service abides by the HTTP/1.1 read/write restrictions based on the HTTP method, you can easily route your database connection to your read slaves on GET and your write master on POST, PUT, and DELETE. This would be a more complex solution in a GET based RPC service because every GET to a URL could potentially write to the server. A special router would have to be written that have to know which procedure writes and which reads. You can gain the same benefit with a custom RPC if you support POST and limit your writes to just POST methods.

Another problem already solved by HTTP/1.1 is caching. If you can assume that a GET will always read and if you know when resource was last changed, you can send the Last-Modified header with the response. The client is then responsible for caching the content and storing the Last-Modified date. The client will now send the If-Modified-Since header with each new request for that resource. If the resource has not been modified, the service will respond with a "304 Not Modified" with no information. The service is done. All it had to do is check if the resource has been modified since the client last requested the content. There are other cache headers such as ETag and Cache-Expires, but I think Last-Modified is the simplest to support.

The beauty of your web service supporting HTTP/1.1 caching is that browsers already support it. All AJAX calls to your service will only return content if it's changed. This makes every web browser out there a replicated node for your service! Instant horizontal scaling, how's that feel?

However, you should not trust clients to cache your content, so you should implement a server side caching layer in front of your web service. Varnish can sit in front of your service and serve all cached content from memory. If you want to get really distributed, you can load balance an expandable pool of nginx servers caching content into a expandable pool of memcached servers that sit in front of your web service. If you always send a Last-Modified header, these super fast asynchronous servers can serve up cached content until the content is changed.

Sometimes it takes a long time to process a write and one solution is to make that request asynchronous. HTTP/1.1 has a standard response code for supporting this, "202 Accepted". This status code tells the client, "I have your content, I will process it when I can". This allows you to respond in a timely matter. The client knows that it's write is being worked on. Using a "202 Accepted" response in conjunction with a request with a Last-Modified header allows the client to poll the service for new content and the service will only work as hard as it has to get the Last-Modified value.

An example of asynchronous processing might be implemented in a search engine. Let's say your search engine indexer is time consuming. It parses out keywords, does named entity extraction, and searches for addresses to geocode. This would be a lot of things to do in one HTTP request. Don't worry HTTP/1.1 lets you tell the client you've queue it up. When a new piece of content needs to be index, the service would queue the document and respond with a "202 Accepted" and the client can move on. Eventually the search engine will index the content and the new content will show up in the search engine.

For security, HTTP/1.1 defines support username/password based authentication and SSL encryption so any web service has the ability to use these technologies.

A RESTful web service also has the benefit of having a limited number of actions that can act on a resource. This simplifies authorization and access control tremendously. If you want to restrict a client from writing to a resource you can forbid a client from doing a PUT, POST or DELETE on the URL. You can even set up an OAuth based system external to the web service that allows users to grant authorization to external applications to do things on their behalf.

For instance, a blogging system's OAuth based system could define a permission of "Add/Update blog entries", and this permission would allow PUT /blog/entry/:slug resources and POST /blog/entry/. The web service does not have to worry about authorizing access to that resource because a layer between the client and the service handles that for it.

These are small number of solutions that HTTP/1.1 provides web services if they abide by a RESTful resource oriented architecture and make use of the functionality that HTTP/1.1 has already defined for you.

There are a number of off the shelf products that can benefit RESTful web services such as caching servers, load balancers, and authentication frameworks. These solutions would have to be implemented within your web service or within your service's overloaded HTTP protocol. One example of this reimplementation is SOAP's WS-* suite of specifications. Reinventing the wheel is rampant in the web development world and this is mainly due to ignorance of existing solutions. There is no reason to keep reimplementing functionality solved by HTTP on top of HTTP.

Announcing django-viewtools

2009-02-17T18:30:37Z

Hi, I'm announcing a project I have called django-viewtools

django-viewtools provides a management command to help in debugging and profiling views

Overview

django-viewtools provides a number of management commands for debugging views.

There are a number of flags that can be used when calling the view

-d, --debug: This sets settings.DEBUG to True before calling the: view. This allows you to retrieve the content from a view in production when settings.DEBUG is on
-n, --no-debug: This sets settings.DEBUG to False, handy to see what: will happen if DEBUG is turned off.
--pdb: This starts PDB before the request is handled. Great for when: you have a traceback email and want to set a breakpoint where it occurs.
--pm: This fires up PDB's post_mortum when an error occurs in the: process of calling a view.
--profile: This dumps a hotshot profile file to the location specified: for profiling the performance of a view.

-q, --quiet: Don't output the response of the view

-m, --mute: Turn off output through stdout and stderr, handy for noisy: views with print statements everywhere.

Example

So you have a traceback that gives you the following

File "/home/eric/Projects/django/env/django-1.0.2/lib/python2.5/site-packages/viewtools/views.py", line 4, in py_error
  raise ValueError("This is an error")

An you don't know why it's happening. Even worse, it's only happening on your production server.

Before django-viewtools, you only had one option in this scenario, edit your production code, put some print statements or forced assertion errors to peek into the code to see the problem.

django-viewtools makes things a lot easier

 (django-1.0.2)eric@knoxpy:~/Projects/django/projects/viewtooltest$ ./manage.py viewtools --pdb /py_error/
 > /home/eric/Projects/django/env/django-1.0.2/lib/python2.5/site-packages/viewtools/management/commands/viewtools.py(135)call_view()
 -> try:
 (Pdb) break /home/eric/Projects/django/env/django-1.0.2/lib/python2.5/site-packages/viewtools/views.py:4
 Breakpoint 1 at /home/eric/Projects/django/env/django-1.0.2/lib/python2.5/site-packages/viewtools/views.py:4
 (Pdb) continue
 > /home/eric/Projects/django/env/django-1.0.2/lib/python2.5/site-packages/viewtools/views.py(4)py_error()
 -> raise ValueError("This is an error")
 (Pdb) list
   1 from django.db import connection as db
   2
   3 def py_error(request):
   4 B-> raise ValueError("This is an error")
   5
   6 def db_error(request):
   7 c = db.cursor()
   8 c.execute("select * from asotuhaosetuhaosethuasote")
 [EOF]
 (Pdb) print request
 <WSGIRequest
 GET:<QueryDict: {}>,
 POST:<QueryDict: {}>,
 COOKIES:{},
 META:{'CONTENT_TYPE': 'text/html; charset=utf-8',
  'HTTP_COOKIE': <SimpleCookie: >,
  'PATH_INFO': u'/py_error/',
  'QUERY_STRING': '',
  'REQUEST_METHOD': 'GET',
  'SCRIPT_NAME': u'',
  'SERVER_NAME': 'testserver',
  'SERVER_PORT': '80',
  'SERVER_PROTOCOL': 'HTTP/1.1',
  'wsgi.errors': <cStringIO.StringO object at 0x10a68b8>,
  'wsgi.input': <django.test.client.FakePayload object at 0x10ec150>,
  'wsgi.multiprocess': True,
  'wsgi.multithread': False,
  'wsgi.run_once': False,
  'wsgi.url_scheme': 'http',
  'wsgi.version': (1, 0)}>
 (Pdb)

Isn't that just great? I can see all the local variables in the view.

Python Encodings and Unicode

2008-11-21T20:24:43Z

I am sure there has been a number of explanations on Unicode and Python but I'm going to do a little write up for my own sake.

Byte Streams VS Unicode Objects

Let's start by defining what a string in Python is. When you use the string type what you're actually doing is storing a string of bytes.

[  a ][  b ][  c ] = "abc"
[ 97 ][ 98 ][ 99 ] = "abc"

With this example, the string "abc" is a string of the bytes, 97, 98, 99 in the ASCII table. A deficiency of Python 2.X is that by default it treats every string as ASCII. Unfortunately ASCII is lowest common denominator of Latin type character sets.

ASCII is the first 127 characters of a number of character mappings. Character mappings such as windows-1252 (aka Latin-1, aka cp1252) and UTF-8 both have the same first 127 characters. You're safe mixing string encodings when every byte value in your string is lower that 127. However it's really dangerous to make that assumption. More on that later.

A problem arises when you mix encodings that have bytes in them that are greater than 126. Let's take another string encoded as windows-1252. Character map windows-1252 is a 8-bit character map so there can be a total of 256 characters in the table. The first 127 match ASCII, the second 127 are extra characters that the windows-1252 encoding defines.

A windows-1252 encoded string looks like this:
[ 97 ] [ 98 ] [ 99 ] [ 150 ] = "abc–"

Windows-1252 is still a string of bytes but do you see that the last byte is greater than 126. If Python attempts to decode that byte stream as the default encoding ASCII, it will throw an error. Let's see what happens when Python tries to decode that string.

>>> x = "abc" + chr(150)
>>> print repr(x)
'abc\x96'
>>> u"Hello" + x
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ASCII' codec can't decode byte 0x96 in position 3: ordinal not in range(128)

Let's take another string encoded as UTF-8

A UTF-8 encoded string looks like this:
[ 97 ] [ 98 ] [ 99 ] [ 226 ] [ 128 ] [ 147 ] = "abc–"
[0x61] [0x62] [0x63] [0xe2]  [ 0x80] [ 0x93] = "abc-"

If you pull up your handy dandy Unicode character table, you'll see that an en dash is Unicode codepoint 8211 (0x2013). That value is greater than the 127 max of ASCII. Hell it's greater than the max value that 1 byte can store. Since 8211 (0x2013) is actually two bytes UTF-8 has to do some magic to tell the system there are three bytes needed to store one character. Again, let's see what happens when Python attempts to use the default ASCII encoding on a UTF-8 encoded string that has characters greater than 126.

>>> x = "abc\xe2\x80\x93"
>>> print repr(x)
'abc\xe2\x80\x93'
>>> u"Hello" + x
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ASCII' codec can't decode byte 0xe2 in position 3: ordinal not in range(128)

As you can see, Python always defaults to ASCII. It hits that fourth byte which has a decimal value of 226 that's greater than 126 so Python raises an error. This is the trouble with mixing encodings.

Decoding Byte Streams

The term decoding can be confusing when you first start learning about Unicode in Python. You decode byte streams to make Unicode objects and encode Unicode objects into byte streams.

Python has to know how to decode a byte stream to Unicode. When you take a byte stream, you call it's "decode" method to create a Unicode object from it.

Your best bet is to decode byte streams to Unicode as early as possible.

>>> x = "abc\xe2\x80\x93"
>>> x = x.decode("utf-8")
>>> print type(x)
<type 'unicode'>
>>> y = "abc" + chr(150)
>>> y = y.decode("windows-1252")
>>> print type(y)
>>> print x + y
abc–abc–

Encoding Unicode to byte streams

Unicode objects are an encoding agnostic representation of text. You can't simply output a Unicode object. It has to be turned into a byte string before it's outputted. Python will be nice enough to do it for you however Python defaults to ASCII when encoding a Unicode object to a byte stream, this default behavior can be the source of many headaches.

>>> u = u"abc\u2013"
>>> print u
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 3: ordinal not in range(128)
>>> print u.encode("utf-8")
abc–

Using the codecs module

The codecs module can help out a lot when ingesting byte streams. You can open files with an encoding defined and what you read from that file will automatically be converted to Unicode objects.

Try this

>>> import codecs
>>> fh = codecs.open("/tmp/utf-8.txt", "w", "utf-8")
>>> fh.write(u"\u2013")
>>> fh.close()

What that did was take an Unicode object and write it out as a utf-8 file. You can do the same in the other direction.

Try this

When reading data from a file codecs.open create a file object that will automatically convert the utf-8 encoded file into a Unicode object.

Let's take the previous example and use the urllib stream directly

>>> stream = urllib.urlopen("http://www.google.com")
>>> Reader = codecs.getreader("utf-8")
>>> fh = Reader(stream)
>>> type(fh.read(1))
<type 'unicode'>
>>> Reader
<class encodings.utf_8.StreamReader at 0xa6f890>

One-liner version

>>> fh = codecs.getreader("utf-8")(urllib.urlopen("http://www.google.com"))
>>> type(fh.read(1))

You have to be careful with the codecs module. Whatever you pass to it must be a Unicode object otherwise it will try to automatically decode the byte stream as ASCII

>>> x = "abc\xe2\x80\x93" # our "abc-" utf-8 string
>>> fh = codecs.open("/tmp/foo.txt", "w", "utf-8")
>>> fh.write(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.5/codecs.py", line 638, in write
  return self.writer.write(data)
File "/usr/lib/python2.5/codecs.py", line 303, in write
  data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3: ordinal not in range(128)

Crap, there Python goes again, trying to decode everything as ASCII.

Problems with slicing up UTF-8 byte streams

Since an UTF-8 encoded string is just a list of bytes, len() and slicing do not work correctly. Take our string from before

[ 97 ] [ 98 ] [ 99 ] [ 226 ] [ 128 ] [ 147 ] = "abc–"

Now do the following

>>> my_utf8 = "abc–"
>>> print len(my_utf8)
6

What? It looks like 4 chars, but len is saying 6. This is because len is counting bytes not characters.

>>> print repr(my_utf8)
'abc\xe2\x80\x93'

Now let's try slice the string

>>> my_utf8[-1] # Get the last char
'\x93'

Crap, that's the last byte, not the last char.

To slice up UTF-8 correctly, your best bet is to decode the byte stream to create a Unicode object from it. Then you can manipulate, count, whatever safely.

>>> my_unicode = my_utf8.decode("utf-8")
>>> print repr(my_unicode)
u'abc\u2013'
>>> print len(my_unicode)
4
>>> print my_unicode[-1]
–

When Python automatically encodes/decodes

There are a number of cases when Python could throw an error when it tries to automatically encode or decode as ascii.

The first case is when it tries to concat unicode and string values together

>>> u"" + u"\u2019".encode("utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0:   ordinal not in range(128)

The same will happen when you try to join a list. Python will automatically decode strings to unicode when there are both strings and unicode objects in the list

>>> ",".join([u"This string\u2019s unicode", u"This string\u2019s utf-8".encode("utf-8")])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 11:  ordinal not in range(128)

Or when trying to format a string

>>> "%s\n%s" % (u"This string\u2019s unicode", u"This string\u2019s  utf-8".encode("utf-8"),)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 11: ordinal not in range(128)

Basically when you are mixing unicode and byte strings together, you can cause an error.

Take for instance you're taking a utf-8 file and then adding some text to it that is an unicode object. There can be a UnicodeDecodeError

>>> buffer = []
>>> fh = open("utf-8-sample.txt")
>>> buffer.append(fh.read())
>>> fh.close()
>>> buffer.append(u"This string\u2019s unicode")
>>> print repr(buffer)
['This file\xe2\x80\x99s got utf-8 in it\n', u'This string\u2019s unicode']
>>> print "\n".join(buffer)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 9: ordinal not in range(128)

You can fix that by using the codecs module to load in the file as unicode

>>> import codecs
>>> buffer = []
>>> fh = open("utf-8-sample.txt", "r", "utf-8")
>>> buffer.append(fh.read())
>>> fh.close()
>>> print repr(buffer)
[u'This file\u2019s got utf-8 in it\n', u'This string\u2019s unicode']
>>> buffer.append(u"This string\u2019s unicode")
>>> print "\n".join(buffer)
This file’s got utf-8 in it

This string’s unicode

As you can see, the stream produced by codecs.open automatically converted the string to unicode when the data is read.

Best Practices

Decode First, Encode Last

When in doubt, encode as utf-8

Use codecs with unicode objects to simplify things

Decode First means whenever taking byte stream input, decode that input to Unicode as early as possible. This will help prevent issues where len() and slice malfunctions with utf-8 byte streams.

Encode last means only encode to byte streams when you're ready to output the text to something. That output could be a file, a database, a socket, whatever. Only encode the Unicode objects when you're done with them. Encode last also means, don't let Python encode your Unicode objects for you. Python will use ASCII and your programs will crash.

When in doubt, encode as UTF-8 means this: Since UTF-8 can handle any Unicode character, your best bet is to use it as opposed to window-1252 or god forbid ASCII.

The codecs module help you skip steps when ingesting streams such as files or sockets. Without the tools provided by the codecs module you'll have to read the content of the files into byte streams and then decode those byte streams into Unicode objects.

The codecs module lets you read the bytes in and decode the bytes on the fly getting your text into unicode objects much simplier and with less overhead.

Explaining UTF-8

This final section will give you primer on UTF-8, you can ignore this unless you're a super-geek.

With UTF-8, any byte between 127 and 255 are special. Those bytes tell the system that the bytes following the current byte are part of a multi-byte sequence.

Our UTF-8 encoded string looks like this:
[ 97 ] [ 98 ] [ 99 ] [ 226 ] [ 128 ] [ 147 ] = "abc–"

The last three bytes are a UTF-8 multi-byte sequence. If you convert the first of the three, 226 to binary you can see what it looks like.

11100010

The first three bits tell the system that it is starting a three byte sequence that is 226, 128, 147

So taking the full sequence bytes

11100010 10000000 10010011

Then you apply the following mask for three byte sequences (described here)

1110xxxx 10xxxxxx 10xxxxxx
XXXX0010 XX000000 XX010011 Remove the X's
0010       000000   010011 Collapse the numbers
00100000 00010011          Get Unicode number 0x2013, 8211 The "–"

This is a basic primer on how UTF-8, refer to the UTF-8 Wikipedia page for greater detail

PDB and django.test.client

2008-11-17T00:14:57Z

So you have a site in production and someone called you up that there is a bug on the site. Your template designer is trying to make a change to a template and the change is isn't showing up on the site. You make the change on you development server and the change takes effect. What is going wrong?

You can do a number of things:

You can call your template designer an idiot and figure it's a problem between the chair and the keyboard

You can spend hours trying to figure out how the development server and the production server differ and bang your head against the closest hard object until you find it.

You can be a bad boy or girl and start changing code on the production server

Or, you can use PDB and django.test.client to keep from spending all your hard earned money on hard liquor.

The url in question is the homepage. We have a simple view, all it does is load a template.

 from django.http import HttpResponse
 from django.template import loader, Context

 def index(request):
     t = loader.get_template("index.html")
     c = Context({})
     return HttpResponse(t.render(c))

First we'll look at the index.html template that loads the page:

<html>
<head><title>Hi</title></head>
<body>
<h1>Welcome to my site, click on a link below to go somewhere</h1>
{% include "includes/navigation.html" %}
</body>
</html>

Something is wrong with the include tag, it's not loading the correct template. Taking a quick look in they site templates shows that the template exists. What's going on!

$ ls pdb_example_templates/includes/
navigation.html  navigation.HTML~

Ok, there's something "wrong" with the {% include %} tag. A quick glance at django/template/loader_tags.py shows that the ContantIncludeNode loads the template on line 102. We're going to drill down the stack to find where the template is actually loaded. Line 102 has get_template, get_template is defined in django.template.loader. Open that up and find the definition of get_template. We see that get_template calls find_template_source. That's up on the top of django.template.loader on line 47. We know that there is something funky with the loading of the template so we have our starting point for PDB.

Let's SSH into the production server and start playing. First, what we need to do is create a simple little script to call the view. Let's call it wtf.py:

def main():
    from django.test.client import Client
    c = Client()
    response = c.get("/")
        print response.content

Since we're using ./manage.py shell for this, we can't just do ol' if __name__ == '__main__' trick to run the script so we create a main() function to call from interactive mode.

$ ~/src/pdb_example/manage.py shell
Python 2.5.1 (r251:54863, Mar  7 2008, 03:39:23)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import wtf

In [2]: wtf.main()
<html>
  <head><title>Hi</title></head>
</html>
<body>
<h1>Welcome to my site, click on a link below to go somewhere</h1>
<ul>
  <li><a href="/blog/">My Blog</a></li>
  <li><a href="http://www.flickr.com/photos/ericmoritz/">My Photos</a></li>
</ul>
</body>

Ok our little script calls up the correct view and generates the incorrect content, good. Next step, get PDB into the mix.

There are two ways of doing this, the python way and the ipython way. I'll do it the python way first for all you dummies that aren't using ipython yet (what are you waiting for?). Edit wtf.py and add import pdb; pdb.set_trace() right before the c.get("/") line.

def main():
    from django.test.client import Client
    c = Client()
    import pdb; pdb.set_trace()
    response = c.get("/")
    print response.content

This will drop you into pdb before the view is called so you can set a breakpoint.

The find_template_source function does some module level caching stuff in the beginning. The guts that we want is near the bottom on line 67.

So let's fire up manage.py shell and set a breakpoint.

$ ~/src/pdb_example/manage.py shell
Python 2.5.1 (r251:54863, Mar  7 2008, 03:39:23)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.1 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import wtf

In [2]: wtf.main()
> /home/eric/wtf.py(5)main()
-> response = c.get("/")
(Pdb) break django/template/loader.py:67
Breakpoint 1 at /opt/django/versions/django-1.0/django/template/loader.py:67
(Pdb) cont
> /opt/django/versions/django-1.0/django/template/loader.py(67)find_template_source()
-> for loader in template_source_loaders:
(Pdb) list
 62                     import warnings
 63                     warnings.warn("Your TEMPLATE_LOADERS setting includes %r, but your Python installation doesn't support that type of template loading. Consider removing that line from TEMPLATE_LOADERS." % path)
 64                 else:
 65                     loaders.append(func)
 66             template_source_loaders = tuple(loaders)
 67 B->     for loader in template_source_loaders:
 68             try:
 69                 source, display_name = loader(name, dirs)
 70                 return (source, make_origin(display_name, loader, name, dirs))
 71             except TemplateDoesNotExist:
 72                 pass
(Pdb) name
Out[2]: 'index.html'

Whoops, we're going to go through the load of every template and probably start drinking. Let's make our breakpoint more specific.

(Pdb) clear 1
Deleted breakpoint 1
(Pdb) break django/template/loader.py:67, name == 'includes/navigation.html'
Breakpoint 2 at /opt/django/versions/django-1.0/django/template/loader.py:67

Ok, now our breakpoint will only break if the name of the template being loaded is called includes/navigation.html. Let's continue.

(Pdb) continue
> /opt/django/versions/django-1.0/django/template/loader.py(67)find_template_source()
-> for loader in template_source_loaders:
(Pdb) name
Out[2]: u'includes/navigation.html'
(Pdb) list
 62                     import warnings
 63                     warnings.warn("Your TEMPLATE_LOADERS setting includes %r, but your Python installation doesn't support that type of template loading. Consider removing that line from TEMPLATE_LOADERS." % path)
 64                 else:
 65                     loaders.append(func)
 66             template_source_loaders = tuple(loaders)
 67 B->     for loader in template_source_loaders:
 68             try:
 69                 source, display_name = loader(name, dirs)
 70                 return (source, make_origin(display_name, loader, name, dirs))
 71             except TemplateDoesNotExist:
 72                 pass

Cool, we've successfully broke at the correct line with the correct template name. Let's walk through the code.

 (Pdb) next
> /opt/django/versions/django-1.0/django/template/loader.py(68)find_template_source()
-> try:
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loader.py(69)find_template_source()
-> source, display_name = loader(name, dirs)

Now we're positioned right before the loader is called. Now we'll use the step command to step into that function.

(Pdb) step
--Call--
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(45)load_template_source()
-> def load_template_source(template_name, template_dirs=None):
(Pdb) list
 40                 yield safe_join(template_dir, template_name)
 41             except ValueError:
 42                 # The joined path was located outside of template_dir.
 43                 pass
 44
 45  -> def load_template_source(template_name, template_dirs=None):
 46         for filepath in get_template_sources(template_name, template_dirs):
 47             try:
 48                 return (open(filepath).read().decode(settings.FILE_CHARSET), filepath)
 49             except IOError:
 50                 pass

We're now at the start of the load_template_source function. Let's walk through this function to see if it finds a template.

(Pdb) next
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(46)load_template_source()
-> for filepath in get_template_sources(template_name, template_dirs):
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(47)load_template_source()
-> try:
(Pdb) filepath
Out[2]: u'/home/eric/src/pdb_example/example/templates/includes/navigation.html'

Ok, the first template it's trying to load is the template in our app's templates folder. This shouldn't work, so let's continue to verify that our assumption is correct.

(Pdb) next
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(48)load_template_source()
-> return (open(filepath).read().decode(settings.FILE_CHARSET), filepath)
(Pdb) next
--Return--
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(48)load_template_source()->(u'<ul>\n...n</ul>\n', u'/home/...ion.html')
-> return (open(filepath).read().decode(settings.FILE_CHARSET), filepath)
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loader.py(70)find_template_source()
-> return (source, make_origin(display_name, loader, name, dirs))
(Pdb) list
 65                     loaders.append(func)
 66             template_source_loaders = tuple(loaders)
 67 B       for loader in template_source_loaders:
 68             try:
 69                 source, display_name = loader(name, dirs)
 70  ->             return (source, make_origin(display_name, loader, name, dirs))
 71             except TemplateDoesNotExist:
 72                 pass
 73         raise TemplateDoesNotExist, name
 74
 75     def get_template(template_name):
(Pdb) source
Out[2]: u'<ul>\n  <li><a href="/blog/">My Blog</a></li>\n  <li><a href="http://www.flickr.com/photos/ericmoritz/">My Photos</a></li>\n</ul>\n'
(Pdb)

We've walked through the loader's function and have returned with a success. However, this is the app_directories loader, the site should of used the filesystem loader first because the template designer has his own set of templates that are outside of the django project code.

Let's look at our TEMPLATE_LOADERS setting to see what's up

# List of callables that know how to import templates from various sources.
TEMPLATE_LOADERS = (
    'django.template.loaders.app_directories.load_template_source',
    'django.template.loaders.filesystem.load_template_source',
)

Doh! Our loaders are backward. Who did that?

$ svn blame settings.py
42 newman TEMPLATE_LOADERS = (
42 newman     'django.template.loaders.app_directories.load_template_source',
42 newman     'django.template.loaders.filesystem.load_template_source',
42 newman )

Newman! Hang on, I have to go smack my other programmer in the back of the head...

Now to use pdb with ipython with all the benefits of ipython (colors, tab completion, etc). There is a module here that will help you out. Just download that script as ipdb.py in the same folder as wtf.py (or somewhere in sys.path)

$ curl http://trac.gotcha.python-hosting.com/file/bubblenet/pythoncode/ipdb/ipdb/__init__.py?format=txt > ipdb.py

Then change your code like this:

def main():
  from django.test.client import Client
  c = Client()
  import ipdb; ipdb.set_trace()
  response = c.get("/")
  print response.content

So, there you have it. How to use pdb and django.test.client to solve bugs in production without editing the code and potentially disrupting service.

Find the number of days until next day of week

2008-07-10T23:03:29Z

I've had to figure this out before and I forgot it. Here it is preserved

 def daysuntilnextdow(start, next):
     """Determine how many days until the next Day of week

     start: The day of the week to start from
     next: The day of the week to go to

     returns a number of days until the next day of week

     Note, start is inclusive, so if next is the same day,
     you will receive 0

     Example:

     >> start = 0 # Sunday
     >> next = 1 # Monday
     >> daysuntilnextdow(start, next)
     1
     >> start = 3 # Wednesday
     >> next = 0 # Sunday
     >> daysuntilnextdow(start, next)
     4
     >> start = 0 # Sunday
     >> next = 0 # Sunday
     >> daysuntilnextdow(start, next)
     0
     """
     return ((next - start) % 7)

Understanding PSQL's MVCC

2008-06-18T16:40:54Z

Understanding PSQL's MVCC

PostgresSQL implements something called MVCC. Which alleviates the need for a Read-Lock in many cases. However when Glenn Franxman and I were trying to figure out how this works we were having trouble understanding it completely.

If you ask about MVCC in the #postgres irc channel on Freenode, they will direct you to the following three links

http://en.wikipedia.org/wiki/Multiversion_concurrency_control

http://www.developer.com/net/vb/article.php/877181

http://www.postgresql.org/docs/current/static/mvcc.html

With SQL there are four transaction isolation levels, unfortunately the Wikipedia entry only describes one, serializable. So relying on the Wikipedia documentation to describe how Postgres' opperates is not completely accurate. I think that internally Postgres may operate the way that the Wikipedia entry describes, but uses a write-lock to emulate a "Read Commit" isolation level to prevent the transaction abort that would normally occur by one transaction writing and comitting data . I don't know if that's true or not.

Another confusing bit about Postgres is that Postgres only implements two levels, "Read Committed" and "Serializable" even though you can set the other isolation levels. In the documentation, the four transaction isolation levels refers to the SQL standard and not Postgres' actually functionality.

So when you look at the four transaction isolation levels, you'll notice that it says that a dirty read is possible if you set the isolation level to "READ UNCOMMITTED", however because "READ UNCOMMITTED" is the same as "READ COMMITTED" internally in Postgres, you can never get a dirty read even if you set the isolation level to READ UNCOMMITTED. That's ok because, as the documentation states, "the four isolation levels only define which phenomena must not happen, they do not define which phenomena must happen."

I'll run some examples of concurrent transaction to show you how the four levels act.

Read Committed

Transaction 1	Transaction 2
mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# update mvcc_test set i = i + 1; UPDATE 1	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
	mvcc_test=# update mvcc_test set i = i + 1; -- 7
mvcc_test=# commit; COMMIT	UPDATE 1 mvcc_test=# -- blocked until the other commited
	mvcc_test=# commit; COMMIT
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 3 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 3 (1 row)

Read Uncommitted

With Read Uncommited I observed the same behavior as Read Commited. This reflects the transaction documentation:

But internally, there are only two distinct isolation levels, which correspond to the levels Read Committed
and Serializable. When you select the level Read Uncommitted you really get Read Committed, and when you select
Repeatable Read you really get Serializable, so the actual isolation level might be stricter than what you select.

Transaction 1	Transaction 2
mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ UNCOMMITTED; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ UNCOMMITTED; SET
mvcc_test=# BEGIN; -- 1 BEGIN	mvcc_test=# BEGIN; -- 2. BEGIN
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# update mvcc_test set i = i + 1; UPDATE 1	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
	mvcc_test=# update mvcc_test set i = i + 1;
mvcc_test=# commit; COMMIT	UPDATE 1 mvcc_test=# -- blocked until the other commited
	mvcc_test=# commit; COMMIT
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 3 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 3 (1 row)

SERIALIZABLE

Transaction 1	Transaction 2
mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# SERIALIZABLE; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# SERIALIZABLE; SET
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# update mvcc_test set i = i + 1; UPDATE 1	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
	mvcc_test=# update mvcc_test set i = i + 1;
mvcc_test=# commit; COMMIT	ERROR: could not serialize access due to concurrent update
	mvcc_test=# select * from mvcc_test; ERROR: current transaction is aborted, commands ignored until end of transaction block
	mvcc_test=# commit; ROLLBACK
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)

Repeatable Read

Refering to the same paragraph in the transaction documentation:

But internally, there are only two distinct isolation levels, which correspond to the levels Read Committed
and Serializable. When you select the level Read Uncommitted you really get Read Committed, and when you select
Repeatable Read you really get Serializable, so the actual isolation level might be stricter than what you select.
<http://www.postgresql.org/docs/current/static/transaction-iso.html>

Repeatable Read is the same as Serializable.

Transaction 1	Transaction 2
mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# REPEATABLE READ; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# REPEATABLE READ; SET
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# update mvcc_test set i = i + 1; UPDATE 1	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
	mvcc_test=# update mvcc_test set i = i + 1;
mvcc_test=# commit; COMMIT	ERROR: could not serialize access due to concurrent update
	mvcc_test=# select * from mvcc_test; ERROR: current transaction is aborted, commands ignored until end of transaction block
	mvcc_test=# commit; ROLLBACK
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)

What is possible in Isolation Levels

So, in the four transaction isolation levels table, it describes what is possible in the different isolation levels. Since I've demostrated that there are only two isolation levels in Postgresql. I'll demostrate the two read phenomena that could happen in both Serializable and Read Committed.

Let me explain what those two phenomena are. Here is a good explanation of read phenomena. I'm just simply repeating what is stated at that link

Nonrepeatable Read: S1 reads data which is later changed and commited by S2. If S1 reads the same data again (after S2's commit) and finds it to have changed or to be deleted (according to S2's changes), this is called a non-repeatable read. It is called non-repeatable because the same select statement doesn't return the same data (within the same transaction).
Phantom read: S1 reads data (select) with a specific where condition. After this read, S2 inserts some data that meets the S1's where condition and commits the inserted data. When S1 issues a select statement with the same where condition, it finds new records. It is called phantom read because the new records seem to be of phantom origin. A phantom read is thus a special case of a non-repeatable read.

Read Committed / Nonrepeatable Read Attempt

The SQL Standard says that with a Read Committed Isolation level, a "Nonrepeatable Read" is possible. Here is my attempt in proving that assertion.

Transaction 1	Transaction 2
mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# update mvcc_test set i = i + 1; UPDATE 1	--
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# commit; COMMIT	--
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row) This is a Nonrepeatable Read. Because the data has change while inside a transaction.

Read Committed / Phantom Read Attempt

The SQL Standard says that with a Read Committed Isolation level, a "Phantom Read" is possible. Here is my attempt in proving that assertion.

mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET
mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- (0 rows)	mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- (0 rows)
mvcc_test=# insert into mvcc_test (id, i) values (2, 1); INSERT 0 1	mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- (0 rows)
mvcc_test=# commit; COMMIT	mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- 2 \| 1 (1 row) This is a phantom read because the row poofed into existance inside the transaction when it didn't exist before.

Serializable / Nonrepeatable Read Attempt

The SQL Standard says that with a Serializable Isolation level, a "Nonrepeatable Read" is not possible. Here is my attempt in proving that assertion.

Transaction 1	Transaction 2
mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# SERIALIZABLE; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# SERIALIZABLE; SET
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# update mvcc_test set i = i + 1; UPDATE 1	--
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row)
mvcc_test=# commit; COMMIT	--
mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 2 (1 row)	mvcc_test=# select * from mvcc_test; id \| i ----+--- 1 \| 1 (1 row) This is not a Nonrepeatable Read. Because the data did not change while inside a transaction.

Read Committed / Phantom Read Attempt

The SQL Standard says that with a Read Committed Isolation level, a "Phantom Read" is possible. Here is my attempt in proving that assertion.

mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET	mvcc_test=# BEGIN; BEGIN mvcc_test=# SET TRANSACTION ISOLATION LEVEL mvcc_test-# READ COMMITTED; SET
mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- (0 rows)	mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- (0 rows)
mvcc_test=# insert into mvcc_test (id, i) values (2, 1); INSERT 0 1	mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- (0 rows)
mvcc_test=# commit; COMMIT	mvcc_test=# select * from mvcc_test where id = 2; id \| i ----+--- (0 row) This is a phantom read because the new row did not appear inside the transaction after it was commit in the first transaction.

Fun with APIs

2008-06-18T16:38:41Z

So, you're looking for a job you say... Need to brush up on your knowledge set... I've got the tool for you!

career_chooser.py

import urllib
import re
import sys
import cgi

API_KEY="USE YOUR OWN"
def get_result(query):
    url = "http://api.indeed.com/apisearch?q=\"%s\"&l=&start=0&limit=1&sort=&filter=on&latlong=off&key=%s&format=xml" % (query, API_KEY)
    result_count_re = re.compile(r"<totalresults>(\d+)</totalresults>")
    content = urllib.urlopen(url).read()

    match = result_count_re.search(content)
    if match:
        return int(match.groups()[0])

def display(label, count):
    count_str = "%10d" % (count)
    spaces = 78 - (len(label) + len(count_str))
    return "%s:%s%s" % (label, " " * spaces, count_str)

result_list = []
assert len(sys.argv) > 1, "Usage %s [Query] [Query] ..." % (sys.argv[0],)
for query in sys.argv[1:]:
    result_list.append((query, get_result(query)))

result_list.sort(lambda x,y: x[1] - y[1], reverse=True)
print "\n".join(map(lambda x: display(*x),result_list))

Try it out

C:                                                                       136105
Java:                                                                     89132
HTML:                                                                     87963
Assembly:                                                                 63423
XML:                                                                      62938
C%2B%2B:                                                                  60774
Javascript:                                                               42368
ASP:                                                                      42067
Perl:                                                                     35742
J2EE:                                                                     31609
Visual+Basic:                                                             28924
PHP:                                                                      17972
ADA:                                                                      11821
Struts:                                                                   11425
Python:                                                                   10416
COBOL:                                                                     6964
Ruby:                                                                      5439
Ruby+on+Rails:                                                             2515
FORTRAN:                                                                   2008
Zend:                                                                       270
Django:                                                                     228
CakePHP:                                                                    109
CodeIgniter:                                                                 31
JRuby:                                                                       30

Template Tag Caveat

2008-06-17T21:46:55Z

In the Django template system. There is a small caveat that you need to recognize when developing your own template tags.

When Django parses the Node tree it creates a template.Node instance for each template tag in the template. The node tree is just like our beloved HTML DOM.

So for instance take the following

 <ul>
   <li>1</li>
   <li>2</li>
   <li>3</li>
   <li>4</li>
   <li>5</li>
 </ul>

That turns into something like this

 <ul:Element>
   <li:HTMLElement>
   <li:HTMLElement>
   <li:HTMLElement>
   <li:HTMLElement>
   <li:HTMLElement>

The indention shows that the li's are child nodes of the ul tag. Each li in a different instance of an HTMLElement, each with their own state.

So for a Django Template take the following

 {% block content %}
   {% firstof var1 var2 var3 %}
   {% firstof var1 var2 var3 %}
   {% firstof var1 var2 var3 %}
   {% firstof var1 var2 var3 %}
 {% endblock %}

The node tree would look like this

 <BlockNode>
   <FirstOfNode>
   <FirstOfNode>
   <FirstOfNode>
   <FirstOfNode>

So we have a block node with four FirstOfNode instances

In python this would translate roughly to

 f1 = FirstOfNode(...)
 f2 = FirstOfNode(...)
 f3 = FirstOfNode(...)
 f4 = FirstOfNode(...)

 f1.render()
 f2.render()
 f3.render()
 f4.render()

Everything looks fine here. Render is called once per node instance.

Now, check this out

 {% block content %}
   {% for i in value_list %}
     {% repr obj %}
   {% endfor %}
 {% endblock %}

The node tree that django builds would look something like this

 <BlockNode>
   <ForNode>
     <ReprNode>

Now each node in that tree is it's own object instance with it's own state. This can be an issue if you don't realize one thing, each node is persistent while the template is rendering.

Let me write out how the template rendering of the a loop would look in python with the ReprNode

 repr_node = ReprNode("obj")

 for i in range(10):
     repr_node.render(context)

Can you tell what the problem? We're calling the render method on the same instance of the Node.

Let's peek under the covers and look at ReprNode's definition

 class ReprNode(template.Node):
     def __init__(self, obj)
         self.obj = obj

     def render(self, context):
         self.obj = template.resolve_variable(self.obj, context)
         return str(self.obj)

     def do_repr(parser, token):
         tag_name, obj = token.split_contents()
         return ReprNode(obj)

Now look at what's going on. self.obj is assumed to be the name of the variable in the context. That's fine when render is called once. When render called a second time, self.obj is now the actual value of obj from the context. What happens when you try to resolve that? You get a VariableDoesNotExist error, uh oh!

I made the assumption that render() is only called once. You don't realize that inside a for tag, the render method is called repeatedly. This can cause tons of issues. If it wasn't a huge design change, Template Nodes should probably be Static classes.

I don't know if this is technically a bug in the Django templating system, bad design, bad documentation, or just simply poor assumptions on my part, but I came across this issue today and it took stepping through PDB to find the problem with my custom template tag (the ReprNode was completely made up for the example)

Example code <http://www.djangosnippets.org/snippets/811/>

iphone-3g

2008-06-11T13:12:05Z

What can I say about the iphone 3g that hasn't been said. All I can say is I want one and I want to develop on it.

The one thing that sold me was the true GPS support. This thing is going to herald in the geotagging revolution that has been conspiring in dark rooms for a couple years now.

Just like podcasting which hadn't taken off until iPods took over, the iphone is going to do the same for geotagging.

We're going to be geotagging everything. I'm anticipating some really awesome apps based on gps.

Remember http://www.dodgeball.com/? Awesome idea.... Now think of dodgeball.com on the iphone with gps, a perfect idea.

appengine vs twitter part 2

2008-05-21T16:54:48Z

I've created a project in launchpad for Meow. You can now submit bug reports and feature requests into launchpad.

If your a Django or Appengine hacker and want to help me out, simply branch the Meow code with bazaar and you can "bzr send" me patches. I'll be happy to merge them if they're handy.

There is now a code link on the bottom of every Meow page to get you to the launchpad page.

Meow has also aquired the Affero GPLv3 license as well to protect it's open source nature on the web. The AGPLv3 requires modified source code to be made available even if it's running behind a web server.

appengine vs twitter

2008-05-20T03:17:11Z

So Glenn Franxman was opining this afternoon about how twitter is going to reach a point where they're going to grow to big and need to make money somehow. He said that they're either going to have to throw ads everywhere or hope to be aquired.

He thinks that they're like every startup in web 2.0 where they're hoping to be aquired by Google.

I said to him, "Why would Google aquire them? I could write twitter on google app engine in like 30 minutes."

Well it took me two hours to shoehorn Django into GAE and about an hour to get a working prototype. It's far from complete, but it works. I still have to integrate textmarks to enable SMS i/o, add following and replies, but it works.

You can view the app at http://meow.appspot.com/ and you can checkout the code at https://launchpad.net/meow

Experimenting

2008-05-09T11:01:00Z

I ran across Smisk two days ago and though. "Hmm, someone should write a wsgi adaptor for that".

Maybe an hour after I had that thought, I started working on one. It was pretty easy. Smisk's classes seem to be inspired by WSGI, all the wsgi environment variables had a Smisk equivilent. So the adaptor was basically putting tab A in Slot B and Tab B is slot A, etc.

If you noticed in my del.icio.us feed. I bookmarked Smisk itself and three hours later, I bookmarked my launchpad branch of smisk-wsgi. Pretty cool.

Fifteen hours later across the globe in Sweden, Smisk's project owner Rasmus Anderson, merged my code in with some changes.

So in a total of 3 hours, something went from concept to reality thanks to Python. I don't think the quickness is a testimate to my mad hacking skills. It's really a testimate to Python's agility.

So what is Smisk and why is it cool? Smisk is a low level web framework built in C but controled by Python. Basically it's a fastcgi interface to Python with some really well thought out classes bolted on.

What does the wsgi adapter get you? It basically allows you to expose your wsgi based framework/app (Django, Cherrypy, etc) to fastcgi via C. Now, what does that get you? Well it gets you a faster fastcgi protocol (I think).

Now, I don't have any quantifiable evidence if it's actually faster or not. Rasmus has some performance tests on the Smisk wiki. Logic would say it would be. If it is faster, it's probably in the range of miliseconds, so it's probably not something to lose sleep over. Generally your application slows down due to database performance.

Is the WSGI adaptor compliant? For the most part. I'm doing tests with the wsgiref's validator app. There are some methods that we have to add to smisk's Stream class to make it valid. That means C work, so I have to brush up on my skills, but it shouldn't be long.

I'm currently running this blog on Django on top of Smisk. I found some nasty bugs (causes segfaults on POST) that were in the Stream class, so I wouldn't recomend doing the same until Rasmus merges the changes I sent him.

Will this replace flup as a fastcgi to wsgi adapter? I doubt it at this point, flup has some nice pooling features that would have to be built into Smisk in order for Smisk to have the flexibility of flup. I don't know if those features are in Rasmus' plans for Smisk or not.

is opensource really all that great for companies

2008-04-29T17:14:33Z

So I released my feedclowd code as open source and I needed to pick license. The requirements I wanted in a license were this, you may use the source code but if you modify that source code you must make your source code available. In my ignorance, that's what I thought the GPL provided.

Well after some discussion with numerous people, that's not the case. What the GPL provides is that if someone takes your code and modifies it they only have to make that code available if they distribute the new code.

This is a one way relationship. The main benefit that a company could reap if their code is open sourced would be the fixes and enhancements that other programmers would make to the company's product. If those modifications and enhancements are never distributed, the original creator of the source code may never have the opportunity of seeing those changes.

This is a huge flaw with creating web systems that are rarely redistibuted. If my company builds a CMS and makes it open source, everyone can use the product but they are not required to benefit my company in anyway. Any benefit my company would reap would be soley in the kindness of others...

feedclowd is now opensource

2008-04-25T20:17:01Z

I've finally made the application that drives www.feedclowd.com to be open source.

Here's the official announcement

feedclowd now in beta

2008-04-21T17:16:03Z

I just released my project as a public beta, feedclowd It's a life aggregator were you can take all your feeds from all your social sites and combine them into one simple feed.

Your own appengine, maybe not yet

2008-04-08T23:14:28Z

Well, I started looking at both Parallel Python and CouchDB.

CouchDB still seems to be a viable replacement for the BigTable backend to datastore. A GQL parser will have to be written to interface with Couch's views, but that's probably easy enough.

After looking at PP (Parallel Python) a little more. It's definitely not the correct solution. PP only works with a limitted python scope. You couldn't say, load all of Django inside of PP.

The way that appengine works is simple CGI, this makes every request autonomous. It doesn't need to know what machine it's running on.

One possible implementation is to distribute your source code across all nodes. With some kind of virtual hosting router and load balance system, you could make your own appengine on a small scale.

mod_fastcgi on lighttpd allows you to load balance application servers inside of lighttpd. You could distribute fastcgi servers across all 20 app servers all with a pool of min_children/20. So if you need at least 10 children you could have 1 app server per site per machine. I'm not completely sure, but I don't think mod_fastcgi load balancing provides any fault tolerance.

So the idea is to have each http request load balanced to each lighttpd server. each lighttpd server load balances fastcgi among all the nodes. Each fastcgi app server can access the couchdb cloud of all the nodes.

Your own appengine

2008-04-08T18:16:23Z

My main gripe with appengine is that while you're sticking your app on a lot of iron, it's pretty much stuck there forever because of the infrastructure. Your app is not portable. You can't just take it off of Google's iron and host it yourself.

So, when I was listening to part 3 of the google app engine intro, I heard him describe BigTable as a "a distributed, fault-tolerant and schema-free", I knew I heard that before. I heard that from the CouchDB project. I never saw a need for CouchDB, but it's looking interesting now.

After looking at the SDK for appengine, the datastore interface is pretty simple, there's no reason it couldn't be implemented with CouchDB as the backend. Giving you your own "distributed, fault-tolerant and schema-free" datastore.

Also in part 3 that they run your python code on a low overhead, distributed, fault-tolerant infrastructure.. I knew I heard that before too. I heard it with the Parallel Python project.

Need a decentralized, fault-tolerant file system? There's Tahoe. Tahoo provides that too.

So let's say you have 20 servers, a mix match of database, media and http servers. Now instead of partitioning them to have their own roles, they become nodes in a cloud all handling http, appserving, media file serving and data storage.

If you create a webserver inside of parallel python that basically brokers requests for different sites to the parallel python cloud, a node in llpy then executes the request. llpy may contact the couchdb cloud. All using the power of the cloud. Therefore if you have 22 sites and only 5 get heavy traffic, you don't have to waste the power of the other 17 machines on the slow sites.

The appengine SDK looks like a start to create such a system. The datastore modules provide a way to interface CouchDB. Recreating the http brokering system with llpy shouldn't be that hard.

And once the django community finds a way to work it's ORM onto Google's datastore api (which I know they will), implementing Django inside your own parellel python/couchdb appengine wouldn't be hard.

Creating your own appengine wouldn't be for the average joe schmoe, It would be for big companies with some iron.

My site is now on slicehost

2008-04-08T17:01:31Z

I just migrated my blog to slicehost because I finally got fed up with site5's janky django setup.

I noticed that I broke my link to the django aggregrator when I moved my blog off wordpress and onto Django. I tried to log into site5 to fix it and I got that damn "-bash: fork: Resource temporarily unavailable" error that prevents me from doing anything in my shell.

I pulled my stuff off and stuck it on my slice. I may get all kinds of errors, but we'll see. Let me know if DNS is still propagating so you may not see this.

Google Apps Engine without Django colored glasses

2008-04-08T15:22:22Z

So after a day of mulling over GAE, I've come up with one conclusion. For one-off apps, this is brilliant. What Ruby and Rails did in it's heyday, this is going to do. No need to figure out how to set up a web server or configure a database. Just write some python and blamo an application.

This also blows Facebook applications out of the water. With GAE, you get a full web framework, written on top of a real language. You aren't forced to use FBML, you can use Django templates. Tie in opensocial and you have a turn-key application platform.

Now, I'm not sure that this is a viable solution to run a website on (what is a website anymore?), but writing a web application, it's brilliant. Next up, Google Gears for appengine.

Google Apps Engine, wait what?!?

2008-04-08T11:53:11Z

So I saw a post to this last night and I thought, "Oh it's like Amazon's EC2 I won't ever need that".

Much to my surprise this morning, it's something I could really need. It's a sandboxed distributed web framework. No need for load balancing, hardware, whatever.

The first thing I saw was it uses WSGI, my first thought, "Sweet Django can run on this." Which is true... sort-of.

There are few limitations that will drive Djangonistas crazy..

First, there is no local file support. You can't upload arbitrary files to the file system. BUT, you can store them in the datastore, so with Marty Alchin's filesystem backends, someone can write a FileSystemBackend to support storing files into google's datastore

Second, and the big one, Django Model support. At first I thought, "oh, GQL, sweet a SQL-like language, we can just make a DBBackend in django to support it." Nope, for one, it's only supports Selects, and second, no joins. So that's absolutely out of the question.

There are other ways of interfacing Django models to google's datastore. One way is to implement dbapi on top of the google db.models, but that's probably absolutely retarded. We'd have to implement a sql parser and basically our own db engine.

I haven't been following the queryset refactoring stuff, but hopefully it'll make this easier. I'd like to see the Django ORM be less reliant on SQL. I know the R in ORM is Relational, but I would be nice to be able to store and retrieve objects from different storage methods.

I'm surprised that there wasn't more effort made to make the two frameworks work together. From what I see, I can't see a reason to run Django on GAE.

So what does Django have going for it. a ORM, and a Templating System. GAE requires you to use the Datastore API, so no ORM. Templating system? GAE implements Django templates inside of webapp. So no need for the Django Templating system.

Django provides tons of contrib apps, and those are great but how many of them rely on models, probably quite a few.

I don't see a reason why you'd really want to load the complete Django stack if you can't use probably 75% of Django.

I am really surprised that they didn't do more work to integrate GAE with Django. I'm sure Guido was working really closely with GAE and it's not like he doesn't know about Django.

So here are benefits of GAE: First, you don't have to worry about hardware, it's all sitting on google's hardware being load balanced and distributed for you. Don't have to worry about database optimization, again, it's all on google's hardware being load balanced and distributed.

So here are the faults of GAE, First, if you build an application on GAE, it's not portable, you can't take your app, export the data, plug it into Apache via mod_wsgi and go. You're app is completely tied to the App Engine. Second, conspiracy alarms go off, google has all your application's data to do what they want with it.

python interface to the mozilla DOM

2008-02-08T14:39:38Z

Glenn Franxman and I were brain storming on how to do unittesting with a real browser DOM with working javascript. Basically everything accessible to the browser could be accessible to python

I ran across this post: http://ejohn.org/blog/bringing-the-browser-to-the-server/ where he uses Rhino and a custom window library to mimic the interface of the window: http://jqueryjs.googlecode.com/svn/trunk/jquery/build/runtest/env.js

He's using Rhino which is a java engine... It would suprise me if Spidermonkey, which as python bindings, would not work.

Update the env.js library has bindings to java classes so it won't work... I'll have to create an interface to those.

Open Search

2008-02-05T11:28:01Z

With the invention of the opensearch specification, it has become super easy to create search plugins for Firefox 2.0+ and IE7. Here's an example:

<?xml version="1.0"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName></ShortName>
<Description></Description>
<Image height="16" width="16" type="image/x-icon"></Image>
<Url type="text/html" method="get" template=""/>
</OpenSearchDescription>

PreResolution follow up

2008-01-04T11:44:59Z

I've almost made it 4 days without a cigarette. I'm pretty much off the nicotine gum, though I'm still carrying around a few pieces of gum in case of emergency.

I am currently reading "Flatland" by E. A. Abbott, it's an interesting book.

I haven't started exercising or the weekly meditation yet because my body is still recovering from the shock of not smoking anymore. I anticipate starting that next week. I plan on jogging Tuesday and Thursday and meditating on Wednesday. We'll see how it goes, I'm not very eager to jog in the cold air.

Donating Bandwidth to Neighbors

2008-01-02T00:14:57Z

My neighbors across the street don't have Internet access so I volunteered my WPA password to them so that they can use my Internet. Today I decided to install a custom WRT54G firmware called CoovaAP to turn my router into an open hotspot like you would see in hotels for my other neighbors to use.

It was very easy to install and configure. It uses CoovaChilli to create a captive portal to allow my neighbors create a user and log in. It also separates my hardwired network from the wireless hotspot.

I also installed ipp2p to block p2p traffic to keep the bandwidth usage down. Besides everyone uses p2p for piracy anyway (Don't try to deny it). Bittorrent is a great technology used for evil. Don't get me started on Limewire... Here's a howto on installing ipp2p on Coova

New Years preresolutions

2008-01-01T06:16:23Z

Ok, I'm not going call these resolutions, they just happen to be commitments I'm making on the first of the year :)

There are three of them, each corresponding to mind, body, and soul.

Mind

Read one book from the Project Gutenberg a month.

Body

This one is two phased. First I'm quitting smoking, I've already started on the gum. That's just a means to make the second one easier. The second one is to exercise more often. I think I'm going to start by running once a week.

Soul

Meditate at home once a week. I already attend service at the local Buddhist center on Sundays which include a short meditation. There just something about meditating in the privacy of your home that's makes it more fulfilling and intimate.

Ok, that's it. Let's see if I stick to them.

Registration now open

2007-12-30T05:41:01Z

I managed to get django-registration installed now. So you all can now create users.

I'll get comments installed as soon as I get a chance. I wanted to get the registration working first before allowing comment.

eric.themoritzfamily.com is now more Djangolicious

2007-12-28T17:38:47Z

Now Running on Django

Thanks to this post, I was able to get Django running on Site5

Just two days

Thanks to djangoprojects.com's blog application I was able to port my wordpress content to django and create some templates in just two days. How do you like the new look?

django-diario

The next day I found django-diario which is a simple blog application that seems to be actively developed. I plan on following diario's tickets and sending some patches. In fact I already sent one in.

Todo

I'd like to enable freecomments in diario for the time being. I'll be looking into django-registration to add users so that I can do proper comments.

10+2*5 bash script

2007-09-14T01:35:30Z

I've been meaning to post this script for a while. It's a little bash script that I wrote to switch xmms playlists using Merlin Mann's 10+2*5 procrastination hack. It's could well be the greatest bit of code I've ever written.

I've been using it for weeks now, it works for me. It may work for you or it may not. It's real simple, you can probably figure out how to fix it if it doesn't work for you. Post a comment if it's broken on your system.

The script itself: tenplustwotimer.sh

A scripts that runs it in small xterm window: tenplustwowin.sh

My two playlists, one EBM stream (for work), one Jazz stream (for break): tenplustwo_pls.tar.gz

A quick tip, to change the playlist I usually just save over the ten.pls or the two.pls and when the phase changes it'll load the new playlist.

What's great about using playlists instead of urls directly to the stream is I can plug in my GMINI 402 and make two playlists from the mp3s and listen to my own stuff.

Xubuntu Day 3

2007-08-02T19:42:53Z

So it's been three days since I started using XFCE flavored Ubuntu. It's been a pretty lovely existence, but I have a few tips that's I've come across

Notification Daemon

Stock Xubuntu doesn't come with a notification daemon that works with XFCE installed. So those little popups that rise out of the task bar in Gnome don't rise anymore and I end up missing things like new IMs, new mail notifications, etc. Unfortunately, there is not a package in the repositories for the xfce friendly notification daemon so you'll have to compile it yourself. Follow these instructions.

Mail Notifications

Since I'm running Xubuntu in VMWare and I need Outlook running so that Activesync will sync appointments to my Blackjack, It's hard to know when new emails come in. Luckly, our Exchange server here provides IMAP. I could run Evolution and get my mail in XFCE, but why should I have two mail clients running. So what I did was set up two Mail Watcher panel Applets, one for my corporate email and one for my personal Google Apps for Domains account. Unfortunately I was missing emails yesterday because by default, all the Mail Watcher applet does when new mail comes in is change the icon. This is what prompted me to get the notification daemon working. Details on how to tie Mail Watcher and Notifications together are here By the way, I absolutely love Google Apps for Domains, it's brilliant. I wrote a little mailto: handler script for my wife's laptop so that when she clicks on mailto: links anywhere, it opens gmail. I'll provide that script sometime, it's a little 10 line beauty.

Watch out Eric's gone Xubuntu

2007-07-31T02:30:13Z

So I was installing Ubuntu onto my wife's puny laptop (400 mhz, 512 ram) and I thought I'd try something different and install Kubuntu because she complained that she didn't like Linux because it doesn't look like XP. I forgot how under powered her laptop was, Ubuntu ran fine on it before she accidentally reformatted the drive by putting some forgotten Fedora Core CD in it and kept clicking "OK", trying to make it go away, but Kubuntu ran dog slow. It was bad, windows were tearing, drives were swapping, it was a computing nightmare. I started googling, "How to Replace Kubuntu with Ubuntu", and somehow I came across Xubuntu, which I never gave the light of day because regular Ubuntu ran fine, and I was unimpressed whenever I used XFCE. On, this puny 400 mhz laptop, it runs pretty good, much better than Gnome ran. So, when I got in to work today I decided to make the switch to Xubuntu on my PC. So I fired up VMWare (I run my linux install in VMWare because I can't touch the internals of this PC) Ran, "sudo apt-get install xubuntu-desktop", reboot and wala, XFCE in all it's glory! In VMWare, Ubuntu(Gnome) didn't run super fast. It was useable, but switching windows was a little laggy. XFCE, works beautifully (not perfect, but a lot better). I didn't really lose functionality, because a desktop environment is a desktop environment, they all do pretty much the same thing. I'm not sure I'm all that crazy with Thunar and the loss of gnome-vfs, but I can make due. So all and all I think switching to Xubuntu (Zoo-bun-tu?) is a great Idea. Why would I not want run a resource intensive desktop environment when I can have the same experience with a lighter desktop. Duh. Seems like a no brainer... Eric.

The KISS approch to GTD project planning

2007-07-05T19:18:03Z

I've never really got how projects work in GTD. I understand how "Next Actions Lists" worked, but never understood how multi-action projects worked in with that. This is what I came up with it. I'm not sure if this is how it's done in GTD or not.

Let's start with how a project is defined in GTD. A project is a goal that has multiple actions associated to it in order to reach completion. From what I've read of GTD, you keep those actions on a separate list than your "Next Actions" list. That's fine and dandy, but I never did that because it seemed to be just one more list to keep.

So what is this "Next Actions" list that I mentioned? Those familiar with the GTD process know it to be a list of atomic actions. Atomic actions are actions that cannot be broken up into subtasks. Since these actions are the simplest actions you can perform, they are the easiest to complete and it's easy to keep your work flowing smoothly.

This morning I have a project that I need to work on. It's a project that consists of a lot of subtasks of many different types (research, programming, documenting, testing) When looking at this project as a whole, it's overwhelming. I don't know where to start. So I thought, "Let me start with the goal, "Write a PUSH service for a search engine". That is an action that could be simply added to my task list, but it's not atomic. This action has many sub-actions, so therefore it's a project, not a next action. I need to break this up into atomic next actions. Anyone that knows how to factoring prime numbers, the process I used is similar to that.

So I started with the task:

Write a PUSH service for a search engine

Next I thought what actions are needed to accomplish that:

1. Write a PUSH service for a search engine 1.1 Find out how to write a Mason template for the search engine system 1.2 Find out how to make it spit out XML 1.3 Write a template that will output the search results as XML

Ok, actions, 1.1 and 1.2 seem to be atomic to me. There isn't much more complexity to those actions. I could split it up further into, "1.1.1 Google for docs about Mason, 1.1.2 Read docs"... but let's not get obsessive now. There has to be some limit to factoring an action.

So 1.1 and 1.2 are atomic, groovy. Let's look at 1.3. I could say that 1.3 is atomic, it seems simple enough, but it really isn't. There are at least two actions I need to do.

1.3 Write a template that will output the search results as XML 1.3.1 Define the XML format that will be generated 1.3.2 Write Template

There might be more that that, but I'll revisit 1.3 when I get to it.

How does the "Next Action" list come into play? The Next Action list acts as a filter to keep only one action of a project in view at a time. So what I do is put, 1.1 on my next action list, when it's completed I put 1.2 on there, then 1.3.1 (not 1.3 because it's not atomic), then 1.3.2. I never have more than one atomic action from a project list on the next action list at a time.

Having only one item from the project list keeps my mind focused on the action at hand and not the whole project. I'm not tempted to look down the list and imagine the mammoth project that the ten actions scattered among the other actions represents. That's the point of GTD, keep the mind focused on the simplest thing and at a time when the mind should be focused on it.

The other advantage of not having all a project's actions on my next action list is my next action list isn't bloated by things I can not take care of before the action before them is completed. The next action list has to be a list of actions that any could be completed at this moment. No action can be dependent on another action.

Review:

1. Determine a goal, factor that goal into atomic actions. This is your project list. 2. Place only one action from the project list on your next action list 3. No action on your next action list can be dependent on another action

Final tip for project lists: Keep your project list simple as well. If your finding that your project has more that three levels, your project goal may be too lofty. Split the first level into sub-projects. You should never have to look at a single piece of paper and feel the world collapse, something is wrong, simplify.

Satechi's Bluetooth A2DP/Handsfree FM Carkit

2007-06-19T20:29:07Z

I recently acquired a Satechi's A2DP/Handsfree FM Carkit. What it does is provide both a A2DP Bluetooth profile and a Handsfree profile and transmits the bluetooth audio to your radio via FM. This was the holy grail I've been looking for, streaming shoutcast to my car stereo.

I don't have 3G in Ft Myers, but in Miami this weekend I had all that 3G goodness to my disposal. I installed TCPMP on my Samsung Blackjack and fired up some EBM shoutcast streams like Bioindustral or Rant Radio and was cruising down`Biscayne Blvd`_ listening to some crazy EBM/Sythpop, all streaming through the Internet on my phone!

I personally love that kind of music but I'm not going to devote a library of MP3s to it. The music is so different from my normal listening habits that putting my PMP on shuffle with EBM mixed in with "The Agency" or "Hot Water Music", just wouldn't go. So I usually get my EBM fix via shoutcast streams (It's great coding music).

So the A2DP functionality is absolutely great. No wires, nice crisp stereo sound. It's just great. The hands free... It works but, not great. I have to turn down my radio to avoid echoing, but it works well enough to: pause my music, take a call and tell them "I'm driving, I'll call back" and start my music up again (that process is automatic on my Blackjack by the way, just press the answer button). Frankly I don't want to talk when I'm listening to some good music, but that's just me.

The overall experience with this product is that it makes a great A2DP to FM transmitter but a lousy hands free device. Fortunately I wanted it for the A2DP and it works great, so I'm not that bummed about it. If you want to send music from your phone to your stereo wirelessly, then this is the one and only device for you.

How Beryl/XGL and Deskbar simplified my life

2006-10-25T07:55:35Z

|Screenshot of my simple gnome| Screenshot of my current desktop

Yesterday I was playing with Gnome's basic layout and I decided because Deskbar was so good at locating what I wanted that it was quicker to use Deskbar than the standard Gnome application list. I decided to take things farther and remove some key GUI standards from my screen because I found that using alternative methods of accomplishing the same thing were quicker.

Removed the Application Menu

First I removed the application menu in favor of using Deskbar. It's much quicker to hit the Deskbar keyboard shortcut and type out "web" or "mail" to launch Firefox or Evolution. The application menu is still accessible by pressing Alt-F1. The benefit of ALT-F1 is that it opens the menu where the cursor is. This reduces the amount of movement the mouse has to do to accomplish the same thing. I'm also able to use the arrow keys to choose an item in the application menu

Replaced the Run Dialog with Deskbar by Mapping it to Alt-F2

Normally the run dialog is mapped to the shortcut Alt-F2. Because Deskbar includes the ability to run an application by the executable name, there was a overlap of functionality between the run dialog and Deskbar. Replacing the run dialog with Deskbar does not remove any functionality. In fact Deskbar gives me more functionality.

Removed the Standard Gnome Window List

|Example of the Scaling Plugin| Scaling Plugin

I removed the standard Gnome Window List in favor of Beryl's Scale plugin. Beryl's Scale functionality resembles OSX's Expose functionality. Because the Scale plugin displays all the windows currently open and unminimized at a keypress, I can quickly find the window I want visually and select it either with the mouse or with the arrow keys.

The one flaw I had always found with the window list is once you get more than five windows open, you can no longer see the window title and easily identify the window you want. What also tends to happen is when I have multiple windows open, the only thing I can see is the logo of the window and no more. This makes it extremely difficult to find a window.

Another problem is when I have multiple Gnome Terminal windows open. The title bar with Gnome Terminal is usually the current directory I am working in. Trying to find the window by the current working directory is not enough to determine the correct window and I end up clicking a number terminals before finding the correct one. Being able to see all the windows currently open, I can see the terminal that resembles the shell I'm looking for.

Change My Perception of "Minimize"

Because I no longer have a window list in my panel, I can no longer easily find windows that I've minimized. The solution I found to alleviate this problem is to change my perception of what minimize is used for.

Instead of using minimize to put the window in the window list but not on my screen I now think of it as hiding the window. To find the window, I added the window selector applet to my main panel. That way I can find the window I hid.

Fitts' law

Fitts law is a well know model in ergonomics and user interfaces. Granted I'm not a usability expert by no means but I doubt you have to be one to understand Fitts. This is an over simplification of the law, but I'll try to explain it. When it applies to the mouse cursor, Fitts' law says that the time to hit a target is related to the target's size and the distance from the starting point.

For instance moving the mouse from the center of the screen to a target a hundred pixels to the right takes a certain amount of time. The size of the target also plays a part because a large target is easier to put the cursor on than a small target. With a small target you have to be careful not to overshoot the target and therefore you have to slow down when you get near the target.

Another principle of Fitts' Law is if the target is in a location where you cannot overshoot, you do not have to slow down to hit it regardless of it's size. The target is thought to have an infinite size and therefore it is easier and quicker to hit the bottom of the screen than it is to hit a 64x64 size icon sitting 10 pixels from the bottom of the screen. This principle is a key reason for Apple putting the menus for applications on the top of the screen as opposed to within the window like Windows/Gnome/KDE does.

The key way to understand the application of Fitts law is to understand that it only applies to one dimension at time. The distance is either horizontal or vertical. Even though you may move the mouse diagonal to reach a target there is actually two distances in play. You are both moving the mouse horizontally and vertically when you move it diagonally.

How Fitts' Law is used on my desktop

Though you can't see most of the applications of Fitts in the my screenshot, it is used in a few places:

The window selector panel applet that I use to locate hidden (minimized) windows is located in the topp left. This gives window selector icon infinite width and height because it's in the corner. If I'm too low and continue to move the mouse cursor to the left past the left edge of the screen the cursor will slide upwards into the corner.
The Scale plugin is activated by simply moving the cursor to anywhere on bottom edge of the screen. This gives that target an infinite height because the target is on the bottom and an infinite width because it doesn't matter where on the bottom I hit. All I have to do is fling the cursor to the bottom of the screen and I see all my visible windows.
Deskbar is on the top center of the screen. While I generally use the shortcut to access Deskbar. Putting it on the top of the screen gives it an infinite height. It is also in the center of the panel, although this does not give it a infinate width, it substantially increasses the ease of hitting the target. Having the deskbar applet in the center makes the max distance horizontally no more than half the screen minus the width of the Destbar applet. If the deskbar was on the left or right side of the panel the max distance horizontally would be the width of the screen minus the width of the deskbar applet. I also made the width of the deskbar applet nearly 50% of the screen. 90% of the time the cursor would be between the second quarter and fourth quarter the of the screen. So the max distance horizontally is essentially zero because more often than not I would not have to move the cursor horizontally to get to the deskbar. Moments when my cursor is within the 1st quarter or the 4th quarter of the screen, the max distance my mouse would have to travel would be a quarter of the screen.
The Scaling plugin has the ability to display only windows from a single application, all applications on the current virtual desktop, and all windows. I've mapped the "Show all windows of a single application" function to the top right of the screen so that it has infinite width and height. This is good when I have multiple Firefox windows open.

The Fitts Optimized Targets All Have Keyboards Shortcuts to reduce hand movement ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I am a Dvorak keyboard user and a huge fan of Ratpoison. The main goal of both the Dvorak layout and Ratpoison is to reduce the movement of your hands. It takes a lot of time, both physically and mentally to switch from keyboard mode to mouse mode and vice-versa.

While I probably use the keyboard more than the average joe, it's no secret that we live in a point and click world. It's common for me to be surfing the web and need to switch to another window. Hence the Fitts optimized trigger for the scale plugin. It's also not uncommon for me to be hacking away in Emacs and need to switch to another window. It would be a waste of energy to move my hand from the keyboard, grab the mouse and fling it to the bottom to see all the windows.

To minimize the movement of my hand from the keyboard to the mouse and back again, all areas of the screen that were chosen because they are Fitts optimized to reduce mouse movement have a keyboard shortcut to reduce hand movement.

If I decide I want to choose another window while my hand is on the mouse, I simply fling the cursor to the bottom and select the window with the mouse cursor. If I have my fingers on the keyboard, I simple hit F8 and use the arrow keys to select the window I want.

Decreased the importance of the Desktop

Normally the Nautilus file browser is the central application when interacting with the computer. I deactivate the icons on the desktop with gconf . There are three reasons for me to do this. The first reason is I wanted to use Nautilus like any other application. The second reason is the desktop usually gets cluttered with files, the third reason is that there is an easier way of accessing your files than desktop icons.

To prevent the clutter, I took a cue from Windows and OSX and created a number of folders in my Desktop folder:

Applications, these are user installed applications that would normally go in opt if installed system wide. Applications like Songbird, Opera, Limewire go in here.
Reference, the idea for this folder was taken from "Getting Things Done". It holds all kinds of reference information, generally saved documents that will need to be referenced later. I avoided calling it the vague term, "Documents", because a document could be anything. The name "Reference" defines the purpose of these files.
Media, obviously media files. This has subfolders to keep this folder from getting cluttered
Images
1. Photos, actual photos taken from my camera
2. Illustrations, Vector based images
3. Backgrounds, I seperate backgrounds from photos or illustrations because backgrounds are generally novalties. The name "Backgrounds" also describes the purpose of these files.
4. Novalites, Images that completely pointless but funny
Videos
1. Movies, Full length movies that I've ripped from DVDs
2. TV Episodes, TV episodes that I've downloaded
3. Novalties, Novalties items like silly you tube videos of people falling over or cats attacking cealing fans. Basically mental junk food.

With these folders I've created bookmarks in Nautilus so that I can easily access these folders from, the nautilus side bar, choose/save file dialog or deskbar's "File and File Bookmarks" plugin.

The reason why this is easier use than using the desktop for storing files is that it's quicker to get to the files you need using deskbar.

The normal way to access the desktop is that you need to either use the show desktop icon, or use the beryl show desktop plugin. Once you get to the desktop, if you keep all your stuff on the desktop, you're done, but you have a cluttered desktop.

If you keep all your stuff in sub folders of the desktop, you need first locate the subfolder on the desktop, then locate the files in that folder. It's a multistep process.

Using Nautilus bookmarks and the "Find File and File Bookmarks" deskbar applet is that to get to the subfolder that's on the desktop, it's a two step process: Click on deskbar and type subfolder name. The normal way would be: Show the desktop, locate the folder, open the folder. It may seem like a one extra step, but the time it takes to locate the folder takes longer to do than typing in the name. You could say that the old method is two steps more because typing a name is so much quicker than scanning the desktop for an icon

Limitations

Although I think that this setup is great, and it's optimized allow me to do many of my common tasks with very little effort and energy, there are a few flaws. This set up is also only a day old and I'm sure I'll find others. I'll append this section with any that I find.

Once the Gnome Volume Manager (the thing that monitors the system for removable media) mounts a drive, I can not unmount the drive from Nautilus. The only way that Gnome made it possible to unmount removable media is by the desktop icon. I'm sure that there is a way to umount a drive with some sort of panel applet but for the moment, I simply run "pumount [mount point]" in the terminal.
When deskbar hangs, I can no longer launch anything. The solution I've found is to hit ALT-F1, open up the application menu, open terminal, and kill deskbar.

Open Source Dental Software!

2006-08-26T16:01:31Z

I went to the dentist the other day and I noticed the app running on their computer had the title, "Open Dental". I thought, as I sat in the dental chair. Is he using an open source dental app? Sure enough, he was. read more | digg story

coastalbeat.com woot woot!

2006-08-26T13:59:54Z

So the past three days have been a blur. I got into work on Wednesday with forty-eight hours of work and twenty-four hours to do it. I was not letting the new site launch without the social app I've been writting for the past week or two. After staying up twenty four hours I finally finished the app and the site went live to the public. Check it out: http://www.coastalbeat.com/

I am very proud of this site. Finally something to work on that I can really feel good about being involved with. Todd's design is awesome,Â Ira and Tovin really busted their asses to get coastalbeat.com on the stands and get content online.Â Levi put in some ungodly hours (I think he has 300 skill points in endurance) to get it up.Â Props go out to all those the helped out that I don't know about.Â Â I'm really proud of the work we all did! Check out my profile, I'm costalbeat's Tom! I think there's talks of "Eric is NOT my friend" and "Eric is my only friend" t-shirts.

RSS increases boredom.

2006-06-23T10:00:08Z

Remember back in the day when you could waste hours clicking on the net finding information.Â It was fun, you felt like you were "surfing" the net.Â Â I just finished reading my feeds on bloglines, going through the links on digg.com, I even checked out slashdot to see artcles that I read yesturday.Â Then I thought, now what.Â I don't know what to do next.

With RSS, everything gets aggregated into a personality-less list of articles.Â After I go through all the artcles, I find myself wondering what to do next.Â Wtih all this new "web 2.0" technology, are we losing the personality of the net?

Test Tube Meat Nears Dinner Table

2006-06-23T08:47:20Z

If they can produce meat in vitro? For vegitarians like myself that don't eat meat because animals have to die, would I eat it? I don't know.

It's an interesting quandery. I don't eat meat because mainly because of the Buddhist belief that if we have had an infinate number of rebirths then every being has once been our mother and that they are still our mothers. Chewing on the flesh of my mother sounds absolutely discusting to me.

So if muscle tissue can be produced in a lab, it's not technically flesh and definately not the flesh of another living being. Is it any less discusting? I doubt it.

I do eat veggie burgers which resemble meat, so I don't know what I would think of eating something that is essentially flesh.

I'll just have to wait and form an oppinion when this stuff is in the market. read more | digg story

URGE and iTunes

2006-06-20T09:30:18Z

I got a chance to take a look at Vista Beta 2. While I'm not normally a guy that likes MS products. At first glance, Vista looks good. I could never forget what my wife said when I showed it to here, "That looks like Linux". What?

Anyhow, this post is not about Vista, but a service that comes bundled with Vista called, URGE. URGE is Microsoft's and MTV's joint venture into the online music sales market. There is one good thing with the service and a few flaws I see with the service.

One of the good things with the service is that you can pay a flat rate every month, $14.95 I believe, and download as much as you want. That sounds like a deal to me. I see many music pirates going legit with such a plan. I just hope they don't screw the pooch on this one and make the songs expire after a month.

And now for the problems I see. URGE is bundled with Vista. This technique has proven a dangerous practice with Microsoft **cough IE **. What I see happening is other music services will cry and say that because URGE is preinstalled, it's hard to get customers.

The second problem is that URGE doesn't work with iPods. It uses MS' Play's for Sure DRM with works with just about every portable music player I've seen except the iPod. I don't know if this was a great plan on Microsoft's part to get all those pmp's out there to gang up on the iPod, but all those iPod owners are gonna see that the iPod restricts them to giving Apple money via the iTunes music store and that they have no freedom of choice (i.e. Apples' Monopoly on iPod's money).

While I don't think the second flaw is a flaw with URGE, but more a revelation of Apple locking iPod owners to iTunes when legally purchasing music. There will be two outcomes to this revelation. Apple will open on their AAC DRM to others, so that other services can provide music for the iPod. The other outcome will be that iPod's will provide support for MS' Play's for Sure DRM. Both outcomes means a loss of control for Apple.

We'll see what happens, until the war of the DRM's begins, iPod owners will be left in out of URGE's 14.95/mo unlimited downloads and will either pay Mr. Jobs for their music or continue pirating.

A cool google map/wiki mash up

2006-05-29T04:55:31Z

I found this the other day, it's pretty cool. You can create a personal map by clicking on the map and type in a description. read more | digg story

Say F.U. to linux passwords

2006-05-16T02:19:19Z

I bought a 128 meg usb drive from radio shack because it was on clearance and fit on my keychain. I was trying to find reasons to use it and I found, pam_usb. PAM_USB allows you to store a dsa key on your usb drive that, when plugged in (it doesn't have to be mounted, just plugged in). Despite a scary description on the page, it was pretty easy to install. read more | digg story

Hong Kong's Symphony of Light (with video)

2006-05-01T21:25:31Z

Honk Kong lights up the skyline every night with this spectacular show of syncronized lasers, leds and fireworks. Makes Epcot look lame. read more | digg story

Budgeting

2006-02-07T22:42:58Z

When I was younger budgeting was easier. The reason it was easier because I didn't use a bank account and I would simply put money in designated envelopes. Life was simple.

Now that I'm a responsible adult, I have a bank account and I rarely have cash in my hands. The envelope idea doesn't work.

Last night on my drive home from work I was day dreaming about how I could budget like I used to in this age of bits and bytes. Budgeting on a computer doesn't give that instant response of "Oh crap, I don't have any cash for clothes left" as an empty envelope.

The idea I came up with was using poker chips to represent the money allocated for a particular catagory.

I'm still pondering how I would want to store the chips so that it's obvious how much is left. I want to easily gaze at each "envelope" and see how much is left so that I can start to curb my spending when the budget is low.

I saw online that there are clear holders for chips that I would be able to easily stack the chips in columns for each category. They usually have 5 column (one for each chip color) but I think I might have more categories than that so that may not be good. Using the holder also means having to either buying the holder online or searching stores for them.

What I came up with for now is using ziplock bags instead of envilopes. They're easy to aquire and they're clear. We'll see how well it works.

I haven't implemented my idea as of yet, I'll let everyone know how well it works.

Oh yeah. GMINI 402 baby.

2006-01-14T09:21:25Z

My wife got me a GMINI 402 for Christmas. I wanted to wait a few weeks before actually writing about my experience with it. My first expression when I opened the box was that it was smaller than I thought it would be. It is a very nice size. The case of the device is metal which makes it feel really solid.

Audio

What is there to say, it's a mp3 player. It plays mp3s. I like the ability to create playlists on the device. I can browse by artist and press add and it adds all that artist's songs to the playlist.

Video

This is were the device shines. I thought before I bought the device that I would have to transcode all my videos to mp4's but after a few weeks of use, I've noticed that I usually can just download the video and dump it on the device as long as it's an avi encoded as divx with mp3 audio which is proving to be the most common format I've found. The only time I have to reencode the video is if it's a mp4 quicktime file (which is what ipods/psps play). Being able to download and drop the file into the device is really nice.

So far, I'm really impressed with the device.

New Position at Naples Daily News

2005-11-29T09:05:50Z

One of the most respected and award-winning newspaper Web teams in the world has moved to Florida and is looking for an experienced server-side Web developer.

NDN Productions -- the online and new media publishing division of the Naples Daily News -- is looking for a full-time Python programmer to develop Internet-based applications that cross from computers to mobile phones to iPods to Sony PSPs.

We strive for innovation and nimble development for sites that embrace relational databases in ways they've rarely been used on the local level coupled with broadband-centric multimedia content that all works together in a platform-independent manner.

In other words, we strive to build the kinds of Web sites and related offshoots that you wish were in your hometown.

We're big believers and contributors to the open-source community. Our primary development platform is Python (mod_python) and PostgreSQL, with a particular emphasis on using the Django infrastructure.

That being said, we believe a solid background in Web application design is more important than knowledge of a particular language or platform. If you're a smart cookie and it shows, we know that you'll have no problems picking up the tools we use.

We embrace a highly creative, non-traditional work environment. We love to work fast and have fun, with the time between having a great idea and that idea being added to one of our sites being measured in days, if not hours.

Our salary is competitive, though not crazy huge, and our benefits package is sweet.

And because our agile and autonomous online team works under the umbrella of one of the largest and most stabile diversified media companies in the nation, you don't have to worry about your checks ever bouncing. If you work with us, your mom and dad will be proud, and your friends will be envious.

So, if you learn things quickly, want to hang out with cool people and build amazing Web content, than this is the place for you.

We're an equal-opportunity employer. In fact, we even hire folks from Missouri.

So, contact Rob Curley at NDN Productions right now before one of your buddies beats you to this great gig.

Rob Curley Director of New Media NDN Productions/The Naples Daily News E >> nerds@bonitanews.com P >> 239.435.347

Hey! You got some emo in my hair-band.

2005-11-24T01:12:53Z

I saw this band on MTV the other day, Coheed and Cambria. I'm really digging their sound. It's like Rush if they decided to dive into 80's hair metal. Many describe them as Emo Progressive rock. The lead singer looks/sounds like the illegitimate son of the mother of the guy from Counting Crows and the singer from Rush. Check them out.

We need a Django Developer

2005-11-09T02:42:35Z

There is now an opening at Naples Daily News for a new developer. If anyone doesn't know what we're doing, let me fill you in.

If you do a search for "Rob Curley" in Google you will notice that it'll take 38 pages of Google before people stop talking about him. He previously worked for World Online and helped propel them into New Media stardom.

Rob left Lawrence and has started making waves at Naples Daily News. It is definitely the place to be if you want to win awards and become famous.

Poynter, which is the one of most prestigious journalism think tanks in the country, this past month sent journalists to follow our work during Hurricane Wilma because they wanted to see how we worked. CBS also sent their reporters to Naples to cover us for the Early Show. We have just started and we already have the world's attention.

I hope I'm not sounding pretentious but I'm very excited about the work we've done and what we are going to do in the coming months. If there is anyone out there that is interested in programming in Django full time with one of the best New Media teams around, please let me know.

We survived.

2005-11-06T20:03:53Z

We finally got the our electric back since Hurricane Wilma. I just wanted to let everyone know that we survived and we are safe. We evaculated to Fort Lauderdale thinking that we'd be safer by putting 100 miles of land between us and were the hurricane makes landfall. If anyone was watching this monster of a hurricane, Wilma was perdicted to pass her eye directly over Naples, Fl. So prior to the Wilma's attack on Florida, we thought that getting the hell out of dodge would be wise.

Unfortunately, Fort Lauderdale was not ready for being raped by the hurricane. We ended up getting stranded in Fort Lauderdale because no gas stations were opened after the storm. Slowly stations opened but with millions without power and millions using generators to watch tv and have lights, chaos insued. There were/are miles of lines up to the few stations opened. People waited in lines for hours. Some people were unfortunate enough to wait in a line at a gas station that was never going to open and they waited all day for gas and never got it. It was insane. As far as I know, this is still happening.

Because of the gas shortage, we were stranded in Fort Lauderdale. Our friend owns a pizzaria and because they have a gas pizza oven they were still being able to make pizza. So being stranded in Fort Lauderdale wasn't all bad, we were getting hot meals, can't complain there. Someone who came in for a hot pizza told he about how he drove to Fort Myers (a little north of Naples) and filled up his truck and all his gas cans and drove back.

So, our friend came back from the restuarant and told us his master plan of driving to Naples and grabbing as much gas as we could. By this time, Naples was basically up and running aside from some residencial power outages. Our car was on E so without gas, we were stranded in Fort Lauderdale.

Driving an hour and a half to Naples was much better than waiting in line four hours for a chance to get gas, no guarntee at getting gas, just for the chance the get gas. So my wife, our friend and I packed up the baby, hopped in her car and drove to Naples.

When we arrived in Naples, we gased up the car and searched for gas cans. We had one flaw in our plan. We expected Naples to have gas cans available to bring back to Fort Lauderdale to fill up our dry car. With tens of thousands of residents without power, people needed gas cans to keep their generators running. We drove to Super Walmart... No cans.. We drove to Home Depot... No cans... We drove to Target... No cans...

We drove to Lowes and finally we found six gas cans. We filled up the cans and drove back to Fort Lauderdale. That night, we filled up our car and drove home. It was quite an adventure.

Overwelmed by Web 2.0

2005-10-16T05:05:34Z

Prior to moving to Naples and taking the job at Naples Daily News, I was kinda resisting the suite of "Web 2.0" technologies, Ajax, RSS, Podcasting, etc.

Sure I tried RSS, and I thought it was cool but I guess I never had a decent client to collect the feeds. I thought it was a little involved to collect the RSS sources. Firefox made it easy with their fandangled little rss icon but Live bookmarks never seemed like the right interface for RSS. I tried out Thunderbird's RSS feature. Again, a little involved to find the feeds. I noticed that a coworker, Levi, used Bloglines, and that was nice. Then I found litefeeds which I really like primarily for their cell phone interface. I'm starting to get a little overwelmed by the shear volume of available RSS feeds. Which is good? So many, I hardly have the time to find those I like...

On Friday I upgraded Ubuntu from Hoary Hedgehog to Breezy Badger... Ok, oh my god, the best Linux distro eva'. Try it out, there's LiveCD's, no commitment needed.This isn't a post about Ubuntu, so let me get back te the current topic. With the upgrade came the latest version of Amarok. No, this isn't that annoying reality tv diva from The Apprentise. This has to be the best audio player for Linux. If you're using Linux, install this app. Even is you love Gnome and hate KDE, install this app.

So, anyhow, with the newest version of Amarok, you get podcast support. Another one of those "Web 2.0" technologies I never got into. So today I got my first taste of podcasts. It was fun, it tasted good. I downloaded our new Bonita News.com podcast and got to listen to Crunk Master Tim sound all professional. I signed up to a few Buddhist themed podcasts, the "Rock and Roll Geek" show, the "Keith and the Girl" podcast, so many podcasts... Seems like anyone with a microphone has a podcast show. Who has the time to listen to all those podcasts to become a fan of a few.

If anyone is actually reading this thing, please post a few comments about your favorite podcasts, blogs, rss feeds, etc. You'd make my "Web 2.0 life" easier.

Old and Busted: Bloglines, New Hotness: litefeeds

2005-10-09T00:08:45Z

Ever since I signed up for bloglines I kept thinking, "Man, RSS feeds are perfect for cell-phones". I flirted with the idea of creating a RSS aggregator that outputted the content in xhtml-mp for my mobile browser. Then I found litefeeds and life is good.

Litefeeds just like bloglines it's a web based RSS reader, but it has one feature that Bloglines doesn't have, a j2me app for your cellphone. What does that me to you? It means that you can get your feeds on your java enabled phone. That's awesome.

Other features: The ability to view images from users of flikr on your phone, import from opml and therefore import from bloglines.

Another cool feature, bloglines has this but it's infinately better on your phone, it's the "clip it" function. What it does is save the article into an online clipboard so that you can read it later. You can do this on your phone, it's a minute detail that I'm very glad they included it.

Check it, let's say Life Hacker, which showcases useful apps and sites, has an article reviewing an awesome application. I'm standing in line at the DMV, I can't check out the application right at that moment, but the article convinced me that this is the killer app and I need to check it out. So, I clip it. When I get back home from DMV hell, I can open up my laptop, bring up the site and get hip to that new app. Awesome or what?

There are a few features that I would add if I was in charge. One would be displaying of images via an RSS' encloser tag so that I can get Dilbert's feeds on my phone. If they can display flikr images on the phone, there's no reason why we couldn't get that to work. Another feature would be a "Add to litefeeds" type button for our blogs, MyYahoo has it, Bloglines has it, no reason litefeeds shouldn't. A bookmarklet would be nice too.

Enjoy.

OK/Cancel

2005-10-08T19:13:40Z

I came across this really awesome comic strip for anyone who's in the website making business. Ok/Cancel It's funny because it's true.

Social Bookmarking

2005-10-08T12:26:37Z

So, I finally joined the social bookmarking craze. I've used Simpy before but I had some trouble with it. I finally got around to using del.icio.us and so far I like it. I even added a Word Press plugin to list my 10 most recent links. Enjoy.

Been busy

2005-09-26T01:17:54Z

Phew,... I've been busy, I've finally gotten a breather to update my blog. Two weeks ago we launched`http://www.bonitanews.com`_. It is the first Ellington/Django site launched outside of World Online. We're very proud of it.

Something else to be proud of, my son, Aiden Jacob, was born on Sept. 8th. He is wonderfully healthy.

Life is good here in Naples, Florida. I think we've finally acclimated to the Neapolitan lifestyle.

Be well.

All Hail the Flying Spaghetti Monster!

2005-08-24T21:07:50Z

Read this

Naples Mindfulness Center

2005-08-22T19:00:00Z

Last night I went to my first Buddhist meditation class in Naples. Anyone who has known me for an extended period of time or has browsed my bookself knows that I am a practicing Buddhist. I've been studying Buddhism for almost four years now. When I decided to take the job at Naples Daily News, I looked into Buddhist groups in Naples and I was saddened to find out that there is very little reputable Buddhist organizations in Naples and the west coast in general.

I found The Naples Community of Mindfulness during my search. They are students of Thich Nhat Hanh, who is a well respected and accomplished Vietnamese Zen Master. Anyone who has browsed Barnes and Nobels' Buddhist section has heard of Thich Nhat Hanh.

The teacher, Fred Eppsteiner, has studied under both Thich Nhat Hanh and Tibetan teachers, which is very comforting to me because I've studied under a Tibetan lineage and I'm completely oblivious of Zen's practices. A Zen/Tibetan teacher would make the transition very nice. He is also a psychotherapist, which will make things very interesting. I like the fact that he has wisdom from both Eastern and Western mind sciences. I'm sure he has much to offer. Unfortunately for me, he is on vacation until the end of September.

Last night, I attended their sunday night meditation class, the class was very nice. The meditation room is styled in the minimalistic Zen style therefore it wasn't as colorful as a Tibetan Gom-pa. I was happy to see a Tibetan thangka of Manjushri, who I've always had an affinity for. The people were very nice and welcoming. I was honestly afraid that I would find to many Gucci-Buddhists in Naples but the people at this class were very genuine praticioners. I'm very excited to experience this new aspect of Buddhism.

National Weather Service XML Feed

2005-08-19T19:33:51Z

The national weather service now provides a Current Weather Conditions xml feed. It's really cool.

Back to Corporate

2005-08-12T18:22:21Z

To Celebrate my re-enter into Corporate life, I added a link category called "Corporate Fun". This is my request, comment to this posting with all your favorite Corp links, jokes, or what have you.

Undocumented Django Wiki

2005-08-11T01:41:31Z

On Monday night before I left for Naples, I installed a Media Wiki for Django. You can find it located here. Feel free to contribute if you have something cool to show everyone.

First Day at Naples Daily News

2005-08-10T10:59:55Z

I had to return to Naples Daily News for a meeting with the Corporate folks. I woke up in the morning to find that my wife didn't pack my Khakis. So I was a little annoyed because I would be under dressed. My wife drove me to the NDN IT building and while I was waiting for Rob Curley (my boss) to arrive, I kept noticing that I was the only one wearing jeans. I felt like it was my first day of high school, checking out what everyone is wearing.

The meeting went well. I'm very excited about what is going to be happening with Django and Ellington. What we are going to be doing with the software is really going to show off the two systems.

I had the opportunity to meet Jacob Kaplan-Moss, who is one of the developers working on the Django project at World Online. He's a really cool guy. He gave me an overview of their Ellington system, which blew me away. The simplistic structure of this large system just showed me why I knew that Django was an amazing framework.

Seeing Ellington in action proved to me that I have some big shoes to fill. Well, it's a good thing I have large feet. Seriously, I'm confidently that we're going to whoop some major ass with this stuff. I wish I could go into more detail about what our intentions are but I'm not sure what I'm allowed to say.

Left Handers day

2005-08-05T00:41:20Z

I'm moving on Aug 13th which is International Left Hander's day. Being that I'm left handed, that must make it an auspicious day to move. So lefties, celebrate International Left Hander's day by lending a hand and helping me move :)

The Django Virus

2005-08-03T22:47:02Z

Well it looks like Django is going to be running in some high profile places. With Adrian's announcement that he's going to be working for the Washington Post soon and of course my announcement of me moving to Naples to work for Naples Daily News and therefore Naples Daily New's parent, media giant Scripps. Django is going to have some really impressive things to put on it's resume (take that Ruby on Rails j/k).

I'm moving to Naples, Fl...

2005-08-03T02:22:25Z

I am sooo moving to Naples, it's so beautiful there. No really, I'm moving to Naples August 13th.

I was in Naples this weekend interviewing for Naples Daily News. For anyone that knows the history of Django, the code was developed at World Online. The man that was in charge at World Online was a fellow named, Rob Curley. Rob then came down to Naples with the intent to kick ass, and that we will.

I'm the newest member of the Naple Daily News. My wife and I are moving to a beautiful apartment. The apartment complex' grounds looks like a park. There is a huge lake, tons of trees, two tennis courts, two pools, a fitness center and this is the best, access by water to the bay. I'm thinking about getting a couple kayaks so that I can drop then in the river from my apartment. That sounds like lots of fun.

Ok, so I have a new job working with Django, the greatest thing for web development since slice bread. Django is written in Python, which is the greatest thing for programming since slice bread. I'm working for a New Media celebrity. I'm working with some of the best in the field. I'm going to be living in a city were the local section of the newspaper has to outsource to report about crime. I'm going to be living in an apartment complex that is like looks like a park.

Someone, please do not pinch me.

Greetings from Naples,Fl

2005-07-30T12:06:21Z

My wife and I are now in Naples, Fl. We are staying in a very nice hotel. It's a Comfort Inn, but like everything I've seen of Naples, it's very beautiful. It's the nicest Comfort Inn I've ever stayed in. I took a few photos of the hotel room, I'll post those tommorrow. I want to photograph the view from the window in the daylight. There are some photos on our family blog, http://www.themoritzfamily.com

Django Ajax and Dojo?

2005-07-29T22:21:40Z

The other day in the #django channel I was msg'd by Alex, a developer for the Dojo Javascript toolkit. (Have anyone noticed that javascript toolkits are popping up like crazy?). Well, Alex asked my, "Why are you using prototype.js for Django Ajax". I told him that I used it for some other projects and it's simple. He told my to take a look at his project.

At first glance I thought, "Man, Dojo is much more than what Django Ajax needs." It looked over complicated and I just didn't want to get into figuring out how this thing works. (It was probably late at night). All I thought that Django Ajax needed was a simple ajax library. I told Alex, "Dojo looks like overkill for my needs." He responded by saying, "This maybe true, but people are going to want this." I thought he was just trying to push his project. Man was I wrong...

Today I took a second look at Dojo because MochiKit was released and it had a reference to Dojo on their site. The other day I looked at this page, Fast Widget Authoring, but I didn't really read it because it didn't think it pertained to DjAj's needs. Today I actually took the time to the page and... Damn... This is cool shit!. The first thing I thought was, "This looks a lot like how Django does things...". Dojo IS for Javascript, what Django is for Python web development. This is going to be big.

If you read the article, you will see what Dojo does. Basically it allows you to create packaged javascript components using html and css. No more, innerHTML-String Splicing crap, no more DOM appendChild, blah, blah, blah, well not as much anyhow.

I'm going to be playing with Dojo a little and most likely integrate Django Ajax with it. Happy Hacking...

For the Gamer Dorks out there...

2005-07-29T00:39:53Z

http://www.vgcats.com/

Django, Dreamhost and FCGI

2005-07-28T18:58:45Z

There have been quite a few people in #django trying to get Django to work with Dreamhost. Thanks to the work that Hugo has done and the folks in the channel hacking away at the problem, the work has been posted on the Dreamhost wiki. Maybe now I can get Site5 to get django up and running on my server.

Stupid Windows Keymap

2005-07-27T23:33:38Z

I've been using the standard windows keyboard layout switcher to remap my keyboard to dvorak. Unfortunately when I log into a machine through remote desktop, the keymap of the machine is used and not my local keymap settings. This is really dumb because I annoy all my non-djorak coworkers because when they log into the same machine, I usually forget to switch the mapping back to qwerty and they get aoeu instead of asdf...

I used a program called keytweak to remap my caplocks key to control, which is very helpful because I use control much more than I use capslock. I thought, "Hey, I can use keytweak to avoid annoying my coworkers" it took some doing but I managed to remap all the keys. I just feel sorry for the poor fellow that uses this computer if I ever leave :)

Another cool utility for emacs users is the XKeymacs program that sets emacs keybindings for stuff like moving the cursor and copy and paste (kill and yank).

Django, lighttpd and FCGI, take two

2005-07-27T19:13:06Z

In case no one reads comments on my site, Hugo posted a comment about his second way of getting Django to work with lighttpd and fcgi (you mean people actually read my blog?) You can find that article here.

Example Django Ajax app

2005-07-27T00:37:01Z

I created an example django ajax application in the svn repos. If anyone is interested, the README.html should help you get that up and running.

Still on the agenda is template tags to ease the use of django ajax for those Javascript impaired

Running Django with FCGI and lighttpd

2005-07-26T18:49:52Z

Hugo's house of weblog horror has an article on how to get Django running with fcgi and lighttp. There are many in the #django IRC that are waiting for someone to do this so I'm linking him in case they missed his article.

Django RSS

2005-07-26T03:19:25Z

From what I can tell, turning on RSS is very easy. Basically you setup a urlpattern like in http://code.djangoproject.com/svn/djangoproject.com/django_website/settings/urls/main.py and then make a module that is the same name as your settings module that ends in _rss, like so: http://code.djangoproject.com/svn/djangoproject.com/django_website/settings/main_rss.py

Happy hacking!

django_ajax update

2005-07-26T02:15:26Z

From the reaction of the folks in the django irc channel, it seems that my Django_Ajax library is a little advanced, everyone wants the power of Ajax without having to write Javascript. I made the false assumption that anyone messing with Ajax would know Javascript.

After taking a look at what Ruby on Rails does with their ajax api, I'm going to be writting, with the help of some of the folks in #django, a templatetag library to rival ROR's ajax helper functions. It should shape up nicely this coming week.

Javascript Name Clobbering

2005-07-22T00:58:46Z

I was working with two different javascript libraries today and ran into the issue that brought about namespaces in other languages.

Jsval has a class called Field. I wanted to use prototype.js and openrico so... guess what, prototype.js, has the class Field also... So, they stepped on each other toes. I renamed all the Field references is Jsval (because prototype.js is a dependancy for many libraries) and then wrote this http://en.wikipedia.org/wiki/Javascript_Namespaces out of frustration. Maybe someone will listen to me and use javascript namespaces.

Pay no attention to the man behind the curtain..

2005-07-20T19:04:16Z

If anyone saw the django_ajax article that I accidentally posted to my blog. Pay no attention to it. It does not exist (cue Jedi Mind Trick)

This is exactly what a blog should not be used for...

2005-07-20T18:54:33Z

Ack,... I ran out of sugar at work today. It's black coffee for me this morning.

Django, Django, Django...

2005-07-18T01:28:36Z

This framework is so freakin' awesome.... They just enough documentation on the site that you can actually get things running without much effort.. I have to say, this is the coolest thing to come out in a long time. You have a object-relational mapping system, obsessive SEO support, a templating system and the coolest thing ever, automatic administive interface! Oh not to mention it's freakin python! The best scripting language ever. Go to here and check it out. NOW!

<lang_en>Silent Bob uses Wordpress.</lang_en><lang_eo>Silent Bob uzas "Wordpress"-on</lang_eo>

2005-07-15T08:06:10Z

Kevin Smith (aka Silent Bob) uses Wordpress for his blog, My Boring Ass Life, tre bone! Kevin Smith (aka Silent Bob) uzas "Wordpress"-on por lia blogo, My Boring Ass Life, tre bone!

<lang_en>Windows is easier than linux?</lang_en><lang_eo>Vindozo estas pli facila pli Linukso?</lang

2005-07-13T23:50:59Z

Who said that Windows is easier than Linux? Today I tried to plug in my SD Card reader into my work computer. At first it didn't show up. I had to remove my network drive mappings in order for them to show up. How stupid.

When I plug in the card reader into my linux box, it appears instantly. I didn't have to do anything. To think, I was afraid that it wasn't going to work in Linux. Shesh

Kiu diris ke Vindozo estas pli facila pli Linukso? Hodiaux mi penis uzi mia SD memorkarton en mia komputilo labora. Komence, gxi ne laboris. Mi havis fortiri mia retodurdiskojn por la memorkarto labori. Tre muta.

Kiam mi uzas la memorkarton kun Linukso, gxi laboras senprokraste. Mi ne havas fari neniajxo. Pensi, mi timis ke gxi ne estis ironta labori en Linukso. sxisx.

<lang_en>Happy Birthday to me</lang_en><lang_eo>Felicxan Naskigxtagon por min</lang_eo>

2005-07-13T21:01:14Z

Happy Birthday to me. Happy Unbirthday to all of you (except Jordon, Felicxan NaskiÄ?tagon por lin). Felicxan Naskigxtagon por min. Felicxan MalNaskigxtagon por vi.

GPG Public Key

2005-07-13T04:16:39Z

A added my GPG Public Key to the bottom of this page if anyone wants to send encrypted/signed messages to me.

The Uncyclopedia

2005-07-13T02:23:58Z

Fans of Wikipedia will love this site, Uncyclopedia. It is like a bizzaro-pedia, where everything on the wiki is wrong. It's a delightful waste of time.

Finally a decent Nintendo DS game

2005-07-10T10:57:37Z

Well I traded in Batman Begins because I beat it and there wasn't much more to do with it. I looked around the Gamestop and I didn't really see any XBox games that really interested me. I want to tryout Paper Mario for the Gamecube but it was $10 more than the store credit I had for Batman.

While I was looking for a game, Gina was playing a game on the Nintendo DS demos. I couldn't really find anything. I went over to see what Gina was playing and it was "Kirby Canvas Curse". I thought, "Oh great, another 'Yoshi's Touch and Go' (A really bad game)". After that we got the clerk to help us find something.

The clerk was helpful in helping me find some XBox games. He recomended some first person shooters but frankly I've been playing FPSes since the first Wolfenstein and I'm a little bored with that genre. Although XIII was an awesome FPS. I recomend grabbing that if you haven't played it yet. He then recommended one of the Mech series games and Buffy the Vampire Slayer. I was willing to finally give a Mech game a try since they have some hardcore fans and must be a something to that. The clerk recommended Buffy, which is a third person action game, he said that it was surpisingly a good game and he doesn't care for Buffy. So I said what the hell (Maybe there is a Willow naked cheat j/k).

While we were waiting in line, I thought I'd try out Kirby's Canvas Curse to see if it was anything good. I played it for about two minutes and then I put the other games back and grabbed a copy of Kirby.

Kirby's Canvas Curse (KCC) is a very cool game. It's just like the other Kirby games except the you don't control Kirby with the keypads, you direct them with the stylus. At first it's seems like a really annoying way the control the character, especially if you played Yoshi's Touch and Go. However, this game got it right.

Gameplay is like this. Kirby is a ball and he continuously moving. You can draw lines and he will roll on the line in the direction that you draw the line. It sounds simply but its really isn't, you have to maneuver Kirby around all kinds of obsticles, block laser beems with you lines, block those pesky cannon balls, etc. It makes it extremly entertaining. Perhaps Yoshi's Touch and Go might of been this much fun if it was longer and more inventive, however it was so short that it was really nothing more than a puzzle game masked as a platformer.

Oh course the usual Kirby abilities are in this game. It wouldn't be Kirby without the cool ass power-ups. The Boss Levels aren't like the other Kirby games where you just tediously battle it out with them. They take the form of mini-games (A NDS specialty).

There are three bosses that I'm aware of. The first is that annoying Cloud guy that shoots lightning. He's a little more tame in this game, however the mini game is the fun part. It's a Kirby version of Breakout. instead of a paddle on the bottom, you draw you paddle to direct Kirby. The object of the minigame is to clear each board of the enemies by directing Kirby into them.

The second is a race against the King Penguin guy, I can't remember his name. You steer kirby by dragging his cart with the stylus and if you steer him into food, his gets a boost. I don't really care for this one.

The third boss is the Painter guy. What you have to do is recreate the drawings he does before kirby walks from the left side of the screen to the left.

In the end, I think that this game is a great adaptation of the 2d Kirby series to the NDS.

Some Buddhist Links

2005-07-10T08:39:21Z

I was writing a friend of mine about some good Buddhist links and I thought I might share them with whoever is reading this Blog.

http://www.tubtenkunga.org This is the website of the Buddhist Center that I goto.

http://www.thubtenchodron.org/ This is the website of a western nun that writes a number of good books. Her site has a lot of resources

http://www.lamayeshe.org/ This site has many free ebooks that you can download.

http://www.lamrim.com/ This website has a lot of great Buddhis teachings by some the great Tibetan teachers and Western monks and nuns. Some good Western teachers there are: Ven Sarah Thresher, Ven Thubten Chodron, and Ven Robina Courtin (I really like her). Generally the people who look Western would be the easiest to understand because they put thinks in a Western context.

Batman Begins The Video Game

2005-07-10T00:08:39Z

We saw Batman Begins in the theaters the day it came out and we both thought it was the best batman movie ever. I don't know if my wife thinks the movie was good because of the quality of the fact that her boyfriend Christian Bale was batman. My only beef with the movie is the Mr Bale's "Batman Voice" sounded really goofy.

Anyhow, while I traded in some games at gamestop yesturday and saw "Batman Begins". I suspended my ban on movie games and picked it up. I must say it was a good choice. The game is amazing. It was fun to play and the graphics were simply amazing. The game wasn't hard but it wasn't to easy that it was boring. I thought it was just plain fun.

The game plays off the concept of fear. You have a fear meter that you can increase by doing scary things. For instance you can hang from a light fixture and throw one of those HF emitters that summon bats to scare the snot out of your enemies.

I beat it in about 4 hours or so. It's is not a long game, so I'd recommend that you rent it. Gina even enjoyed watching me play it.

So, if you are in the mood for a fun game, pick up this one.

I have bronchitis

2005-07-07T23:10:08Z

Last night the wife and I had to go to the hospital because we both got sick. We went to Memorial Pembroke(don't go there) and sat for four hours. We ended up leaving and going to Memorial West(go there). We were admitted immediatly.

At about 6am the doctor saw me. He diagnosed me with bronchitis. So, I'm stuck at home again.

One reason I love Dvorak

2005-07-07T02:31:39Z

One reason why I love the Dvorak keyboard is that I can be sick as a dog, sit in my chair like Steven Hawkings and still be able to type without much effort.

Kids... Don't try this at home...

2005-07-05T08:54:33Z

Well, the 4th of July for us was eventful. We went to a bbq at a friends house in the afternoon. We swam, got crispy in the sun. It was fun.

Our Arsenal

After the bbq, we went to my wife's parents house to light off fireworks. We had the regular arsenal of fireworks. Roman Candles, Bottle Rockets, Blackcats(firecrackers). We were in for a good time.

We started off tame lighting bottle rockets from the bottle. When we got bored of that we lit them while holding the bottle. When we got bored of that we get a pipe and aimed them at trees.

When we got bored of that we brought out the roman candles. First we shot them up in the air. Then we shot them at a soda can. Next about 10 packs of firecrackers. Then we played Roman Candle Harry Potter....

The point of Roman Candle Harry Potter is to act like wizards and shoot 2d8 fireballs at each other.

Tell me, would you let this man shoot fireballs at you:

Cardinal Sin.

At first it was fun. I shot one over his head. He shot one over mine. He shot one to the left of me. I shot one to the left of him... Just not that far left of him. Then he shot one very close to me. I dodged. Then another, and another, oh the horror.... Then one hit my stomach.

This is wait happens when you play Roman Candle Harry Potter:

Burn in my shirt, about the size of a quater.

Kids-- Don't play Roman Candle Harry Potter.

"Oh Crap, my steering wheel fell off"

2005-07-03T21:10:27Z

Chris

While I was in the middle of typing my previous post I heard a knock on my door. When I answered the door, my friend Chris who lives across the street from me asked, "Do you have a tap and die set?" For those that don't know, A tap and die set is for making new threads on bolts and nuts when they are stripped. Of course I didn't have one so I told him, "No, I don't have one. Why what is stripped?". He told me that his steering wheel had fallen off his car and the stud that holds the wheel on is completely stripped off. I went over to look at the stud and it was completely stripped, there were no threads on it at all. I told him, sure lets go to the store.

When I was getting ready, I remembered that my dad had a tap and die set in his garage. I called my dad an asked if I could borrow the set. He was reluctant and suggested what we could do. Most of his suggestion required getting the car to my dad's house to work on the car. We had no idea how to get it over there aside from towing it.

Chris' Shirt

I had to go to the bank first so we went there first. On the way to the bank we were joking around trying to think of ways to hold the wheel on the stud to get it to my dads. I suggested duct tape. He suggested welding. I told him, you know you could most likely hold the wheel on the stud to get it there. The crazy bastard thought that was a great idea.

Our Trip

The problem is that my parents live on the other side of town about 9 miles from where we live. 9 miles isn't far when you have a working steering wheel but it's very long without one. So, we had to break the news to his girlfriend that this might be the last day she will every see him. She started freaking out, "You're going to die", "You're crazy", "You better use your seat belt!". Then we were off.

I told him that we need to drive around the block to make sure he can drive it. He takes off and I fallow behind him. We go around the corner and he starts to swerve. He lost his steering wheel! He recovers and continues around the block. I pull up beside him and he says, "Yeah, I'm comfortable driving it.", crazy bastard...

We goes off, I drive infront of him and he's behind me. We drive down our main street and he's ok. That's good, but we are not on I95 yet. We get on I95. I'm constantly looking in my rear view every two minutes to make sure he didn't lose it and drive off the road. A few times he swerves a little and I think he's a gonner but he recovers and we make it to my parents.

He gets the car into my parents garage and about twenty minutes later, he's done.

Deep Impact

2005-07-02T08:58:20Z

This is amazing, Nasa is smashing one of their space craft into the comet Tempel 1 in hopes to see what is inside it.

I heard this on NPR this afternoon. They were interviewing a reporter from Sky and Telescope. Apparently if you live west of the Mississippi River you will be able to see the impact on July 3rd at around 10:53pm with your naked eye.

It's a shame I'm on the east coast :'( the comet will be under the horizon when impact is to occur. It would of been a damn good excuse to dust off my ol' Meade ETX-70.

All is not lost for us eastside geeks, however. The reporter at Sky and Telescope said that we will be able to see the aftermath of the impact on the Forth of July... yeah, ok, sure bud...

All sarcasm aside, the Article at Sky and Telescope gives westsiders directions on how to locate the comet. For the eastside? There are also links to several webcasts of the event at the end of the article.

Keep Looking up!

My Host

2005-07-02T00:08:53Z

In my previous post, I told you about my family's website. I forgot to mention the hosting company that I'm using. The hosting company is Site 5.

A coworker of mine told me about them, I've used them for a couple days now and I am very impressed with the sophistication of the control panel they are using. If you want web site hosting, their features are amazing for the price. Check them out.

Just another blog

2005-07-01T23:18:37Z

Just another blog, nothing to see here...

Ok, well this is my adventure into the blogosphere. I'm not sure where I'm going with this blog, but after creating my new family's website I was given unlimited subdomains and the ability to use Fantastico so I had just had to create a personal blog.

My friend Jordon at The World of Stuff, a fellow Esperantist (Saluton Gxordono!) and Dvorak keyboad user, uses the blog software Word Press. I clicked a button in Fantasico and a moment later, I had a blog!

Generally the blog will be my forum for showcasing stuff that I find interesting. Stuff about My wife, Computers, Linux, Programming, Buddhism, Jazz, Chess, Punk/Ska (alright, I thinks that's enough keywords to make the search engine robots happy :) )

Well I hope you will enjoy my blog. I hope I enjoy it too :)