So you have a site in production and someone called you up that there is a bug
on the site. Your template designer is trying to make a change to a template
and the change is isn't showing up on the site. You make the change on you
development server and the change takes effect. What is going wrong?
You can do a number of things:
- You can call your template designer an idiot and figure it's a problem between the chair and the keyboard
- You can spend hours trying to figure out how the development server and the production server differ and bang
your head against the closest hard object until you find it.
- You can be a bad boy or girl and start changing code on the production server
- Or, you can use PDB and django.test.client to keep from spending all your hard earned money on hard liquor.
The url in question is the homepage. We have a simple view, all it does is
load a template.
1 from django.http import HttpResponse
2 from django.template import loader, Context
3
4 def index(request):
5 t = loader.get_template("index.html")
6 c = Context({})
7 return HttpResponse(t.render(c))
First we'll look at the index.html template that loads the page:
<html>
<head><title>Hi</title></head>
<body>
<h1>Welcome to my site, click on a link below to go somewhere</h1>
{% include "includes/navigation.html" %}
</body>
</html>
Something is wrong with the include tag, it's not loading the correct template.
Taking a quick look in they site templates shows that the template exists.
What's going on!
$ ls pdb_example_templates/includes/
navigation.html navigation.HTML~
Ok, there's something "wrong" with the {% include %} tag. A quick glance at
django/template/loader_tags.py shows that the ContantIncludeNode loads the
template on line 102. We're going to drill down the stack to find where the
template is actually loaded. Line 102 has get_template, get_template is
defined in django.template.loader. Open that up and find the definition of
get_template. We see that get_template calls find_template_source. That's
up on the top of django.template.loader on line 47. We know that there
is something funky with the loading of the template so we have our starting
point for PDB.
Let's SSH into the production server and start playing. First, what we need
to do is create a simple little script to call the view. Let's call it wtf.py:
def main():
from django.test.client import Client
c = Client()
response = c.get("/")
print response.content
Since we're using ./manage.py shell for this, we can't just do ol'
if __name__ == '__main__' trick to run the script so we create a main()
function to call from interactive mode.
$ ~/src/pdb_example/manage.py shell
Python 2.5.1 (r251:54863, Mar 7 2008, 03:39:23)
Type "copyright", "credits" or "license" for more information.
IPython 0.8.1 -- An enhanced Interactive Python.
? -> Introduction to IPython's features.
%magic -> Information about IPython's 'magic' % functions.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.
In [1]: import wtf
In [2]: wtf.main()
<html>
<head><title>Hi</title></head>
</html>
<body>
<h1>Welcome to my site, click on a link below to go somewhere</h1>
<ul>
<li><a href="/blog/">My Blog</a></li>
<li><a href="http://www.flickr.com/photos/ericmoritz/">My Photos</a></li>
</ul>
</body>
Ok our little script calls up the correct view and generates the incorrect
content, good. Next step, get PDB into the mix.
There are two ways of doing this, the python way and the ipython way. I'll
do it the python way first for all you dummies that aren't using ipython yet
(what are you waiting for?). Edit wtf.py and add import pdb; pdb.set_trace()
right before the c.get("/") line.
def main():
from django.test.client import Client
c = Client()
import pdb; pdb.set_trace()
response = c.get("/")
print response.content
This will drop you into pdb before the view is called so you can set a
breakpoint.
The find_template_source function does some module level caching stuff in
the beginning. The guts that we want is near the bottom on line 67.
So let's fire up manage.py shell and set a breakpoint.
$ ~/src/pdb_example/manage.py shell
Python 2.5.1 (r251:54863, Mar 7 2008, 03:39:23)
Type "copyright", "credits" or "license" for more information.
IPython 0.8.1 -- An enhanced Interactive Python.
? -> Introduction to IPython's features.
%magic -> Information about IPython's 'magic' % functions.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.
In [1]: import wtf
In [2]: wtf.main()
> /home/eric/wtf.py(5)main()
-> response = c.get("/")
(Pdb) break django/template/loader.py:67
Breakpoint 1 at /opt/django/versions/django-1.0/django/template/loader.py:67
(Pdb) cont
> /opt/django/versions/django-1.0/django/template/loader.py(67)find_template_source()
-> for loader in template_source_loaders:
(Pdb) list
62 import warnings
63 warnings.warn("Your TEMPLATE_LOADERS setting includes %r, but your Python installation doesn't support that type of template loading. Consider removing that line from TEMPLATE_LOADERS." % path)
64 else:
65 loaders.append(func)
66 template_source_loaders = tuple(loaders)
67 B-> for loader in template_source_loaders:
68 try:
69 source, display_name = loader(name, dirs)
70 return (source, make_origin(display_name, loader, name, dirs))
71 except TemplateDoesNotExist:
72 pass
(Pdb) name
Out[2]: 'index.html'
Whoops, we're going to go through the load of every template and probably start
drinking. Let's make our breakpoint more specific.
(Pdb) clear 1
Deleted breakpoint 1
(Pdb) break django/template/loader.py:67, name == 'includes/navigation.html'
Breakpoint 2 at /opt/django/versions/django-1.0/django/template/loader.py:67
Ok, now our breakpoint will only break if the name of the template being loaded
is called includes/navigation.html. Let's continue.
(Pdb) continue
> /opt/django/versions/django-1.0/django/template/loader.py(67)find_template_source()
-> for loader in template_source_loaders:
(Pdb) name
Out[2]: u'includes/navigation.html'
(Pdb) list
62 import warnings
63 warnings.warn("Your TEMPLATE_LOADERS setting includes %r, but your Python installation doesn't support that type of template loading. Consider removing that line from TEMPLATE_LOADERS." % path)
64 else:
65 loaders.append(func)
66 template_source_loaders = tuple(loaders)
67 B-> for loader in template_source_loaders:
68 try:
69 source, display_name = loader(name, dirs)
70 return (source, make_origin(display_name, loader, name, dirs))
71 except TemplateDoesNotExist:
72 pass
Cool, we've successfully broke at the correct line with the correct template
name. Let's walk through the code.
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loader.py(68)find_template_source()
-> try:
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loader.py(69)find_template_source()
-> source, display_name = loader(name, dirs)
Now we're positioned right before the loader is called. Now we'll use the
step command to step into that function.
(Pdb) step
--Call--
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(45)load_template_source()
-> def load_template_source(template_name, template_dirs=None):
(Pdb) list
40 yield safe_join(template_dir, template_name)
41 except ValueError:
42 # The joined path was located outside of template_dir.
43 pass
44
45 -> def load_template_source(template_name, template_dirs=None):
46 for filepath in get_template_sources(template_name, template_dirs):
47 try:
48 return (open(filepath).read().decode(settings.FILE_CHARSET), filepath)
49 except IOError:
50 pass
We're now at the start of the load_template_source function. Let's walk
through this function to see if it finds a template.
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(46)load_template_source()
-> for filepath in get_template_sources(template_name, template_dirs):
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(47)load_template_source()
-> try:
(Pdb) filepath
Out[2]: u'/home/eric/src/pdb_example/example/templates/includes/navigation.html'
Ok, the first template it's trying to load is the template in our app's
templates folder. This shouldn't work, so let's continue to verify that our
assumption is correct.
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(48)load_template_source()
-> return (open(filepath).read().decode(settings.FILE_CHARSET), filepath)
(Pdb) next
--Return--
> /opt/django/versions/django-1.0/django/template/loaders/app_directories.py(48)load_template_source()->(u'<ul>\n...n</ul>\n', u'/home/...ion.html')
-> return (open(filepath).read().decode(settings.FILE_CHARSET), filepath)
(Pdb) next
> /opt/django/versions/django-1.0/django/template/loader.py(70)find_template_source()
-> return (source, make_origin(display_name, loader, name, dirs))
(Pdb) list
65 loaders.append(func)
66 template_source_loaders = tuple(loaders)
67 B for loader in template_source_loaders:
68 try:
69 source, display_name = loader(name, dirs)
70 -> return (source, make_origin(display_name, loader, name, dirs))
71 except TemplateDoesNotExist:
72 pass
73 raise TemplateDoesNotExist, name
74
75 def get_template(template_name):
(Pdb) source
Out[2]: u'<ul>\n <li><a href="/blog/">My Blog</a></li>\n <li><a href="http://www.flickr.com/photos/ericmoritz/">My Photos</a></li>\n</ul>\n'
(Pdb)
We've walked through the loader's function and have returned with a
success. However, this is the app_directories loader, the site should of used
the filesystem loader first because the template designer has his own
set of templates that are outside of the django project code.
Let's look at our TEMPLATE_LOADERS setting to see what's up
# List of callables that know how to import templates from various sources.
TEMPLATE_LOADERS = (
'django.template.loaders.app_directories.load_template_source',
'django.template.loaders.filesystem.load_template_source',
)
Doh! Our loaders are backward. Who did that?
$ svn blame settings.py
42 newman TEMPLATE_LOADERS = (
42 newman 'django.template.loaders.app_directories.load_template_source',
42 newman 'django.template.loaders.filesystem.load_template_source',
42 newman )
Newman! Hang on, I have to go smack my other programmer in the back of the
head...
Now to use pdb with ipython with all the benefits of ipython
(colors, tab completion, etc). There is a module here
that will help you out. Just download that script as ipdb.py in the same
folder as wtf.py (or somewhere in sys.path)
$ curl http://trac.gotcha.python-hosting.com/file/bubblenet/pythoncode/ipdb/ipdb/__init__.py?format=txt > ipdb.py
Then change your code like this:
def main():
from django.test.client import Client
c = Client()
import ipdb; ipdb.set_trace()
response = c.get("/")
print response.content
So, there you have it. How to use pdb and django.test.client to solve bugs
in production without editing the code and potentially disrupting service.
