Saturday, March 30, 2013

Check python package version in program

I have always wanted to check the version of the python packages which I am using. Today I found a way to find that.

As stated here,

http://stackoverflow.com/questions/710609/checking-python-module-version-at-runtime
>>> pkg_resources.get_distribution('web.py')
web.py 0.37 (/usr/local/lib/python2.7/dist-packages/web.py-0.37-py2.7.egg)

To get just the version, you can do

>>> pkg_resources.get_distribution('web.py').version
'0.37'

Thursday, March 28, 2013

Deploying web.py application in Apache with mod_wsgi

Today, after a lot of struggle, I managed to deploy my web.py application on Apache with mod_wsgi module. There is no step at which I didn't face any problem. It took me about 8 hours to figure things out and get things working. Lets see, one by one. This post assumes that the reader knows the basics of installing a software in linux.

Try all these at your own risk, if something breaks I am not responsible.


1) Installing Apache

This was pretty simple, I just had to follow the steps mentioned here http://httpd.apache.org/docs/current/install.html. Including the apache's bin directory in PATH variable would be good.

2) Installing Python

This is also pretty straight-forward. Download http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz and extract it. Just do configure, make and make install. Only thing is, configure with shared libraries enabled, like this

 ./configure --enable-shared

Because the mod_wsgi module which we install later is dependent on the python's shared binary objects. Once the installation is completed, do not forget to include the python's lib directory in the LD_LIBRARY_PATH variable and python's bin directory in PATH variable. I would suggest including them in the login/profile script.

3) Installing mod_wsgi

WSGI stands for Web Server Gateway Interface. web.py provides the libraries necessary for this. People say, this makes life easier (atleast not for me :( ). Installation follows the same configure, make and make install drill. Download it from http://code.google.com/p/modwsgi/ and extract. (Make sure that LD_LIBRARY_PATH has the python's lib directory as well).

Once the installation finishes, find the mod_wsgi.so file using "find . -name mod_wsgi.so" (execute this from the directory in which compilation was done) and then copy it to <Apache installation directory>/modules/ directory.

4) Configuring mod_wsgi

Now that Apache has got wsgi in its body, its still not activated and configured yet. There is a good documentation for integrating mod_wsgi with Apache here http://code.google.com/p/modwsgi/wiki/IntegrationWithWebPy (Though it didn't work for me, I got to know a lot from this)
. I am going to share the things which worked for me.

First, enabling mod_wsgi. Find the list of lines beginning with LoadModule in httpd.conf and insert this line there.
LoadModule wsgi_module modules/mod_wsgi.so

This is how it looks in my httpd.conf
...
LoadModule wsgi_module modules/mod_wsgi.so
LoadModule authn_file_module modules/mod_authn_file.so
#LoadModule authn_dbm_module modules/mod_authn_dbm.so
#LoadModule authn_anon_module modules/mod_authn_anon.so
...
In that file, whatever line begins with a hash symbol (#) is a commented line. This will let Apache know that, its armed with wsgi module now. Whenever I restart my apache, I get this line in the error_log file. 
AH00489: Apache/2.4.3 (Unix) mod_wsgi/3.4 Python/2.7.3 configured -- resuming normal operations
next, configuring application's URL. Find the tag <IfModule alias_module>...</IfModule> and insert the following lines in that tag.
WSGIScriptAlias /testapp "/home/myhome/testapp/app.py"
Alias /testapp/static "
/home/myhome/testapp/static/"
You can read about what WSGIScriptAlias means here http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIScriptAlias. Basically we map the user's request to the files which serve them. For example, when a request is made for http://localhost/testapp, Apache will execute /home/myhome/testapp/app.py and return the result. /home/myhome/testapp/is the directory where the web.py application is kept.

Now that we have instructed Apache to treat the requests to /testapp with app.py which is a wsgi script, we need to define permissions and options for the physical directory in which it is stored.
<Directory "/home/myhome/testapp/">
   Options +ExecCGI +FollowSymLinks +Indexes
   Order allow,deny
   Allow from all
   Require all granted
   AddHandler wsgi-script .py
</Directory>
You can read about each and every option mentioned, here http://httpd.apache.org/docs/2.2/mod/core.html#directory. With the AddHandler wsgi-script .py line, we tell Apache that, treat only the .py files as wsgi scripts. I spent an hour at this point. Instead of AddHandler line, I had "SetHandler wsgi-scirpt". That made Apache to treat even image files, css, java script files as wsgi scripts. So, Apache tried to compile each and every file and throwing internal error.

That's it. We are pretty much done with the Apache configuration. :) But that's not the end of it. 

The sample code.py file given here http://webpy.org/cookbook/mod_wsgi-apache would work perfectly. It imports nothing, no templates and so, no confusions. But practical applications might not be like that.

The code in app.py file, with which I tested my application was like this
import web, urls
app = web.application(urls.urls, globals())
app.run()
but when I deployed with wsgi enabled, it became this
import web, os, sys
 
root = os.path.join(os.path.dirname(__file__)+"/")
sys.path.insert(0, root)
templates = os.path.join(os.path.dirname(__file__)+"/templates/")
sys.path.insert(1, templates)
os.chdir(root)
 
import urls
 
app = web.application(urls.urls, globals())
application = app.wsgifunc()
Basically the application is unable to understand no directory even the current directory. So everything has to be added to the sys.path manually. Notice that, without these path changes to sys, it was not even able to recognize the urls module which was in the same directory as the app.py. And whenever templates are used, we cannot use relative locations like this
render = web.template.render('templates/')
We need to get the physical location of the templates directory like this
templates = os.path.join(os.path.dirname(__file__)+"/templates/")
and use it in the render like this
render = web.template.render(templates)
That's all, I have to say. Good luck :)


Monday, March 25, 2013

Installing python-ldap in cygwin

Installing python-ldap in cygwin took a while for me. When I tried to install with

easy_install python-ldap

I got this error

Searching for python-ldap
Reading http://pypi.python.org/simple/python-ldap/
Reading http://www.python-ldap.org/
Best match: python-ldap 2.4.10
Downloading http://pypi.python.org/packages/source/p/python-ldap/python-ldap-2.4.10.tar.gz#md5=a15827ca13c90e9101e5e9405c1d83be
Processing python-ldap-2.4.10.tar.gz
Running python-ldap-2.4.10/setup.py -q bdist_egg --dist-dir /tmp/easy_install-JXhRAd/python-ldap-2.4.10/egg-dist-tmp-1wRZxP
defines: HAVE_SASL HAVE_TLS HAVE_LIBLDAP_R
extra_compile_args:
extra_objects:
include_dirs: /opt/openldap-RE24/include /usr/include/sasl /usr/include
library_dirs: /opt/openldap-RE24/lib /usr/lib
libs: ldap_r
file Lib/ldap.py (for module ldap) not found
file Lib/ldap/controls.py (for module ldap.controls) not found
file Lib/ldap/extop.py (for module ldap.extop) not found
file Lib/ldap/schema.py (for module ldap.schema) not found
warning: no files found matching 'Makefile'
warning: no files found matching 'Modules/LICENSE'
file Lib/ldap.py (for module ldap) not found
file Lib/ldap/controls.py (for module ldap.controls) not found
file Lib/ldap/extop.py (for module ldap.extop) not found
file Lib/ldap/schema.py (for module ldap.schema) not found
file Lib/ldap.py (for module ldap) not found
file Lib/ldap/controls.py (for module ldap.controls) not found
file Lib/ldap/extop.py (for module ldap.extop) not found
file Lib/ldap/schema.py (for module ldap.schema) not found
build/temp.cygwin-1.7.17-i686-2.7/Modules/LDAPObject.o: In function `l_ldap_result4':
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/LDAPObject.c:988: undefined reference to `_ber_bvfree'
build/temp.cygwin-1.7.17-i686-2.7/Modules/ldapcontrol.o: In function `encode_rfc3876':
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:216: undefined reference to `_ber_alloc_t'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:227: undefined reference to `_ber_flatten'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:237: undefined reference to `_ber_free'
build/temp.cygwin-1.7.17-i686-2.7/Modules/ldapcontrol.o: In function `decode_rfc2696':
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:315: undefined reference to `_ber_init'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:320: undefined reference to `_ber_scanf'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:330: undefined reference to `_ber_free'
build/temp.cygwin-1.7.17-i686-2.7/Modules/ldapcontrol.o: In function `encode_rfc2696':
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:258: undefined reference to `_ber_alloc_t'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:263: undefined reference to `_ber_printf'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:272: undefined reference to `_ber_printf'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:278: undefined reference to `_ber_printf'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:284: undefined reference to `_ber_flatten'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:293: undefined reference to `_ber_free'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/ldapcontrol.c:270: undefined reference to `_ber_printf'
build/temp.cygwin-1.7.17-i686-2.7/Modules/message.o: In function `LDAPmessage_to_python':
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/message.c:160: undefined reference to `_ber_free'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/message.c:199: undefined reference to `_ber_memvfree'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/message.c:243: undefined reference to `_ber_bvfree'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/message.c:134: undefined reference to `_ber_free'
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/message.c:237: undefined reference to `_ber_bvfree'
build/temp.cygwin-1.7.17-i686-2.7/Modules/options.o: In function `LDAP_set_option':
/tmp/easy_install-JXhRAd/python-ldap-2.4.10/Modules/options.c:70: undefined reference to `_ber_pvt_opt_on'
collect2: ld returned 1 exit status
error: Setup script exited with error: command 'gcc' failed with exit status 1


Then I just tried to install a version of python-ldap prior to 2.4.10, like this

easy_install -N -S /usr/lib/python2.7/site-packages  -d /usr/lib/python2.7/site-packages -s /usr/local/bin  python-ldap==2.4.3

Voila. It worked fine :)

Installing node.js in Linux

I tested these steps in RHEL 5.4 and Ubuntu 12.10.

  1. wget http://nodejs.org/dist/v0.10.1/node-v0.10.1.tar.gz
  2. tar -xvf node-v0.10.1.tar.gz
  3. cd node-v0.10.1
  4. ./configure --prefix=<a directory to which you have write and execute access>
  5. make
  6. make install
  7. Create a file, say Test.js and add the following lines to it
    var http = require('http');

    http.createServer(function (request, response) {
      response.writeHead(200, {'Content-Type': 'text/plain'});
      response.end('Hello World\n');
    }).listen(8124);

    console.log('Server running at http://127.0.0.1:8124/');
  8. node Test.js
  9. Open a browser and visit http://127.0.0.1:8124/. If all went fine, you should see Hello World in the browser.
It is as simple as that. I did not hit any road block at all. They all went fine.